Model defense method, device, equipment and computer-readable storage medium

Abstract

The present invention discloses a model defense method, apparatus, device, and computer-readable storage medium. The method comprises: upon receiving an image to be predicted, determining a target template image set based on the image to be predicted; performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set; and inputting the input image set into a deep learning model to obtain a prediction result. The present invention determines a target template image set based on the image to be predicted, performs feature extraction, quantization, and preprocessing on the target template image set and the image to be predicted, determines an input image set, and inputs the input image set into a deep learning model to obtain a prediction result. This method balances the effectiveness of model defense with the prediction effect of the deep learning model, thereby improving the practicality of model defense.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a model defense method, apparatus, device, and computer-readable storage medium. Background Art

[0002] Deep learning models have excellent performance in computer vision and other fields. However, their robustness is easily affected by noise, so model defense is needed. Existing model defenses destroy the structural information of the input image in order to eliminate the influence of input noise, resulting in poor image quality input to the model. The model's prediction effect is therefore greatly reduced, resulting in the low practicality of existing model defenses. Therefore, how to improve the practicality of model defenses is an urgent problem that needs to be solved. Summary of the Invention

[0003] The main purpose of the present invention is to propose a model defense method, device, equipment and computer-readable storage medium, aiming to solve the problem of how to improve the practicality of model defense.

[0004] To achieve the above object, the present invention provides a model defense method, which includes the following steps:

[0005] When receiving an image to be predicted, determining a target template image set according to the image to be predicted;

[0006] Performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set;

[0007] The input image set is input into a deep learning model to obtain a prediction result.

[0008] Preferably, the step of determining a target template image set according to the image to be predicted includes:

[0009] Fusing each template image in the template image library with the image to be predicted to obtain a fused image set;

[0010] Inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set;

[0011] A target template image set is determined in the template image library according to the first vector and the second vector set.

[0012] Preferably, the step of determining a target template image set in the template image library according to the first vector and the second vector set includes:

[0013] Calculating a vector distance set based on the first vector and the second vector set, and sorting each vector distance in the vector distance set to obtain a sorting result;

[0014] According to the sorting result, a target template image set is determined in the template image library.

[0015] Preferably, the step of performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain the input image set includes:

[0016] Pairing each target template image in the target template image set with the image to be predicted, and inputting the pairings into a feature extraction module to obtain a feature extraction result;

[0017] Inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result;

[0018] The quantization result is input into a normalization module to obtain a normalized result, and the normalized result is input into a decoding and reconstruction module for reconstruction processing to obtain an input image set.

[0019] Preferably, the step of inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result includes:

[0020] Inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module;

[0021] The dynamic quantization module obtains a quantization result based on the feature extraction result, the pixel point set and the attention weight set.

[0022] Preferably, when receiving the image to be predicted, before the step of determining the target template image set based on the image to be predicted, the model defense method includes:

[0023] Acquire a training image set, generate a target training image set based on the training image set and a template image set, perform model training based on the target training image set, and obtain a deep learning pre-model;

[0024] Acquire a target verification image set, verify the deep learning pre-model based on the target verification image set, and determine the deep learning model based on the verification result.

[0025] Preferably, the step of verifying the deep learning pre-model according to the target verification image set and determining the deep learning model according to the verification result includes:

[0026] Inputting the target verification image set into the deep learning pre-model to obtain a prediction result set, and determining the prediction accuracy rate based on the prediction result set;

[0027] The prediction accuracy is compared with the accuracy threshold to obtain a comparison result, and a deep learning model is determined based on the comparison result.

[0028] In addition, to achieve the above-mentioned purpose, the present invention further provides a model defense device, which includes:

[0029] A receiving module, configured to determine a target template image set based on the image to be predicted when the image to be predicted is received;

[0030] a determination module, configured to perform feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set;

[0031] The input module is used to input the input image set into the deep learning model to obtain a prediction result.

[0032] Preferably, the receiving module further includes a fusion module, and the fusion module is configured to:

[0033] Fusing each template image in the template image library with the image to be predicted to obtain a fused image set;

[0034] Inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set;

[0035] A target template image set is determined in the template image library according to the first vector and the second vector set.

[0036] Preferably, the fusion module further includes a sorting module, and the sorting module is used to:

[0037] Calculating a vector distance set based on the first vector and the second vector set, and sorting each vector distance in the vector distance set to obtain a sorting result;

[0038] According to the sorting result, a target template image set is determined in the template image library.

[0039] Preferably, the determining module is further configured to:

[0040] Pairing each target template image in the target template image set with the image to be predicted, and inputting the pairings into a feature extraction module to obtain a feature extraction result;

[0041] Inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result;

[0042] The quantization result is input into a normalization module to obtain a normalized result, and the normalized result is input into a decoding and reconstruction module for reconstruction processing to obtain an input image set.

[0043] Preferably, the determining module is further configured to:

[0044] Inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module;

[0045] The dynamic quantization module obtains a quantization result based on the feature extraction result, the pixel point set and the attention weight set.

[0046] Preferably, the receiving module further includes a training module, and the training module is used to:

[0047] Acquire a training image set, generate a target training image set based on the training image set and a template image set, perform model training based on the target training image set, and obtain a deep learning pre-model;

[0048] Acquire a target verification image set, verify the deep learning pre-model based on the target verification image set, and determine the deep learning model based on the verification result.

[0049] Preferably, the training module further includes a verification module, and the verification module is used to:

[0050] Inputting the target verification image set into the deep learning pre-model to obtain a prediction result set, and determining the prediction accuracy rate based on the prediction result set;

[0051] The prediction accuracy is compared with the accuracy threshold to obtain a comparison result, and a deep learning model is determined based on the comparison result.

[0052] In addition, to achieve the above-mentioned purpose, the present invention also provides a model defense device, which includes: a memory, a processor, and a model defense program stored in the memory and executable on the processor. When the model defense program is executed by the processor, the steps of the model defense method described above are implemented.

[0053] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a model defense program is stored. When the model defense program is executed by a processor, the steps of the model defense method described above are implemented.

[0054] The model defense method proposed in the present invention determines a target template image set based on the image to be predicted upon receiving the image to be predicted; performs feature extraction, quantization, and preprocessing on the target template image set and the image to be predicted to determine an input image set; and inputs the input image set into a deep learning model to obtain a prediction result. The present invention determines a target template image set based on the image to be predicted, performs feature quantization and reconstruction on the target template image set and the image to be predicted to obtain an input image set. This ensures that the input image set meets the input requirements of the deep learning model and does not affect the prediction results of the deep learning model. This balances the effectiveness of model defense with that of the deep learning model, thereby improving the practicality of model defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;

[0056] Figure 2 This is a flow chart of the first embodiment of the model defense method of the present invention;

[0057] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0060] The device in the embodiment of the present invention may be a PC or a server device.

[0061] like Figure 1As shown, the device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0063] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a model defense program.

[0064] Among them, the operating system is a program that manages and controls the portable model defense device and software resources, and supports the operation of the network communication module, user interface module, model defense program and other programs or software; the network communication module is used to manage and control the network interface 1002; the user interface module is used to manage and control the user interface 1003.

[0065] exist Figure 1 In the model defense device shown, the model defense device calls the model defense program stored in the memory 1005 through the processor 1001 and executes the operations in each embodiment of the following model defense method.

[0066] Based on the above hardware structure, an embodiment of the model defense method of the present invention is proposed.

[0067] Reference Figure 2 , Figure 2 This is a flow chart of a first embodiment of the model defense method of the present invention, which includes:

[0068] Step S10, upon receiving the image to be predicted, determining a target template image set according to the image to be predicted;

[0069] Step S20, performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set;

[0070] Step S30: input the input image set into a deep learning model to obtain a prediction result.

[0071] The model defense method of this embodiment is applied to a model defense device of a deep learning model. The model defense device can be a terminal or a PC device, which can process the input image to obtain an input image that meets the conditions. For the convenience of description, the model defense device is used as an example for description. When the model defense device receives the image to be predicted, each template image in the template image library is fused with the image to be predicted to obtain a fused image set. The image to be predicted is input into the classification model to obtain a first vector, and each fused image in the fused image set is input into the classification model to obtain a second vector set. Based on the first vector and the second vector set, a target template image set is determined in the template image library. The model defense device pairs each target template image in the target template image set with the image to be predicted, and inputs them into a feature extraction module to obtain a feature extraction result. The feature extraction result is input into an attention weight module to obtain an attention weight set, and the attention weight set and the feature extraction result are input into a dynamic quantization module to obtain a quantization result. The quantization result is input into a normalization module to obtain a normalization result, and the normalization result is input into a decoding and reconstruction module for reconstruction processing to obtain an input image set. The model defense device inputs the input image set into the deep learning model to obtain an output result. It should be noted that deep learning models have excellent performance in areas such as computer vision and natural language processing. However, their obvious disadvantage is their poor robustness, so model defense is necessary.

[0072] The model defense method of this embodiment, upon receiving an image to be predicted, determines a target template image set based on the image to be predicted; performs feature extraction, quantization, and preprocessing on the target template image set and the image to be predicted to determine an input image set; and inputs the input image set into a deep learning model to obtain a prediction result. The present invention determines a target template image set based on the image to be predicted, performs feature quantization and reconstruction on the target template image set and the image to be predicted to obtain an input image set, and inputs the input image set into a deep learning model to obtain a prediction result, so that the input image set meets the input requirements of the deep learning model and does not affect the prediction effect of the deep learning model, thereby balancing the effect of model defense with the prediction effect of the deep learning model, and improving the practicality of model defense.

[0073] The following describes each step in detail:

[0074] Step S10, upon receiving the image to be predicted, determining a target template image set according to the image to be predicted;

[0075] In this embodiment, when the model defense device receives the image to be predicted, it determines the target template image based on the image to be predicted. Preferably, the model defense device fuses all the template images in the template image library with the image to be predicted to obtain a fused image set, and calculates the first vector of the image to be predicted and the second vector set of the fused image set, and determines the target template image set in the template image library based on the first vector and the second vector set; optionally, the model defense device identifies the type of the image to be predicted, and selects a preset number of template images in the template image library as the target template image set according to the type of the image to be predicted.

[0076] Specifically, the step of determining a target template image set according to the image to be predicted includes:

[0077] Step a, fusing each template image in the template image library with the image to be predicted to obtain a fused image set;

[0078] In this step, the model defense device fuses each template image in the template image library with the image to be predicted, generating a fused image set. It should be understood that the template image library is pre-stored in the model defense device and includes multiple template images of different styles. The model defense device fuses each template image in the template image library with the image to be predicted, generating multiple images of different styles to be predicted, forming the fused image set. It should be noted that these multiple template images of different styles are used to simulate the effects of different types of noise signals on the image to be predicted.

[0079] Step b: inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set;

[0080] In this step, the model defense device inputs the image to be predicted into the classification model to obtain a first vector, and inputs each fused image in the fused image set into the classification model to obtain a second vector set. It can be understood that the first vector is used to represent the type corresponding to the image to be predicted, and the second vector set is used to represent the type corresponding to each fused image in the fused image set. It should be noted that multiple classification models are pre-stored in the model defense device, and the model defense device selects the corresponding classification model according to the received image to be predicted, and calculates the first vector and the second vector set respectively.

[0081] Step c: determining a target template image set in the template image library according to the first vector and the second vector set.

[0082] In this step, the model defense device determines the target template image set in the template image library based on the first vector and the second vector set; preferably, the model defense device calculates a vector distance set based on the first vector and the second vector set, and sorts each vector distance in the vector distance set to obtain a sorting result, and then determines the target template image set in the template image library based on the sorting result; optionally, the model defense device calculates a vector distance set based on the first vector and the second vector set, and compares each vector distance in the vector distance set with a preset distance threshold, and takes the corresponding template image in the fused image corresponding to the vector distance greater than the preset distance threshold as the target template image, thereby determining the target template image set.

[0083] Furthermore, step c includes:

[0084] Step c1, calculating a vector distance set based on the first vector and the second vector set, and sorting each vector distance in the vector distance set to obtain a sorting result;

[0085] In this step, the model defense device calculates a vector distance set based on the first vector and the second vector set, and sorts each vector distance in the vector distance set to obtain a sorting result; for example, the model defense device inputs the image to be predicted into the classification model to obtain the first vector corresponding to the image to be predicted as s0, and inputs each fused image in the fused image set into the classification model to obtain the second vector set corresponding to the fused image set as Where i represents the second vector corresponding to different fused images in the fused image set, according to The vector distance between each second vector and the first vector is calculated to obtain a vector distance set, and each vector distance in the vector distance set is sorted in descending order to obtain a sorting result.

[0086] Step c2: determining a target template image set in the template image library according to the sorting result.

[0087] In this step, the model defense device selects a preset number of template images corresponding to larger vector distances as target template images based on the sorting results of the vector distance set, and then determines the target template image set from the template image library. For example, if the preset number is 5, the model defense device selects the template images corresponding to the top five vector distances as target template images based on the sorting results from largest to smallest, and selects the corresponding template images from the template image library to determine the target template image set. It should be noted that the preset number is pre-installed in the model defense device and is generally 3, 4, or 5.

[0088] Step S20, performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set;

[0089] In this embodiment, the model defense device performs feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set. It can be understood that feature quantization includes feature extraction and quantization, and reconstruction processing includes normalization processing and decoding reconstruction processing. Correspondingly, the model defense device includes but is not limited to: feature extraction module, attention weight module, dynamic quantization module, normalization module and decoding reconstruction module.

[0090] Specifically, step S20 includes:

[0091] Step d, pairing each target template image in the target template image set with the image to be predicted, and inputting the pair into a feature extraction module to obtain a feature extraction result;

[0092] In this step, the model defense device pairs each target template image in the target template image set with the image to be predicted, and inputs them into the feature extraction module to obtain the feature extraction results; for example: assuming that the target template image set includes five target template images of different styles, each target template image is paired with the image to be predicted to obtain five pairs of images, and each pair of images is input into the feature extraction module to obtain the target template image feature map and the image to be predicted feature map corresponding to each pair of images, which are the corresponding feature extraction results.

[0093] Step e: inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result;

[0094] In this embodiment, the model defense device inputs the feature extraction results into the attention weight module to obtain an attention weight set, and inputs the attention weight set and the feature extraction results into the dynamic quantization module to obtain a quantization result; for example: the model defense device inputs the feature extraction results, that is, the target template image feature map and the to-be-predicted image feature map corresponding to each pair of images into the attention weight module, processes the target template image feature map and the to-be-predicted image feature map through the attention weight module to obtain a corresponding attention weight set, and then inputs the corresponding attention weight set and the feature extraction results into the dynamic quantization module, quantizes them through the dynamic quantization module to obtain a quantization result. It should be noted that the dynamic quantization module can effectively eliminate the influence of input noise.

[0095] Furthermore, step e includes:

[0096] Step e1, inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module;

[0097] In this step, the model defense device inputs the attention weight set and feature extraction results into the dynamic quantization module, and the dynamic quantization module determines the pixel set based on the feature extraction results. For example, the model defense device determines the image information carried by each pixel of the image to be predicted based on the feature map of the image to be predicted in the feature extraction results through the dynamic quantization module, and then determines the pixel set. It should be noted that for each pixel in the image to be predicted, the image information carried therein is different. When the image information carried by a certain pixel is unimportant information such as the background information of the image to be predicted, the attention weight corresponding to the pixel will be relatively small. When the image information carried by a certain pixel is important information such as the character information or foreground information of the image to be predicted, the attention weight corresponding to the pixel will be relatively large, so that the model defense device can concentrate its computing power to process the important image information in the image to be predicted.

[0098] Step e2: obtaining a quantization result through the dynamic quantization module according to the feature extraction result, the pixel point set and the attention weight set.

[0099] In this step, the model defense device quantizes the feature extraction results, pixel set, and attention weight set through the dynamic quantization module to obtain a quantized result. For example, the model defense device inputs the target template image feature map and the image feature map to be predicted, the pixel set, and the attention weight set corresponding to each pair of images into the dynamic quantization module, and the dynamic quantization module calculates the quantized result according to the formula:

[0100]

[0101] Calculate the corresponding quantization results, where f ij is a pixel point (i, j) in the feature map of the image to be predicted, G ij is the attention weight value corresponding to a pixel point (i, j) in the feature map of the image to be predicted.

[0102] Step f: inputting the quantization result into a normalization module to obtain a normalized result, and inputting the normalized result into a decoding and reconstruction module for reconstruction processing to obtain an input image set.

[0103] In this step, the model defense device inputs the quantization results into the normalization module to obtain a normalized result, which is then input into the decoding and reconstruction module for reconstruction processing to obtain an input image set. The model defense device inputs the quantization results into the normalization module to obtain a normalized feature map, which is then input into the decoding and reconstruction module to transfer the style of the image to be predicted, resulting in an input image set including multiple images of different styles. These images of different styles have a similar style to the corresponding target template image, but all retain the image structure information of the image to be predicted.

[0104] Step S30: input the input image set into a deep learning model to obtain a prediction result.

[0105] In this embodiment, the model defense device inputs an image set into a deep learning model, and the deep learning model processes the input image set to obtain multiple prediction results. The decision module then analyzes the multiple prediction results to determine a final output prediction result.

[0106] When the model defense device of this embodiment receives an image to be predicted, it determines a target template image set based on the image to be predicted; performs feature extraction, quantization, and preprocessing on the target template image set and the image to be predicted to determine an input image set; and inputs the input image set into a deep learning model to obtain a prediction result. The present invention determines a target template image set based on the image to be predicted, performs feature extraction, quantization, normalization, and decoding reconstruction on the target template image set and the image to be predicted, determines an input image set, and inputs the input image set into a deep learning model to obtain a prediction result. This balances the effectiveness of model defense with the prediction effectiveness of the deep learning model, thereby improving the practicality of model defense.

[0107] Furthermore, based on the first embodiment of the model defense method of the present invention, a second embodiment of the model defense method of the present invention is proposed.

[0108] The second embodiment of the model defense method differs from the first embodiment of the model defense method in that, before step S10, the method further includes:

[0109] Step g, obtaining a training image set, generating a target training image set based on the training image set and the template image set, performing model training based on the target training image set, and obtaining a deep learning pre-model;

[0110] Step h: obtain a target verification image set, verify the deep learning pre-model based on the target verification image set, and determine the deep learning model based on the verification result.

[0111] The model defense device of this embodiment obtains a training image set, generates a target training image set based on the training image set and the template image set, performs model training based on the target training image set, and obtains a deep learning pre-model; the model defense device obtains a target verification image set, verifies the deep learning pre-model based on the target verification image set, and determines the deep learning model based on the verification result. Among them, the training image set for deep learning model training is an image after style transfer, that is, the training image is fused with each template image in the template image library to obtain the final training image set. The deep learning model is trained by the training image set. The information learned by the deep learning model is the target structure information in the image, not the texture information. Similarly, the target verification image set is also an image after style transfer. In this way, it helps to improve the robustness of the deep learning model, and takes into account both the effect of model defense and the prediction effect of the deep learning model, thereby improving the practicality of model defense.

[0112] Specifically, step h includes:

[0113] Step h1: input the target verification image set into the deep learning pre-model to obtain a prediction result set, and determine the prediction accuracy based on the prediction result set;

[0114] In this step, after the model defense device obtains the deep learning pre-model through training, it inputs the target verification image set into the deep learning pre-model to obtain a prediction result set, and determines the prediction accuracy based on the prediction result set; for example: after the deep learning model predicts the target verification image set and obtains the prediction result, the model defense device counts all the prediction results and then calculates the prediction accuracy.

[0115] Step h2: compare the prediction accuracy with the accuracy threshold to obtain a comparison result, and determine the deep learning model based on the comparison result.

[0116] In this step, the model defense device compares the prediction accuracy with the accuracy threshold to obtain a comparison result. When the comparison result shows that the prediction accuracy is greater than or equal to the accuracy threshold, the deep learning pre-model is determined to be the final deep learning model. When the comparison result shows that the prediction accuracy is less than the accuracy threshold, the deep learning pre-model continues to be trained until the prediction accuracy of the deep learning pre-model is greater than or equal to the accuracy threshold, and the deep learning pre-model is determined to be the final deep learning model.

[0117] The model defense device of this embodiment acquires a training image set, generates a target training image set based on the training image set and a template image set, performs model training based on the target training image set, and obtains a deep learning pre-model. The model defense device also acquires a target verification image set, verifies the deep learning pre-model based on the target verification image set, and determines the deep learning model based on the verification results. This helps improve the robustness of the deep learning model and balances the effectiveness of model defense with the prediction effect of the deep learning model, thereby improving the practicality of model defense.

[0118] The present invention also provides a model defense device. The model defense device of the present invention comprises:

[0119] A receiving module, configured to determine a target template image set based on the image to be predicted when the image to be predicted is received;

[0120] a determination module, configured to perform feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set;

[0121] The input module is used to input the input image set into the deep learning model to obtain a prediction result.

[0122] Preferably, the receiving module further includes a fusion module, and the fusion module is configured to:

[0123] Fusing each template image in the template image library with the image to be predicted to obtain a fused image set;

[0124] Inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set;

[0125] A target template image set is determined in the template image library according to the first vector and the second vector set.

[0126] Preferably, the fusion module further includes a sorting module, and the sorting module is used to:

[0127] Calculating a vector distance set based on the first vector and the second vector set, and sorting each vector distance in the vector distance set to obtain a sorting result;

[0128] According to the sorting result, a target template image set is determined in the template image library.

[0129] Preferably, the determining module is further configured to:

[0130] Pairing each target template image in the target template image set with the image to be predicted, and inputting the pairings into a feature extraction module to obtain a feature extraction result;

[0131] Inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result;

[0132] The quantization result is input into a normalization module to obtain a normalized result, and the normalized result is input into a decoding and reconstruction module for reconstruction processing to obtain an input image set.

[0133] Preferably, the determining module is further configured to:

[0134] Inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module;

[0135] The dynamic quantization module obtains a quantization result based on the feature extraction result, the pixel point set and the attention weight set.

[0136] Preferably, the receiving module further includes a training module, and the training module is used to:

[0137] Acquire a training image set, generate a target training image set based on the training image set and a template image set, perform model training based on the target training image set, and obtain a deep learning pre-model;

[0138] Acquire a target verification image set, verify the deep learning pre-model based on the target verification image set, and determine the deep learning model based on the verification result.

[0139] Preferably, the training module further includes a verification module, and the verification module is used to:

[0140] Inputting the target verification image set into the deep learning pre-model to obtain a prediction result set, and determining the prediction accuracy rate based on the prediction result set;

[0141] The prediction accuracy is compared with the accuracy threshold to obtain a comparison result, and a deep learning model is determined based on the comparison result.

[0142] The invention also provides a model defense device.

[0143] The model defense device includes: a memory, a processor, and a model defense program stored in the memory and executable on the processor. When the model defense program is executed by the processor, the steps of the model defense method described above are implemented.

[0144] Among them, the method implemented when the model defense program running on the processor is executed can refer to the various embodiments of the model defense method of the present invention, and will not be repeated here.

[0145] The present invention also provides a computer-readable storage medium.

[0146] The computer-readable storage medium of the present invention stores a model defense program, and when the model defense program is executed by a processor, the steps of the model defense method described above are implemented.

[0147] Among them, the method implemented when the model defense program running on the processor is executed can refer to the various embodiments of the model defense method of the present invention, and will not be repeated here.

[0148] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0149] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0150] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0151] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A model defense method, characterized in that: The model defense method comprises the following steps: When receiving an image to be predicted, determining a target template image set according to the image to be predicted; The step of determining a target template image set based on the image to be predicted includes: Fusing each template image in the template image library with the image to be predicted to obtain a fused image set; Inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set; Determining a target template image set in the template image library according to the first vector and the second vector set; Performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set; The step of performing feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set includes: Pairing each target template image in the target template image set with the image to be predicted, and inputting the pairings into a feature extraction module to obtain a feature extraction result; Inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result; Inputting the quantization result into a normalization module to obtain a normalized result, and inputting the normalized result into a decoding and reconstruction module for reconstruction processing to obtain an input image set; The step of inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result includes: Inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module; Obtaining a quantization result by the dynamic quantization module according to the feature extraction result, the pixel point set and the attention weight set; The input image set is input into a deep learning model to obtain a prediction result.

2. The model defense method according to claim 1, characterized in that: The step of determining a target template image set in the template image library according to the first vector and the second vector set comprises: Calculating a vector distance set based on the first vector and the second vector set, and sorting each vector distance in the vector distance set to obtain a sorting result; According to the sorting result, a target template image set is determined in the template image library.

3. The model defense method according to claim 1, wherein: Before the step of determining a target template image set based on the image to be predicted when the image to be predicted is received, the model defense method includes: Acquire a training image set, generate a target training image set based on the training image set and a template image set, perform model training based on the target training image set, and obtain a deep learning pre-model; Acquire a target verification image set, verify the deep learning pre-model based on the target verification image set, and determine the deep learning model based on the verification result.

4. The model defense method according to claim 3, characterized in that: The step of verifying the deep learning pre-model according to the target verification image set and determining the deep learning model according to the verification result includes: Inputting the target verification image set into the deep learning pre-model to obtain a prediction result set, and determining the prediction accuracy rate based on the prediction result set; The prediction accuracy is compared with the accuracy threshold to obtain a comparison result, and a deep learning model is determined based on the comparison result.

5. A model defense device, characterized in that: The model defense device comprises: A receiving module, configured to determine a target template image set based on the image to be predicted when the image to be predicted is received; The receiving module further includes a fusion module, and the fusion module is used to: Fusing each template image in the template image library with the image to be predicted to obtain a fused image set; Inputting the image to be predicted into a classification model to obtain a first vector, and inputting each fused image in the fused image set into the classification model to obtain a second vector set; Determining a target template image set in the template image library according to the first vector and the second vector set; a determination module, configured to perform feature quantization and reconstruction processing on the target template image set and the image to be predicted to obtain an input image set; Wherein, the determining module is further used for: Pairing each target template image in the target template image set with the image to be predicted, and inputting the pairings into a feature extraction module to obtain a feature extraction result; Inputting the feature extraction result into an attention weight module to obtain an attention weight set, and inputting the attention weight set and the feature extraction result into a dynamic quantization module to obtain a quantization result; Inputting the quantization result into a normalization module to obtain a normalized result, and inputting the normalized result into a decoding and reconstruction module for reconstruction processing to obtain an input image set; Wherein, the determining module is further used for: Inputting the attention weight set and the feature extraction result into a dynamic quantization module, and determining a pixel point set according to the feature extraction result by the dynamic quantization module; Obtaining a quantization result by the dynamic quantization module according to the feature extraction result, the pixel point set and the attention weight set; The input module is used to input the input image set into the deep learning model to obtain a prediction result.

6. A model defense device, characterized in that: The model defense device includes: a memory, a processor, and a model defense program stored in the memory and executable on the processor. When the model defense program is executed by the processor, the steps of the model defense method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a model defense program, which, when executed by a processor, implements the steps of the model defense method according to any one of claims 1 to 4.

Images