Adversarial attack method, device and storage medium for image aesthetic evaluation model

Abstract

The present application provides an adversarial attack method, device and storage medium for an image aesthetics evaluation model, and relates to the field of image aesthetics evaluation. The method includes: obtaining an original image sample, adjusting the features of the original image sample to obtain a target attack sample; conducting an adversarial attack on the image aesthetics evaluation model to be attacked according to the original image sample and the target attack sample; inputting the original image sample into the image aesthetics evaluation model to be attacked to obtain the original image scoring information, and inputting the target attack sample into the image aesthetics evaluation model to be attacked to obtain the target attack sample scoring information; determining whether the adversarial attack conditions are met according to the original image scoring information and the target attack sample scoring information, and if so, completing the adversarial attack; if not, conducting an adversarial attack on the image aesthetics evaluation model to be attacked until the adversarial attack conditions are met. The method of the present application achieves the technical effect of improving the accuracy of the image aesthetics evaluation model.

Description

Technical Field

[0001] This application relates to the field of image aesthetic evaluation, and in particular to an adversarial attack method, device and storage medium for an image aesthetic evaluation model. Background Art

[0002] With the rapid development of computer vision and artificial intelligence, image aesthetic evaluation, as an important task in image processing and computer graphics, has gradually attracted widespread attention from researchers. With the explosive growth of AI-generated content and the self-media market, image aesthetic evaluation has become a crucial standard for automatically determining whether an image meets user aesthetic preferences. It is also an indispensable step in imaging measurement and image recommendation. The goal of image aesthetic evaluation is to simulate human aesthetic perception, using algorithms to predict the aesthetic appeal and quality of images. This allows for the automated evaluation and improvement of image quality, thereby providing a better user experience.

[0003] Currently, most image aesthetic evaluation models are trained using datasets with human-annotated labels. However, these labels may be highly subjective, suffering from sample bias, long-tailed distribution, and limited size. Consequently, these datasets restrict and weaken the model's ability to handle scarce, anomalous, or adversarial images. Therefore, existing technologies suffer from technical problems such as the need to improve the training methods for image aesthetic evaluation models and the low accuracy of aesthetic evaluation. Summary of the Invention

[0004] This application provides an adversarial attack method, device, and storage medium for an image aesthetic evaluation model, in order to solve the technical problem of low aesthetic evaluation accuracy of existing image aesthetic evaluation models.

[0005] Firstly, this application provides an adversarial attack method for an image aesthetic evaluation model, the method comprising:

[0006] Obtain the original image sample, and perform feature adjustment on the original image sample to obtain the target attack sample;

[0007] Adversarial attacks are performed on the aesthetic evaluation model of the image to be attacked based on the original image samples and the target attack samples;

[0008] The original image sample is input into the aesthetic evaluation model of the image to be attacked to obtain the original image score information, and the target attack sample is input into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample score information.

[0009] Based on the original image scoring information and the target attack sample scoring information, determine whether the adversarial attack conditions are met. If yes, the adversarial attack is completed; otherwise, perform an adversarial attack on the aesthetic evaluation model of the image to be attacked until the adversarial attack conditions are met.

[0010] Optionally, the original image samples are feature-adjusted to obtain the target attack samples, including:

[0011] Multiple attack parameter generation network models are determined, where each attack parameter generation network model corresponds to an image feature;

[0012] Substitute the original image samples into each attack parameter to generate the network model and obtain the corresponding initial attack samples;

[0013] Determine the initial weight information for each initial attack sample;

[0014] The target attack sample is determined based on multiple initial attack samples and multiple initial weight information.

[0015] Optionally, the target attack sample is determined based on multiple initial attack samples and multiple initial weight information, including:

[0016] Multiple attack samples are obtained by multiplying each initial attack sample with its initial weight information.

[0017] The target attack sample is obtained by adding multiple attack samples together.

[0018] Optionally, the aesthetic evaluation model of the image to be attacked is subjected to adversarial attacks until the adversarial attack conditions are met, including:

[0019] The initial weight information of each initial attack sample is adjusted;

[0020] The target attack sample is determined based on multiple initial attack samples and multiple adjusted weight information.

[0021] Adversarial attacks are conducted on the aesthetic evaluation model of the image to be attacked based on the target attack sample and the original image sample.

[0022] Optionally, multiple attack parameters are determined to generate a network model, including:

[0023] Determine the initial attack parameters to generate image features corresponding to the network model;

[0024] Natural image samples are acquired, and the initial attack parameters are generated by the network model to adjust the image features of the natural image samples to obtain repaired image samples.

[0025] The similarity information between natural image samples and repaired image samples is determined by a loss function until the similarity information meets the preset conditions. Then, the attack parameters corresponding to the image features are determined to generate the network model.

[0026] Optionally, the initial attack parameter generation network model adjusts the image features of natural image samples to obtain repaired image samples, including:

[0027] The perturbation modification parameters are determined based on Gaussian noise, and the image feature modification function adjusts the image features of the natural image sample according to the perturbation modification parameters to obtain the perturbation image sample.

[0028] The network model is generated based on the initial attack parameters to determine the repair and modification parameters. The image feature modification function adjusts the image features of the perturbed image sample according to the repair and modification parameters to obtain the repaired image sample.

[0029] A second aspect of this application provides an adversarial attack device for an image aesthetic evaluation model, comprising:

[0030] The first processing module acquires the original image sample and performs feature adjustment on the original image sample to obtain the target attack sample;

[0031] The second processing module performs adversarial attacks on the aesthetic evaluation model of the image to be attacked based on the original image samples and the target attack samples.

[0032] The third processing module inputs the original image sample into the aesthetic evaluation model of the image to be attacked to obtain the original image score information, and inputs the target attack sample into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample score information.

[0033] The fourth processing module determines whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If so, the adversarial attack is completed; otherwise, the aesthetic evaluation model of the image to be attacked is subjected to an adversarial attack until the adversarial attack conditions are met.

[0034] Optionally, the first processing module is also used for:

[0035] Multiple attack parameter generation network models are determined, where each attack parameter generation network model corresponds to an image feature;

[0036] Substitute the original image samples into each attack parameter to generate the network model and obtain the corresponding initial attack samples;

[0037] Determine the initial weight information for each initial attack sample;

[0038] The target attack sample is determined based on multiple initial attack samples and multiple initial weight information.

[0039] Optionally, the first processing module is also used for:

[0040] Multiple attack samples are obtained by multiplying each initial attack sample with its initial weight information.

[0041] The target attack sample is obtained by adding multiple attack samples together.

[0042] Optionally, the fourth processing module is also used for:

[0043] The initial weight information of each initial attack sample is adjusted;

[0044] The target attack sample is determined based on multiple initial attack samples and multiple adjusted weight information.

[0045] Adversarial attacks are conducted on the aesthetic evaluation model of the image to be attacked based on the target attack sample and the original image sample.

[0046] Optionally, the first processing module is also used for:

[0047] Determine the initial attack parameters to generate image features corresponding to the network model;

[0048] Natural image samples are acquired, and the initial attack parameters are generated by the network model to adjust the image features of the natural image samples to obtain repaired image samples.

[0049] The similarity information between natural image samples and repaired image samples is determined by a loss function until the similarity information meets the preset conditions. Then, the attack parameters corresponding to the image features are determined to generate the network model.

[0050] Optionally, the first processing module is also used for:

[0051] The perturbation modification parameters are determined based on Gaussian noise, and the image feature modification function adjusts the image features of the natural image sample according to the perturbation modification parameters to obtain the perturbation image sample.

[0052] The network model is generated based on the initial attack parameters to determine the repair and modification parameters. The image feature modification function adjusts the image features of the perturbed image sample according to the repair and modification parameters to obtain the repaired image sample.

[0053] Thirdly, this application provides an adversarial attack device for an image aesthetic evaluation model, comprising: a processor, and a memory communicatively connected to the processor, including:

[0054] Memory stores computer-executable instructions;

[0055] The processor executes computer execution instructions stored in memory to implement the adversarial attack method of the first aspect of the image aesthetic evaluation model.

[0056] Fourthly, a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement an adversarial attack method for the image aesthetic evaluation model of the first aspect.

[0057] Fifthly, a computer program product includes a computer program that, when executed by a processor, implements an adversarial attack method for the image aesthetic evaluation model of the first aspect.

[0058] This application provides an adversarial attack method, device, and storage medium for an image aesthetic evaluation model. The method involves acquiring an original image sample, performing feature adjustments on the original image sample to obtain a target attack sample, and then performing an adversarial attack on the image aesthetic evaluation model to be attacked based on the original image sample and the target attack sample. The method involves inputting the original image sample into the image aesthetic evaluation model to obtain original image scoring information, and inputting the target attack sample into the same model to obtain target attack sample scoring information. The method determines whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If yes, the adversarial attack is completed; otherwise, the adversarial attack continues on the image aesthetic evaluation model until the conditions are met. This method utilizes the target attack sample to perform an adversarial attack on the image aesthetic evaluation model, testing the model's handling of local image details and identifying potential errors. This further improves the detail sensitivity and accuracy of the image aesthetic evaluation, achieving the technical effect of enhancing the applicability and accuracy of the image aesthetic evaluation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] Figure 1 The flowchart of the adversarial attack method for the image aesthetic evaluation model provided in the embodiments of this application Figure 1 ;

[0061] Figure 2 The flowchart of the adversarial attack method for the image aesthetic evaluation model provided in the embodiments of this application Figure 2 ;

[0062] Figure 3 The process of determining attack parameters and generating a network model for the adversarial attack method of the image aesthetic evaluation model provided in the embodiments of this application. Figure 3 ;

[0063] Figure 4 A network structure diagram of the attack parameter generation network model for the adversarial attack method of the image aesthetic evaluation model provided in the embodiments of this application;

[0064] Figure 5 The process for determining the target attack sample in the adversarial attack method for the image aesthetic evaluation model provided in the embodiments of this application. Figure 4 ;

[0065] Figure 6 A schematic diagram of the structure of an adversarial attack device for the image aesthetic evaluation model provided in this application embodiment;

[0066] Figure 7 This is a hardware schematic diagram of an adversarial attack device for an image aesthetic evaluation model provided in an embodiment of this application.

[0067] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. DETAILED DESCRIPTION

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0069] Most existing image aesthetic evaluation models are vulnerable to adversarial attacks in real-world scenarios, especially when faced with adversarial examples designed to mislead them. This is primarily due to the current training methods for these models, which mostly rely on datasets with human-labeled annotations. These labels can be highly subjective, exhibiting sample bias, long-tailed distributions, and limited size. These models lack generalization and robustness in assessing image aesthetics, and the limitations of these datasets weaken their ability to handle scarce, anomalous, or adversarial images. Therefore, existing technologies suffer from technical problems such as the need for improved training methods for image aesthetic evaluation models and low accuracy in aesthetic evaluation.

[0070] This application provides an adversarial attack method, device, and storage medium for an image aesthetic evaluation model. The method involves acquiring an original image sample, performing feature adjustments on the original image sample to obtain a target attack sample, and then performing an adversarial attack on the image aesthetic evaluation model to be attacked based on the original image sample and the target attack sample. The method involves inputting the original image sample into the image aesthetic evaluation model to obtain original image scoring information, and inputting the target attack sample into the image aesthetic evaluation model to obtain target attack sample scoring information. The method determines whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If yes, the adversarial attack is completed; otherwise, the image aesthetic evaluation model to be attacked is further... The adversarial attack is carried out until the adversarial attack conditions are met. By adjusting the features of the original image samples to become the target attack samples, the adversarial attack is carried out using the target attack samples and the original image samples. The original image score information and the target attack sample score information output by the aesthetic evaluation model of the image to be attacked are used to determine whether the adversarial attack is completed. In this way, we can explore the way the aesthetic evaluation model of the image to be attacked processes local details of the image and the possible errors, understand the evaluation preferences of the aesthetic evaluation model of the image to be attacked, and thus improve the detail sensitivity and accuracy of the image aesthetic evaluation, and further achieve the technical effect of improving the applicability and accuracy of the image aesthetic evaluation model.

[0071] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0072] Figure 1 Adversarial attack method flow for the image aesthetic evaluation model provided in the embodiments of this application Figure 1 .like Figure 1 As shown, the adversarial attack method for the image aesthetic evaluation model provided in this embodiment includes:

[0073] S101. Obtain the original image sample and perform feature adjustment on the original image sample to obtain the target attack sample;

[0074] In this embodiment, the target attack sample refers to the image sample formed after the image features of the original image sample have been adjusted and changed. The purpose of the target attack sample is to cause the aesthetic evaluation model of the image to be attacked to output incorrect prediction results.

[0075] S102. Perform adversarial attacks on the aesthetic evaluation model of the image to be attacked based on the original image samples and the target attack samples;

[0076] In this embodiment, adversarial attack refers to deceiving the scoring result of the aesthetic evaluation model of the image to be attacked by introducing minor changes to the original image. The original image sample and the target attack sample are input into the aesthetic evaluation model of the image to be attacked, and the aesthetic evaluation model of the image to be attacked scores the image sample.

[0077] S103. Input the original image sample into the aesthetic evaluation model of the image to be attacked to obtain the original image score information, and input the target attack sample into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample score information.

[0078] S104. Determine whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If yes, complete the adversarial attack; otherwise, perform an adversarial attack on the aesthetic evaluation model of the image to be attacked until the adversarial attack conditions are met.

[0079] In this embodiment, adversarial attack information is obtained based on the original image information and the target attack sample scoring information. It is then determined whether the adversarial attack information meets the preset conditions. If the adversarial attack information does not meet the preset conditions, the target attack sample is adjusted, a new target attack sample is obtained, and the adversarial attack continues on the aesthetic evaluation model of the image to be attacked until the adversarial attack conditions are met. If the adversarial attack information meets the preset conditions, the training of the aesthetic evaluation model of the image to be attacked is completed. Based on the training results, the aesthetic logic of the aesthetic evaluation model of the image to be attacked can be analyzed. Based on the analysis results, the parameters of the aesthetic evaluation model of the image to be attacked can be adjusted to improve the accuracy and applicability of the image aesthetic evaluation model.

[0080] This application provides an adversarial attack method for an image aesthetic evaluation model. The method involves acquiring an original image sample, adjusting its features to obtain a target attack sample, and then performing an adversarial attack on the image aesthetic evaluation model to be attacked based on the original image sample and the target attack sample. The method involves inputting the original image sample into the model to obtain original image scoring information, and inputting the target attack sample into the model to obtain target attack sample scoring information. The method determines whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If yes, the adversarial attack is completed; otherwise, the adversarial attack continues on the image aesthetic evaluation model until the conditions are met. The image features of the original image sample are adjusted to obtain the target attack sample. The original image sample and the target attack sample are then input into the image aesthetic evaluation model to be attacked, where the model scores the image sample and utilizes the target attack. The sample causes the aesthetic evaluation model of the image to be attacked to output incorrect prediction results. The original image sample is input into the aesthetic evaluation model of the image to be attacked to obtain the original image score information, and the target attack sample is input into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample score information. Based on the original image information and the target attack sample score information, adversarial attack information is obtained. It is determined whether the adversarial attack information meets the preset conditions. If the adversarial attack information does not meet the preset conditions, the target attack sample is adjusted and a new target attack sample is obtained. The adversarial attack continues to be performed on the aesthetic evaluation model of the image to be attacked until the adversarial attack conditions are met. If the adversarial attack information meets the preset conditions, the training of the aesthetic evaluation model of the image to be attacked is completed. Based on the training results, the aesthetic logic of the aesthetic evaluation model of the image to be attacked can be analyzed. Based on the analysis results, the parameters of the aesthetic evaluation model of the image to be attacked can be adjusted to improve the accuracy and applicability of the image aesthetic evaluation model.

[0081] Figure 2 Adversarial attack method flow for the image aesthetic evaluation model provided in the embodiments of this application Figure 2 This embodiment is... Figure 1 Based on the embodiments, the adversarial attack method for the image aesthetic evaluation model is described in detail. Multiple attack parameters can be determined to generate a network model according to steps S201-S202; feature adjustment of the original image sample using the network model with multiple attack parameters to obtain the target attack sample can be achieved through steps S203-S204; the adversarial attack on the image aesthetic evaluation model to be attacked can be completed using the target attack sample and the original image sample through steps S205-S207; as shown below. Figure 2 As shown, the adversarial attack method for the image aesthetic evaluation model provided in this embodiment includes:

[0082] S201. Determine the image features corresponding to the initial attack parameters and generate the network model; obtain natural image samples, determine the perturbation modification parameters based on Gaussian noise, and use the image feature modification function to adjust the image features of the natural image samples according to the perturbation modification parameters to obtain perturbed image samples; wherein the repair modification parameters are determined based on the initial attack parameters and the image feature modification function is used to adjust the image features of the perturbed image samples according to the repair modification parameters to obtain repaired image samples.

[0083] In this embodiment, image features include, but are not limited to, image brightness, contrast, saturation, color temperature, hue, tone, sharpness, resolution, color balance, graininess, and exposure. Gaussian noise is a common type of noise in digital images, characterized by noise appearing at almost every point with random noise depth. An image feature modification function is a differentiable function that can quantitatively modify image features based on input feature parameters. Such functions, when given the original image and modification parameters, can output the modified image.

[0084] S202. The similarity information between natural image samples and repaired image samples is determined according to the loss function until the similarity information meets the preset conditions. Then, the attack parameters corresponding to the image features are determined to generate the network model.

[0085] The loss function can determine similarity information using natural image samples and restored image samples. The goal of obtaining restored image samples is to modify the image features of natural image samples to make the restored image samples as similar as possible to the natural image samples. The similarity information can be calculated using the following formula:

[0086]

[0087] Where x is a natural image sample, To repair the image sample, H is the length of the image; W is the width of the image; C is the number of channels of the image, C=3.

[0088] In one possible implementation, when the similarity information calculated by the loss function based on the repaired image sample and the natural image sample obtained in the nth time is the smallest among the n similarity information, the attack parameter generation network model corresponding to the image feature can be determined.

[0089] In one possible implementation, when the similarity calculated by the loss function based on the repaired image sample and the natural image sample obtained in the nth time meets a preset threshold, the attack parameter generation network model corresponding to the image feature can be determined.

[0090] Figure 3 The process of determining attack parameters and generating a network model for the adversarial attack method of the image aesthetic evaluation model provided in the embodiments of this application. Figure 3 ,like Figure 3 As shown, taking saturation as the image feature corresponding to the network model generated by determining the initial attack parameters as an example, after obtaining natural image samples, perturbation modification parameters related to saturation are determined through Gaussian noise. The image feature modification function adjusts the saturation of the natural image samples according to the perturbation modification parameters to obtain perturbed image samples. Then, repair modification parameters related to saturation are determined based on the network model generated by the initial attack parameters. The image feature modification function further adjusts the saturation of the perturbed image samples according to the repair modification parameters to obtain repaired image samples. Finally, the similarity information between the repaired image samples and the natural image samples is calculated using a loss function. If the similarity information meets the preset threshold, the attack parameter generation network model for adjustable saturation image features is determined. If the similarity information does not meet the preset threshold, the repair and modification parameters are determined again based on the initial attack parameter generation network model. The image feature modification function adjusts the saturation of the perturbed image sample according to the adjusted repair and modification parameters to obtain the adjusted repair image sample. Then, it is determined whether the similarity between the adjusted repair image sample and the natural image sample meets the preset threshold. Once the preset threshold is met, the attack parameter generation network model can be determined. This attack parameter generation network model can adjust the saturation of the image.

[0091] Figure 4 The network structure diagram of the attack parameter generation network model for the adversarial attack method of the image aesthetic evaluation model provided in the embodiments of this application is as follows: Figure 4 As shown, the first layer of the attack parameter generation network model is a label encoding layer, which transforms the input image tensor of shape 3×256×256 into a 32×32×3 label sequence. In addition to the image input, the attack parameter generation network model also incorporates additional data: filter prior information. Customized prior information is provided for attack parameter generation models with different image features, ensuring the uniqueness of the attack parameter generation model for each image feature. Several layers of normalization and linear networks transform the image sequence into filter-modified parameters and output them. The filter prior information is used to regress the filter parameter norms and filter parameter offsets of the latent space features of each attack parameter generation network model, and then these are merged into an adaptive normalization layer. This adaptive latent space adjustment method not only enhances the generalization to feature distribution variations in different latent spaces but also enables it to skillfully learn inductive biases specific to certain image features. Finally, a conditional regression attention matrix is ​​used to selectively activate image features, which helps to focus attention on important information related to a specific image feature, significantly improving the effectiveness of the attack parameter generation network model.

[0092] S203. Determine multiple attack parameter generation network models, where each attack parameter generation network model corresponds to an image feature. Substitute the original image samples into each attack parameter generation network model to obtain the corresponding initial attack samples, and determine the initial weight information for each initial attack sample.

[0093] In this embodiment, the original image samples are substituted into each attack parameter to generate a network model to obtain multiple image feature modification parameters. The image feature modification function determines multiple initial attack samples based on each image feature modification parameter. Each initial attack sample corresponds to a modified image feature. The initial weight information of each initial image sample is determined. The weight information corresponding to each image feature represents the degree of importance that the image aesthetic evaluation model to be trained attaches to each image feature. The initial weight information of multiple initial image samples is added together to get 1.

[0094] S204. Multiply each initial attack sample by its initial weight information to obtain multiple attack samples; add the multiple attack samples to obtain the target attack sample.

[0095] In this embodiment, each initial attack sample is multiplied by its initial weight information to obtain multiple attack samples. The image features corresponding to each attack sample have different degrees of importance in the aesthetic evaluation model of the image to be attacked. By adding multiple attack samples together, multiple modified image features are superimposed on the original image sample to obtain the target attack sample.

[0096] Figure 5 The process for determining the target attack sample in the adversarial attack method for the image aesthetic evaluation model provided in the embodiments of this application. Figure 4 ,like Figure 5As shown, taking image features including saturation, brightness, color temperature, and contrast as an example, according to steps S201-S202, the attack parameter generation network models corresponding to saturation, brightness, color temperature, and contrast are determined respectively, generating saturation modification parameters, brightness modification parameters, color temperature modification parameters, and contrast modification parameters. The image feature modification function modifies the saturation, brightness, color temperature, and contrast of the original image sample using the saturation modification parameters, brightness modification parameters, color temperature modification parameters, and contrast modification parameters respectively, determining the initial attack sample after saturation modification, the initial attack sample after brightness modification, the initial attack sample after color temperature modification, and the initial attack sample after contrast modification. Based on the modified feature fusion network, the initial attack sample after saturation modification is set. The initial attack sample with modified brightness has a weight of 0.2, the initial attack sample with modified color temperature has a weight of 0.3, and the initial attack sample with modified contrast has a weight of 0.1 and 0.4. The initial attack sample with modified saturation is multiplied by a weight of 0.2, the initial attack sample with modified brightness by a weight of 0.3, the initial attack sample with modified color temperature by a weight of 0.1, and the initial attack sample with modified contrast by a weight of 0.4, resulting in attack samples corresponding to saturation, brightness, color temperature, and contrast. These four attack samples are then added together, effectively superimposing the effects of modifications to saturation, brightness, color temperature, and contrast to obtain the target attack sample.

[0097] S205. Perform adversarial attacks on the aesthetic evaluation model of the image to be attacked based on the original image samples and the target attack samples;

[0098] S206. Input the original image sample into the aesthetic evaluation model of the image to be attacked to obtain the original image scoring information, and input the target attack sample into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample scoring information.

[0099] S207. Determine whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If yes, complete the adversarial attack. If not, adjust the initial weight information of each initial attack sample and determine the target attack sample based on multiple initial attack samples and multiple adjusted weight information. Perform an adversarial attack on the aesthetic evaluation model of the image to be attacked based on the target attack sample and the original image sample.

[0100] In one possible implementation, the adversarial attack conditions are calculated using the original image scoring information and the target attack sample scoring information, as shown in the following formula:

[0101]

[0102] Where M is the aesthetic evaluation model of the image to be attacked; x is the original image sample; M(x) is the original image score information obtained by inputting the original image sample into the aesthetic evaluation model of the image to be attacked; y is the target attack sample; M(y) is the target attack sample score information obtained by inputting the target attack image sample into the aesthetic evaluation model of the image to be attacked; Loss is the adversarial attack information.

[0103] Optionally, the purpose of adversarial attacks is to make the target attack sample obtain a higher score in the aesthetic evaluation model of the image to be attacked than the original image scoring information. When the adversarial attack information obtained when the aesthetic evaluation model of the image to be attacked is trained for the nth time is the smallest value among the n adversarial attack information, the adversarial attack is completed.

[0104] In this embodiment, when the original image scoring information and the target attack sample scoring information fail to meet the anti-training conditions, the modified feature fusion network adjusts the initial weight information of each initial attack sample. During the adversarial attack, due to the occasional similarities in the early training data, adjusting the weights may sometimes only combine a few attack parameters to generate the modified network model's effect, without exploring the combined attack effect of more attack parameters. This results in underutilization of image features and encountering local minima. Therefore, an additional term R is introduced, which penalizes the distribution of the n filter weights w exhibiting high concentration and low entropy. The calculation formula is shown below:

[0105]

[0106] Where n is the number of attack parameter generation network models, and w represents the weight information; w i These are the weights assigned to the i-th attack parameter to generate the network model.

[0107] In one possible implementation, the following formula can be used to determine whether the original image scoring information and the target attack sample scoring information satisfy the adversarial attack conditions:

[0108] ;

[0109] ;

[0110]

[0111] Among them, L attack To counter attack information; α is the regularization parameter; β is the regularization parameter; The difference between the original image score and the target attack sample score can be used to measure the attack effect; M(x) is the original image score obtained by the aesthetic evaluation model of the original image sample input to the image to be attacked; F i(x) represents the initial attack image, w i Weights for each initial attack image; F i (x) and w i The sum of the products represents the target attack sample; R(w) is the introduced additional term; where n is the number of attack parameter generation network models, and w represents the weight information; w i is the weight assigned to the i-th attack parameter to generate the network model; G(x) is used to measure the pixel difference between the original image and the target attack sample; H is the length of the image; W is the width of the image; C is the number of channels of the image; i is the three-dimensional X coordinate of an RGB image, ranging from [0, c], j is the three-dimensional Y coordinate of an RGB image, ranging from [0, h], and k is the three-dimensional Z coordinate of an RGB image, ranging from [0, k].

[0112] Optionally, by controlling the two regularization parameters α and β in the above formula, the weight information of each initial attack sample can be adjusted to achieve different types of adversarial attacks. For example, an unrestricted attack can be achieved, making β less than α, so that the training focuses more on the attack effect without restricting the modification of image pixels. That is, the goal of adversarial attack training is to improve the score of the target attack sample on the aesthetic evaluation model of the image to be attacked as much as possible. Alternatively, a restricted attack can be achieved, making β greater than α. That is, when adversarial attack training improves the score of the target attack sample, it is also necessary to pay attention to reducing the pixel difference between the target attack sample and the original image sample.

[0113] Optionally, after completing the adversarial attack on the image aesthetic evaluation model, the weight of each image feature in the image aesthetic evaluation model can be obtained based on the weight information corresponding to each initial attack sample after training. That is, the degree of importance that the image aesthetic evaluation model attaches to each image feature. Based on the weight of each image feature in the image aesthetic evaluation model, the evaluation preference and scoring logic of the image aesthetic evaluation model can be analyzed, thereby adjusting the parameters of the image aesthetic evaluation model in a targeted manner to improve the evaluation accuracy and practicality of the image aesthetic evaluation model for complex and detailed images.

[0114] This application provides an adversarial attack method for an image aesthetic evaluation model. The method involves determining initial attack parameters to generate image features corresponding to a network model; acquiring natural image samples; determining perturbation modification parameters based on Gaussian noise; using an image feature modification function to adjust the image features of the natural image samples according to the perturbation modification parameters to obtain perturbed image samples; determining repair modification parameters based on the initial attack parameters to generate a network model; using the image feature modification function to adjust the image features of the perturbed image samples according to the repair modification parameters to obtain repaired image samples; and determining the similarity information between the natural image samples and the repaired image samples using a loss function until the similarity information meets preset conditions. This process determines the attack parameter generation network model corresponding to each image feature. Each network model corresponds to one image feature. The original image samples are substituted into each attack parameter to generate the network model, resulting in multiple image feature modification parameters. The image feature modification function determines multiple initial attack samples based on each parameter, with each initial attack sample corresponding to a modified image feature. Initial weight information for each initial image sample is determined, and each initial attack sample is multiplied by its initial weight information to obtain multiple attack samples. The image features corresponding to each attack sample have different levels of importance in the aesthetic evaluation model of the image to be attacked. Adding these multiple attack samples together results in multiple modified image features superimposed on the original image sample, thus obtaining the target attack sample. Using the original image samples… The process involves an adversarial attack on the aesthetic evaluation model of the image to be attacked, using both the original image sample and the target attack sample. The original image sample is input into the aesthetic evaluation model to obtain its score, and the target attack sample is input into the same model to obtain its score. Based on the original and target attack sample scores, it is determined whether the adversarial attack conditions are met. If so, the adversarial attack is completed; otherwise, the initial weights of each initial attack sample are adjusted. The target attack sample is then determined based on multiple initial attack samples and adjusted weights. An adversarial attack is then performed on the aesthetic evaluation model of the image to be attacked using both the target attack sample and the original image sample. Finally, the initial attack parameters are generated by applying natural image samples to the network model. Multiple attack parameter generation network models are obtained through training. Each attack parameter generation network model corresponds to an image feature. After the original image samples are processed by multiple attack parameter generation network models, multiple initial attack samples are obtained. The target attack sample is determined by the initial attack samples and the weight information of each initial attack sample. By continuously adjusting the weight information of each initial attack sample, the target attack sample performs an adversarial attack on the image aesthetic evaluation model to be attacked. Based on the weight of each image feature in the image aesthetic evaluation model, the evaluation preference and scoring logic of the image aesthetic evaluation model can be analyzed. Thus, the parameters of the image aesthetic evaluation model can be adjusted in a targeted manner to improve the evaluation accuracy and practicality of the image aesthetic evaluation model for complex and detailed images.

[0115] Figure 6 A schematic diagram of the structure of an adversarial attack device for an image aesthetic evaluation model provided in this application embodiment. Figure 1 .like Figure 6 As shown in the embodiment of this application, an adversarial attack device 600 for an image aesthetic evaluation model is provided. The device includes: a first processing module 601, a second processing module 602, a third processing module 603, and a fourth processing module 604.

[0116] The first processing module acquires the original image sample and performs feature adjustment on the original image sample to obtain the target attack sample;

[0117] The second processing module performs adversarial attacks on the aesthetic evaluation model of the image to be attacked based on the original image samples and the target attack samples.

[0118] The third processing module inputs the original image sample into the aesthetic evaluation model of the image to be attacked to obtain the original image score information, and inputs the target attack sample into the aesthetic evaluation model of the image to be attacked to obtain the target attack sample score information.

[0119] The fourth processing module determines whether the adversarial attack conditions are met based on the original image scoring information and the target attack sample scoring information. If so, the adversarial attack is completed; otherwise, the aesthetic evaluation model of the image to be attacked is subjected to an adversarial attack until the adversarial attack conditions are met.

[0120] In one possible implementation, the first processing module is further used for:

[0121] Multiple attack parameter generation network models are determined, where each attack parameter generation network model corresponds to an image feature;

[0122] Substitute the original image samples into each attack parameter to generate the network model and obtain the corresponding initial attack samples;

[0123] Determine the initial weight information for each initial attack sample;

[0124] The target attack sample is determined based on multiple initial attack samples and multiple initial weight information.

[0125] In one possible implementation, the first processing module is further used for:

[0126] Multiple attack samples are obtained by multiplying each initial attack sample with its initial weight information.

[0127] The target attack sample is obtained by adding multiple attack samples together.

[0128] In one possible implementation, the fourth processing module is also used for:

[0129] The initial weight information of each initial attack sample is adjusted;

[0130] The target attack sample is determined based on multiple initial attack samples and multiple adjusted weight information.

[0131] Adversarial attacks are conducted on the aesthetic evaluation model of the image to be attacked based on the target attack sample and the original image sample.

[0132] In one possible implementation, the first processing module is further used for:

[0133] Determine the initial attack parameters to generate image features corresponding to the network model;

[0134] Natural image samples are acquired, and the initial attack parameters are generated by the network model to adjust the image features of the natural image samples to obtain repaired image samples.

[0135] The similarity information between natural image samples and repaired image samples is determined by a loss function until the similarity information meets the preset conditions. Then, the attack parameters corresponding to the image features are determined to generate the network model.

[0136] In one possible implementation, the first processing module is further used for:

[0137] The perturbation modification parameters are determined based on Gaussian noise, and the image feature modification function adjusts the image features of the natural image sample according to the perturbation modification parameters to obtain the perturbation image sample.

[0138] The network model is generated based on the initial attack parameters to determine the repair and modification parameters. The image feature modification function adjusts the image features of the perturbed image sample according to the repair and modification parameters to obtain the repaired image sample.

[0139] This application provides an adversarial attack device for an image aesthetic evaluation model. The device obtains a target attack sample by acquiring an original image sample and performing feature adjustments on the original image sample. It then performs an adversarial attack on the image aesthetic evaluation model to be attacked based on the original image sample and the target attack sample. The original image sample is input into the image aesthetic evaluation model to obtain original image scoring information, and the target attack sample is input into the model to obtain target attack sample scoring information. Based on the original image scoring information and the target attack sample scoring information, the device determines whether the adversarial attack conditions are met. If yes, the adversarial attack is completed; otherwise, the adversarial attack continues on the image aesthetic evaluation model until the adversarial attack conditions are met. The process involves: determining initial attack parameters to generate image features corresponding to the network model; acquiring natural image samples; determining perturbation modification parameters based on Gaussian noise; adjusting the image features of the natural image samples using an image feature modification function based on the perturbation modification parameters to obtain perturbed image samples; determining repair modification parameters based on the initial attack parameters to generate the network model; adjusting the image features of the perturbed image samples using an image feature modification function based on the repair modification parameters to obtain repaired image samples; determining the similarity information between the natural image samples and the repaired image samples using a loss function until the similarity information meets preset conditions, thereby determining the attack parameter generation network model corresponding to the image features. Each attack parameter generation network model corresponds to... For each image feature, the original image sample is substituted into each attack parameter to generate a network model, resulting in multiple image feature modification parameters. The image feature modification function determines multiple initial attack samples based on each image feature modification parameter. Each initial attack sample corresponds to a modified image feature. The initial weight information of each initial image sample is determined. Each initial attack sample is multiplied by its initial weight information to obtain multiple attack samples. The image feature corresponding to each attack sample has different levels of importance in the aesthetic evaluation model of the image to be attacked. The multiple attack samples are added together, and the multiple modified image features are superimposed on the original image sample to obtain the target attack sample. Adversarial attacks are performed on the aesthetic evaluation model of the image to be attacked using original image samples and target attack samples. The original image samples are input into the aesthetic evaluation model to obtain original image scoring information, and the target attack samples are input into the same model to obtain target attack sample scoring information. Based on the original image scoring information and the target attack sample scoring information, it is determined whether the adversarial attack conditions are met. If yes, the adversarial attack is completed; otherwise, the initial weight information of each initial attack sample is adjusted. The target attack sample is determined based on multiple initial attack samples and multiple adjusted weight information. Finally, an adversarial attack is performed on the aesthetic evaluation model of the image to be attacked using both the target attack sample and the original image samples.Multiple attack parameter generation network models are obtained by training an initial attack parameter generation network model using natural image samples. Each attack parameter generation network model corresponds to an image feature. After the original image samples are processed by these multiple attack parameter generation network models, multiple initial attack samples are obtained. The target attack sample is determined using these initial attack samples and their weights. By continuously adjusting the weights of each initial attack sample, the target attack sample performs an adversarial attack on the image aesthetic evaluation model. Based on the weight of each image feature in this aesthetic evaluation model, the evaluation preferences and scoring logic of the model can be analyzed. This allows for targeted adjustment of the model's parameters, improving its evaluation accuracy and practicality for complex and detailed images.

[0140] Figure 7 A schematic diagram illustrating the adversarial attack structure of the image aesthetic evaluation model provided in this application. (See diagram for example.) Figure 7 As shown, the electronic device 700 provided in this embodiment includes at least one processor 701 and a memory 702. Optionally, the device 700 further includes a communication component 703. The processor 701, memory 702, and communication component 703 are connected via a bus 704.

[0141] In a specific implementation, at least one processor 701 executes computer execution instructions stored in memory 702, causing at least one processor 701 to perform the above-described method.

[0142] The specific implementation process of processor 701 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0143] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0144] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0145] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0146] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0147] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0148] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0149] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0150] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0153] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0155] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for adversarial attack on an image aesthetics assessment model, characterized in that: include: Acquire an original image sample, and perform feature adjustment on the original image sample to obtain a target attack sample; Performing an adversarial attack on the aesthetic evaluation model of the image to be attacked according to the original image sample and the target attack sample; Inputting the original image sample into the to-be-attacked image aesthetic evaluation model to obtain original image score information, and inputting the target attack sample into the to-be-attacked image aesthetic evaluation model to obtain target attack sample score information; Determine whether the adversarial attack condition is met according to the original image scoring information and the target attack sample scoring information, and if so, complete the adversarial attack; if not, perform the adversarial attack on the aesthetic evaluation model of the image to be attacked until the adversarial attack condition is met; The original image sample is subjected to feature adjustment to obtain a target attack sample, including: Determine a plurality of attack parameter generation network models, wherein each attack parameter generation network model corresponds to an image feature; Substituting the original image samples into each attack parameter generation network model to obtain corresponding initial attack samples; Determine the initial weight information of each initial attack sample; Determine a target attack sample according to a plurality of initial attack samples and a plurality of initial weight information; The step of determining a plurality of attack parameters to generate a network model comprises: Determine the image features corresponding to the initial attack parameter generation network model; Obtaining a natural image sample, using the initial attack parameter generation network model to adjust image features of the natural image sample to obtain a repaired image sample; The natural image sample and the restored image sample determine similarity information according to a loss function until the similarity information meets a preset condition, and then determine the attack parameter corresponding to the image feature to generate a network model.

2. The method according to claim 1, characterized in that The step of determining a target attack sample according to a plurality of initial attack samples and a plurality of initial weight information includes: Multiplying each of the initial attack samples and the initial weight information of each of the initial attack samples to obtain a plurality of attack samples; The multiple attack samples are added together to obtain a target attack sample.

3. The method according to claim 2, characterized in that The image aesthetics assessment model to be attacked performs the adversarial attack until the adversarial attack condition is met, including: Adjusting the initial weight information of each initial attack sample; Determine a target attack sample according to the multiple initial attack samples and the multiple adjusted weight information; The adversarial attack is performed on the image aesthetics assessment model to be attacked according to the target attack sample and the original image sample.

4. The method according to claim 3, characterized in that The initial attack parameter generation network model performs image feature adjustment on the natural image sample to obtain a repaired image sample, including: Determining a disturbance modification parameter according to Gaussian noise, and using an image feature modification function to adjust image features of the natural image sample according to the disturbance modification parameter to obtain a disturbed image sample; A network model is generated according to the initial attack parameters to determine a repair modification parameter, and the image feature modification function performs image feature adjustment on the disturbed image sample according to the repair modification parameter to obtain a repaired image sample.

5. A device for countering attacks on an image aesthetics evaluation model, characterized in that: include: A first processing module obtains an original image sample and performs feature adjustment on the original image sample to obtain a target attack sample; A second processing module performs an adversarial attack on the aesthetic evaluation model of the attacked image according to the original image sample and the target attack sample; A third processing module, inputting the original image sample into the to-be-attacked image aesthetic evaluation model to obtain original image score information, and inputting the target attack sample into the to-be-attacked image aesthetic evaluation model to obtain target attack sample score information; A fourth processing module determines whether the adversarial attack condition is met according to the original image scoring information and the target attack sample scoring information, and if so, completes the adversarial attack; if not, performs the adversarial attack on the aesthetic evaluation model of the image to be attacked until the adversarial attack condition is met; The first processing module is specifically used for: Determine a plurality of attack parameter generation network models, wherein each attack parameter generation network model corresponds to an image feature; Substitute the original image samples into each attack parameter generation network model to obtain the corresponding initial attack samples; Determine the initial weight information of each initial attack sample; Determine a target attack sample according to a plurality of initial attack samples and a plurality of initial weight information; When determining multiple attack parameters to generate a network model, the first processing module is specifically used to: Determine the image features corresponding to the initial attack parameter generation network model; Obtain natural image samples, generate a network model with initial attack parameters to adjust image features of the natural image samples, and obtain repaired image samples; The natural image samples and the repaired image samples determine the similarity information according to the loss function until the similarity information meets the preset conditions, and then determine the attack parameters corresponding to the image features to generate the network model.

6. A device for countering attacks on an image aesthetics evaluation model, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the adversarial attack method for an image aesthetics assessment model according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the adversarial attack method for the image aesthetics evaluation model according to any one of claims 1 to 4.

8. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 4 when being executed by a processor.

Images