Disturbance sample pair generation method

Abstract

The invention provides a disturbance sample pair generation method and a multi-modal model performance evaluation method based on enhanced and complementary space search. The disturbance sample pair generation method comprises the following steps: acquiring a sample pair and a matching label of the sample pair; for the sample pair, repeatedly executing the following operations until a disturbance strategy meets a preset condition: for each disturbance person in the plurality of disturbance persons, respectively adding disturbance data to the image and the text to obtain respective disturbance images and disturbance texts of the plurality of disturbance persons, the disturbance data being generated based on the disturbance strategy; respectively inputting the disturbance images and the disturbance texts of the plurality of disturbance persons into a multi-modal model to obtain a plurality of disturbance matching data; adjusting a disturbance strategy based on a target reward value determined by the multiple pieces of disturbance matching data and the matching label; and determining the disturbance image and the disturbance text under the condition that the disturbance strategy meets a preset condition as a target disturbance sample pair.

Description

Technical Field

[0001] The present invention relates to the field of multimodal model security technology, and more specifically to a method for generating perturbation sample pairs, and a method, device, and equipment for evaluating multimodal model performance based on reinforcement and complementary space search. Background Art

[0002] Vision-language pre-trained models are usually pre-trained on a large number of image-text pairs and learn cross-modal representations so that the model can understand and associate information in images and text. Due to the emergence of various adversarial perturbations, the adversarial robustness of vision-language pre-trained models has attracted attention. The purpose of security assessment is to identify and quantify the vulnerability of the model in the face of adversarial attacks. Therefore, the security of vision-language pre-trained models can be tested by designing perturbation samples and perturbing the vision-language pre-trained models.

[0003] In related technologies, each modality in the visual-language pre-training model is often perturbed separately through multiple single-modality perturbation data, and the effect of collaborative perturbation is not good. However, since it is impossible to know the specific perturbation process within the visual-language pre-training model, effective optimization cannot be performed. In addition, due to excessive focus on salient features, the generated perturbation data has poor generalization effect. Summary of the invention

[0004] In view of the above problems, the present invention provides a method for generating perturbation sample pairs, and a method, device, and equipment for evaluating the performance of a multimodal model based on reinforcement and complementary space search.

[0005] According to a first aspect of the present invention, a method for generating a perturbation sample pair is provided, comprising: obtaining a sample pair and a matching label of the sample pair, the sample pair comprising an image and a text, the matching label representing the consistency of the image and the text; for the sample pair, repeatedly performing the following operations until a perturbation strategy meets a preset condition: for each of a plurality of perturbers, adding perturbation data to the image and the text respectively, to obtain a perturbation image and a perturbation text of each of the plurality of perturbers, the perturbation data being generated based on the perturbation strategy; inputting the perturbation image and the perturbation text of each of the plurality of perturbers into a multimodal model respectively, to obtain a plurality of perturbation matching data; adjusting the perturbation strategy based on a target reward value determined based on the plurality of perturbation matching data and the matching label; and determining the perturbation image and the perturbation text when the perturbation strategy meets the preset condition as a target perturbation sample pair.

[0006] According to an embodiment of the present invention, the above-mentioned perturbation matching data includes similarity and perturbation matching results, and the above-mentioned similarity represents the degree of consistency between the above-mentioned image and text. The above-mentioned method also includes: for each of the above-mentioned disturbers, using an alignment reward function, determining a sub-alignment reward value based on the above-mentioned similarity; determining a sub-perturbation reward value based on the above-mentioned perturbation matching result and the above-mentioned matching label; determining a candidate reward value of the above-mentioned disturber based on the above-mentioned sub-alignment reward value and the above-mentioned sub-perturbation reward value; and determining the above-mentioned target reward value based on the candidate reward values ​​of each of the above-mentioned multiple disturbers.

[0007] According to an embodiment of the present invention, the perturbation strategy includes an image perturbation direction, a text perturbation direction and a strategy distribution parameter, the image perturbation direction represents the probability of adding perturbation to a random sub-region in the image, the text perturbation direction represents the probability of adding perturbation to a random sub-text in the text, the random sub-region and the random sub-text have a corresponding relationship, the strategy distribution parameter is used to control the consistency difference between the image perturbation caused by the image perturbation direction and the text perturbation caused by the text perturbation direction, and the perturbation strategy is adjusted based on the target reward value, including: determining the image perturbation direction and the text perturbation direction based on the sub-alignment reward value of the target reward value; adjusting the strategy distribution parameter based on the sub-disturbance reward value of the target reward value to obtain an adjusted strategy distribution function.

[0008] According to an embodiment of the present invention, each of the above-mentioned disturbers has a preset disturbance iteration value, and the above-mentioned preset conditions include: the number of iterations reaches the above-mentioned preset disturbance iteration value or the above-mentioned disturbance strategy makes the above-mentioned target reward value converge.

[0009] According to an embodiment of the present invention, the adding of disturbance data to the image and the text to obtain the disturbance image and the disturbance text respectively includes: adding random noise to the image to obtain the disturbance image; performing synonym replacement or word embedding on sub-text in the text to obtain the disturbance text.

[0010] According to an embodiment of the present invention, the adding of disturbance data to the image and the text to obtain the disturbance image and the disturbance text respectively includes: adding random noise to the image to obtain the disturbance image; performing synonym replacement or word embedding on sub-text in the text to obtain the disturbance text.

[0011] According to an embodiment of the present invention, the step of adding random noise to the image to obtain the disturbed image includes: determining a target sub-region of the image based on a disturbance direction of the image; and adding noise to the target sub-region to obtain the disturbed image.

[0012] According to a second aspect of the present invention, a multimodal model performance evaluation method based on reinforcement and complementary space search is provided, comprising: obtaining a target perturbation sample pair, the target perturbation sample pair comprising target perturbation image data and target perturbation text data, the target perturbation sample pair being obtained using the method of the first aspect; using the target perturbation sample pair, performing a performance evaluation on the multimodal model to obtain a performance evaluation result, wherein the performance evaluation result is used to improve the security of the multimodal model.

[0013] A third aspect of the present invention provides a perturbation sample pair generation device, comprising: an acquisition module, used to acquire a sample pair and a matching label of the sample pair, the sample pair comprising an image and a text, the matching label characterizing the consistency of the image and the text; a perturbation data adding module, used to add perturbation data to the image and the text respectively for each of a plurality of perturbers, to obtain a perturbation image and a perturbation text of each of the plurality of perturbers, the perturbation data being generated based on the perturbation strategy; an input module, used to input the perturbation image and the perturbation text of each of the plurality of perturbers into a multimodal model, to obtain a plurality of perturbation matching data; an adjustment module, used to adjust the perturbation strategy based on a target reward value determined by the plurality of perturbation matching data and the matching label; and a determination module, used to determine the perturbation image and the perturbation text as a target perturbation sample pair when the perturbation strategy satisfies the preset condition.

[0014] The fourth aspect of the present invention provides a multimodal model performance evaluation device based on enhanced and complementary space search, comprising: a target sample pair acquisition module, used to obtain a target perturbation sample pair, the target perturbation sample pair includes target perturbation image data and target perturbation text data, and the target perturbation sample pair is obtained using the method of the first aspect; a performance evaluation module, used to use the target perturbation sample pair to perform performance evaluation on the multimodal model to obtain a performance evaluation result, and the performance evaluation result is used to improve the security of the multimodal model.

[0015] A fifth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0016] The sixth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0017] The seventh aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0018] According to an embodiment of the present invention, multiple disturbers add disturbances generated by a perturbation strategy to images and texts respectively, to obtain disturbed images and perturbated texts of the multiple disturbers respectively, and the perturbated images and perturbated texts of the multiple disturbers respectively are input into a multimodal model to obtain multiple perturbation matching data, and the perturbation strategy in the perturbation multimodal model is optimized according to a target reward value determined by the perturbation data and the matching label, thereby optimizing the collaborative perturbation in the perturbation process and improving the generalization effect of the generated perturbation sample pairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0020] Figure 1 A diagram showing application scenarios of a method for generating perturbation sample pairs, a method for evaluating multimodal model performance based on reinforcement and complementary space search, and a device according to an embodiment of the present invention.

[0021] Figure 2 A flow chart of a method for generating perturbation sample pairs according to an embodiment of the present invention is shown.

[0022] Figure 3 A schematic diagram showing a comparison between disturbance results generated by multiple disturbers and sample pairs according to an embodiment of the present invention is shown;

[0023] Figure 4 A schematic diagram of an architecture according to an embodiment of the present invention is shown;

[0024] Figure 5 An example diagram of sample pair generation according to an embodiment of the present invention is shown;

[0025] Figure 6 A flowchart of a multimodal model performance evaluation method based on reinforcement and complementary space search according to an embodiment of the present invention is shown;

[0026] Figure 7 A structural block diagram of a disturbance sample pair generating device according to an embodiment of the present invention is shown;

[0027] Figure 8 A structural block diagram of a multimodal model performance evaluation device based on enhanced and complementary space search according to an embodiment of the present invention is shown;

[0028] Fig. 9 A block diagram of an electronic device suitable for implementing a method for generating a disturbance sample pair according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] Vision-language pre-trained models are usually pre-trained on a large number of image-text pairs and learn cross-modal representations so that the model can understand and associate information in images and text. Due to the emergence of various adversarial perturbations, the adversarial robustness of vision-language pre-trained models has attracted attention. The purpose of security assessment is to identify and quantify the vulnerability of the model in the face of adversarial perturbations. Therefore, the security of vision-language pre-trained models can be revealed by designing perturbation methods. Collaborative perturbation is a perturbation method designed specifically for multimodal models. Collaborative perturbation achieves a higher perturbation success rate on multimodal models by exploiting the inherent consistency and complementarity between modalities, thereby achieving a high perturbation success rate on multimodal models. Studies have shown that vision-language pre-trained models are more sensitive to collaborative perturbations than single modal ones.

[0034] Current research on the collaborative perturbation of visual-language pre-trained models mainly focuses on white-box perturbations and transfer-based perturbations. These perturbations require all or part of the prior knowledge of the perturber's visual-language pre-trained model. It is challenging to obtain the structural knowledge of the black-box victim model, and it is difficult for perturbations between images and texts to collaborate effectively. Currently, only the downstream task outputs of the visual-language pre-trained model can be accessed. In addition, the existing visual-language pre-trained collaborative perturbation methods iteratively optimize adversarial samples on a set of image-text pairs, which may ignore other potential perturbation targets and lead to local overfitting. The overfitting problem mentioned in the present invention refers to the fact that adversarial samples focus too much on salient features at the expense of details, which will affect the generalization ability of adversarial samples on tasks and models. This is because these methods use embedding guidance and self-enhancement to increase the diversity of adversarial examples on the optimization path to increase the transferability of adversarial. However, since they only search for optimization targets in fixed image-text pairs, the performance and generalization ability of perturbations are relatively weak.

[0035] In view of this, an embodiment of the present invention provides a method for generating a perturbation sample pair, the method comprising: obtaining a sample pair and a matching label of the sample pair, the sample pair comprising an image and a text, the matching label representing the consistency of the image and the text; for the sample pair, repeatedly performing the following operations until the perturbation strategy meets a preset condition: for each of a plurality of perturbers, adding perturbation data to the image and the text respectively, obtaining a perturbed image and a perturbed text of each of the plurality of perturbers, the perturbation data being generated based on the perturbation strategy; inputting the perturbation images and perturbation texts of each of the plurality of perturbers into a multimodal model respectively, obtaining a plurality of perturbation matching data; adjusting the perturbation strategy based on a target reward value determined by the plurality of perturbation matching data and the matching label; and determining the perturbation image and the perturbation text when the perturbation strategy meets the preset condition as a target perturbation sample pair.

[0036] Figure 1 A diagram showing application scenarios of a method for generating perturbation sample pairs, a method for evaluating multimodal model performance based on reinforcement and complementary space search, and a device according to an embodiment of the present invention.

[0037] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0038] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0040] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0041] It should be noted that the perturbation sample pair generation method and the multimodal model performance evaluation method based on enhanced and complementary space search provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the perturbation sample pair generation device and the multimodal model performance evaluation device based on enhanced and complementary space search provided in the embodiment of the present invention can generally be set in the server 105. The perturbation sample pair generation method and the multimodal model performance evaluation method based on enhanced and complementary space search provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the perturbation sample pair generation device and the multimodal model performance evaluation device based on enhanced and complementary space search provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0043] The following will be based on Figure 1 The scene described by Figure 2~Figure 5The disturbance sample pair generation method according to the embodiment of the present invention is described in detail.

[0044] Figure 2 A flow chart of a method for generating perturbation sample pairs according to an embodiment of the present invention is shown.

[0045] like Figure 2 As shown, the perturbation sample pair generation method of this embodiment includes operations S210 to S250.

[0046] In operation S210 , sample pairs and matching labels of the sample pairs are obtained.

[0047] The sample pairs include images and texts, and the matching labels represent the consistency between the images and texts.

[0048] For the sample pair, the following S210 to S240 are repeatedly executed until the disturbance strategy meets the preset conditions:

[0049] In operation S220, for each of the multiple disturbers, disturbance data is added to the image and the text respectively to obtain the disturbed images and disturbed texts of the multiple disturbers respectively.

[0050] Among them, the perturbation data is generated based on the perturbation strategy.

[0051] According to an embodiment of the present invention, the process of finding the best perturbation strategy can be modeled as a strategy search process in a complementary space according to a Markov decision process.

[0052] The process of finding the optimal perturbation strategy can be modeled by the following formula (1).

[0053]

[0054] Among them, S is the state set, A is the action set, is the perturbation strategy, R is the reward function, γ is the discount rate, I is the perturbation search space, and t represents the finite number of steps.

[0055] For the multimodal model, the state set S contains the feature information of the image, the feature information of the text, and the interaction information between the image and the text. The state set S can be described by the following formula (2).

[0056]

[0057] in, Represents the characteristic information of the image. Represents the feature information of the text, Represents the interaction information between two modalities.

[0058] According to an embodiment of the present invention, considering the difference in features between images and texts, the perturbation of the multimodal model is divided into two parts. The image and text are perturbed simultaneously by adding perturbation variables. The perturbation performed by each perturber can be expressed by the following formula (3).

[0059]

[0060] in, Represents the perturbation of the image, and the perturbation image can be expressed as , Represents the perturbation of the text, and the perturbation text can be expressed as .

[0061] In operation S230, the disturbed images and disturbed texts of the multiple disturbers are respectively input into the multimodal model to obtain a plurality of disturbance matching data.

[0062] Figure 3 A schematic diagram showing comparison between disturbance results generated by multiple disturbers and sample pairs according to an embodiment of the present invention is shown.

[0063] like Figure 3 As shown, the figure may include a sample pair 310, perturbation data 320 generated by a first perturber, and perturbation data 330 generated by a second perturber; the sample pair 310 includes an image 311 and a text “This is a cat lying on the lawn” 312; the perturbation data 320 generated by the first perturber includes a first perturbed image 321 and a first perturbation text “This is a cat sitting on the lawn” 322; the perturbation data 330 generated by the second perturbator includes a second perturbation image 331 and a second perturbation text “This is a cat lying on the lawn” 332. The perturbation data 320 generated by the first perturber and the perturbation data 330 generated by the second perturbator are respectively input into the multimodal model to obtain first perturbation matching data and second perturbation matching data.

[0064] In operation S240 , the perturbation strategy is adjusted based on the plurality of perturbation matching data and the target reward value determined by the matching tags.

[0065] According to an embodiment of the present invention, the first disturbance matching data and the second disturbance matching data obtained as above can be calculated respectively by a preset reward function to obtain a first disturbance reward value and a second disturbance reward value, and a target reward value is selected according to the above two reward values, so as to adjust the disturbance strategy.

[0066] In operation S250 , the disturbed image and the disturbed text under the condition that the perturbation strategy satisfies a preset condition are determined as a target perturbation sample pair.

[0067] According to an embodiment of the present invention, multiple disturbers add disturbances generated by a perturbation strategy to images and texts respectively, to obtain disturbed images and perturbated texts of the multiple disturbers respectively, and the perturbated images and perturbated texts of the multiple disturbers respectively are input into a multimodal model to obtain multiple perturbation matching data, and the perturbation strategy in the perturbation multimodal model is optimized according to a target reward value determined by the perturbation data and the matching label, thereby optimizing the collaborative perturbation in the perturbation process and improving the generalization effect of the generated perturbation sample pairs.

[0068] According to an embodiment of the present invention, the above-mentioned perturbation matching data includes similarity and perturbation matching results, and the similarity represents the degree of consistency between the image and the text. The method also includes: for each disturber, using an alignment reward function, determining a sub-alignment reward value based on the similarity; determining a sub-perturbation reward value based on the perturbation matching result and the matching label; determining a candidate reward value of the disturber based on the sub-alignment reward value and the sub-perturbation reward value; and determining a target reward value based on the candidate reward values ​​of each of the multiple disturbers.

[0069] The candidate reward value for each perturbator can be described by the following formula (4).

[0070]

[0071] in, is the sub-alignment reward value, is the sub-perturbation reward value, It can be calculated by the following formula (5).

[0072]

[0073] in, represents the perturbation of the image, represents the perturbation of the text, represents the probability of data alignment after applying perturbations to the image and text respectively, It means minimizing the alignment loss, that is, maximizing the alignment probability.

[0074] According to an embodiment of the present invention, the sub-perturbation reward value may represent the error rate of the multimodal model under perturbation.

[0075] The overall environment of images, texts and their interaction can be expressed by the following formula (6):

[0076]

[0077] Among them, I represents an overall environment including images, texts and their interaction process. v Represents the image environment, I t Indicates the text environment, I c Represents the environment of the interaction process.

[0078] According to an embodiment of the present invention, in order to determine the target reward value from the candidate reward values, the state and action set can be recorded for each action of each perturber. The perturbation strategy can be regarded as the probability of taking action A under state S, which can be expressed by the following formula (7).

[0079]

[0080] Among them, s 0 Indicates the state of step 0, a 0 Indicates in s 0 The action taken in the state, s 1 Indicates the state of step 1, a 1 Indicates in s 1 Actions taken in a state.

[0081] According to an embodiment of the present invention, a sub-alignment reward value is determined based on similarity through an alignment reward function to evaluate the semantic consistency between an image and a text, and a sub-perturbation reward value is determined based on the perturbation matching result and the matching label, thereby evaluating the perturbation result of the perturber, and combining the sub-alignment reward value and the sub-perturbation reward value to determine the target reward value to optimize the generated sample pairs, thereby enabling a full performance test of the multimodal model.

[0082] According to an embodiment of the present invention, each of the above-mentioned disturbers has a preset disturbance iteration value, and the preset conditions include: the number of iterations reaches the preset disturbance iteration value or the disturbance strategy makes the target reward value converge.

[0083] According to an embodiment of the present invention, each of the above-mentioned disturbers is assigned a preset disturbance iteration value, which represents the number of steps that can be used in the entire search process. At the beginning of the search, each disturber is assigned a preset iterative disturbance value, which can be used freely before the end of the disturbance. That is, for each disturbance of the disturber, it is not necessary for each disturbance to reach the alignment state, as long as the maximum disturbance rate is achieved before reaching the alignment state.

[0084] According to an embodiment of the present invention, the perturbation strategy includes an image perturbation direction, a text perturbation direction and a strategy distribution parameter, the image perturbation direction represents the probability of adding perturbations to random sub-regions in the image, the text perturbation direction represents the probability of adding perturbations to random sub-texts in the text, the random sub-regions and the random sub-texts have a corresponding relationship, the strategy distribution parameter is used to control the consistency difference between the image perturbation caused by the image perturbation direction and the text perturbation caused by the text perturbation direction, and the perturbation strategy is adjusted based on the target reward value, including: determining the image perturbation direction and the text perturbation direction based on the sub-alignment reward value of the target reward value; adjusting the strategy distribution parameter based on the sub-disturbance reward value of the target reward value to obtain an adjusted strategy distribution function.

[0085] According to the embodiment of the present invention, the disturbance strategy can be expressed by the following formula (8).

[0086]

[0087] Among them, d v represents the image perturbation direction, d l represents the text perturbation direction, p v represents the image perturbation intensity, p l represents the text perturbation strength, and θ represents the strategy distribution parameter.

[0088] According to the embodiment of the present invention, the tendency of strategy selection can be changed by adjusting the value of the strategy distribution parameter θ, thereby optimizing the disturbance effect, which helps to find the optimal disturbance strategy in different scenarios.

[0089] According to an embodiment of the present invention, the above-mentioned adding disturbance data to the image and text respectively to obtain the disturbance image and the disturbance text includes: adding random noise to the image to obtain the disturbance image; performing synonym replacement or word embedding on the sub-text in the text to obtain the disturbance text.

[0090] According to an embodiment of the present invention, the above-mentioned image perturbation can also be performed on the features of the image extracted by the neural network, so as to change the feature distribution. The image perturbation direction can also be performed on the color, texture and other directions in the image. The present invention does not impose any restrictions on this.

[0091] According to an embodiment of the present invention, the above-mentioned text disturbance direction may be, for example, determining to perform synonym replacement or word embedding on the text, adjusting the sentence structure, etc., and the present invention does not impose any limitation on this.

[0092] According to the embodiment of the present invention, the image disturbance intensity and the text disturbance intensity can be adjusted adaptively, and can also be adjusted according to demand.

[0093] According to an embodiment of the present invention, the sub-region where the disturbance is generated in the image is controlled by the image disturbance direction in the disturbance strategy, and a disturbance image is generated in the sub-region by adding random noise; the sub-text to which disturbance needs to be added is determined by the text disturbance direction in the disturbance strategy, and the sub-text is disturbed by synonym replacement or word embedding, so that diversified disturbance data can be quickly generated, and the consistency difference between the image disturbance and the text disturbance can be controlled by the strategy distribution in the disturbance strategy, so as to improve the collaborative disturbance effect of the image and the text.

[0094] According to an embodiment of the present invention, adding random noise to an image to obtain a disturbed image includes: determining a target sub-region of the image based on an image disturbance direction; and adding noise to the target sub-region to obtain a disturbed image.

[0095] For example, in the above Figure 3 When performing perturbations, it is found that adding noise to the area where the cat is located in the image and replacing the verb in "This is a cat lying on the lawn" will result in a higher sub-alignment reward value. In the next iteration, it will be more inclined to determine the target sub-area as the area where the cat is located in the image, and determine the verb "lie" in the text as the target sub-text for adjustment.

[0096] According to the embodiment of the present invention, the corresponding cumulative reward will be calculated. Our ultimate goal is to find a perturbation strategy that maximizes the reward, that is, the optimal perturbation effect, which can be expressed by the following formula (9).

[0097]

[0098] in, represents the perturbation strategy, R(τ) represents the cumulative reward of trajectory τ, and pπ(τ) represents the perturbation strategy Under this condition, the probability of trajectory τ appearing is dπ, and dπ represents the state distribution under strategy π.

[0099] In order to obtain the above perturbation strategy, for each perturbator, the search state S=0 is initialized, starting from the last time step Start forward derivation. When calculating the perturbation strategy, we are actually calculating the optimal perturbation strategy for step t and each state in the strategy The sub-perturbation reward value can be expressed as xrj, where , the sub-perturbation reward value represents the loss caused by the perturber j when perturbing the perturbed r. In order to concisely express the formula, we stack xrj into an mn-dimensional vector , that is, the strategy distribution parameter, in which the perturbed state is transferred to The probability of ,in, In step s t The true state of all perturbed entities in , and the probability is between 0 and 1, in order to optimize To maximize the specific true state vector Next, from arrive The optimal perturbation for a single time step t can be expressed by the following formulas (10)~(13):

[0100]

[0101] in, Indicates that given the policy distribution parameter θ, the state and state transfer In the case of transfer to the state The probability of Indicates in status The optimal value function in this case, T represents the total time step, and the above formula (10) must satisfy the conditions shown in formulas (11) to (13).

[0102] Figure 4 A schematic diagram of an architecture according to an embodiment of the present invention is shown.

[0103] like Figure 4 As shown, real images and real sample pairs are input into the multimodal model and processed by different modules therein. The step of finding the target perturbation strategy is regarded as finding the problem of maximizing the target reward value through a finite number of steps in the complementary space established by the image and the text. The search space is gradually compressed, and the perturbation strategy is optimized through inter-step and intra-step constraints to determine the target perturbation strategy and generate perturbation sample pairs for the multimodal model.

[0104] Figure 5 An example diagram of sample pair generation according to an embodiment of the present invention is shown.

[0105] like Figure 5 As shown, the upper picture and the real description as "a businessman wearing a yellow tie with a frustrated expression" are input into the method according to the embodiment of the present invention for perturbation iteration. It can be seen that in step 10, the "businessman" in the text is changed to "person", in step 24, the "yellow" in the text is changed to "black", in step 45, "exposed" is changed to "dressed", and in step 91, "expression" is changed to "suit". The lower picture and the real description as "a man standing on a ladder to clean windows" are input into the method according to the embodiment of the present invention for iteration. It can be seen that in step 9, the "man" in the text is changed to "person"; in step 36, "clean" is changed to "wipe", in step 60, it is changed from "wipe" to "hit", and in step 79, it is changed from "hit" to "hit".

[0106] Figure 6 A flowchart of a multimodal model performance evaluation method based on enhanced and complementary space search according to an embodiment of the present invention is shown.

[0107] like Figure 6 As shown, the perturbation sample pair generation method of this embodiment includes operations S610 to S620.

[0108] In operation S610 , a target perturbation sample pair is obtained.

[0109] The target perturbation sample pair includes target perturbation image data and target perturbation text data, and the target perturbation sample pair is obtained by using the above-mentioned perturbation sample pair generation method.

[0110] In operation S620, the performance of the multimodal model is evaluated using the target perturbation sample pair to obtain a performance evaluation result.

[0111] Among them, the performance evaluation results are used to improve the security of the multimodal model.

[0112] According to an embodiment of the present invention, by inputting a perturbation sample pair into a multimodal model, recording the recognition result of the multimodal model and comparing it with the correct result of the original sample to calculate the accuracy, and recording the stability of the model after inputting the perturbation sample pair to determine the robustness of the model, the security vulnerabilities of the multimodal model can be found to optimize and improve the multimodal model in a targeted manner, thereby increasing the security of the multimodal model.

[0113] Based on the above perturbation sample pair generation method, the present invention also provides a perturbation sample pair generation device. Figure 7 The device is described in detail.

[0114] Figure 7 A structural block diagram of a disturbance sample pair generating device according to an embodiment of the present invention is shown.

[0115] like Figure 7 As shown, the disturbance sample pair generating device 700 of this embodiment includes an acquisition module 710 , a disturbance data adding module 720 , an input module 730 , an adjustment module 740 and a determination module 750 .

[0116] The acquisition module 710 is used to acquire sample pairs and matching labels of the sample pairs, the sample pairs include images and texts, and the matching labels represent the consistency of the images and texts. In one embodiment, the acquisition module 710 can be used to perform the operation S210 described above, which will not be described in detail here.

[0117] The disturbance data adding module 720 is used to add disturbance data to the image and text for each of the multiple disturbers, respectively, to obtain the disturbed images and disturbed texts of the multiple disturbers, and the disturbance data is generated based on the disturbance strategy. In one embodiment, the disturbance data adding module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0118] The input module 730 is used to input the disturbance images and disturbance texts of the multiple disturbers into the multimodal model to obtain multiple disturbance matching data. In one embodiment, the input module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0119] The adjustment module 740 is used to adjust the perturbation strategy based on the target reward value determined by the plurality of perturbation matching data and the matching tags. In one embodiment, the adjustment module 740 can be used to perform the operation S240 described above, which will not be described in detail here.

[0120] The determination module 750 is used to determine the disturbed image and the disturbed text as a target disturbed sample pair when the disturbance strategy meets the preset conditions. In one embodiment, the determination module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0121] According to an embodiment of the present invention, any multiple modules of the acquisition module 710, the disturbance data adding module 720, the input module 730, the adjustment module 740 and the determination module 750 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the disturbance data adding module 720, the input module 730, the adjustment module 740 and the determination module 750 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 710 , the disturbance data adding module 720 , the input module 730 , the adjustment module 740 , and the determination module 750 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0122] Based on the above multimodal model performance evaluation method based on enhanced and complementary space search, the present invention also provides a multimodal model performance evaluation device based on enhanced and complementary space search. Figure 8 The device is described in detail.

[0123] Figure 8 A structural block diagram of a multimodal model performance evaluation device based on enhanced and complementary space search according to an embodiment of the present invention is shown.

[0124] like Figure 8 As shown, the multimodal model performance evaluation device 800 based on enhanced and complementary space search of this embodiment includes a target sample pair acquisition module 810 and a performance evaluation module 820.

[0125] The target sample pair acquisition module 810 is used to acquire a target perturbation sample pair, the target perturbation sample pair includes target perturbation image data and target perturbation text data, and the target perturbation sample pair is obtained using the perturbation sample pair generation method described above. In one embodiment, the target sample pair acquisition module 810 can be used to perform the operation S610 described above, which will not be repeated here.

[0126] The performance evaluation module 820 is used to use the target perturbation sample pair to perform a performance evaluation on the multimodal model to obtain a performance evaluation result, and the performance evaluation result is used to improve the security of the multimodal model. In one embodiment, the performance evaluation module 820 can be used to perform the operation S620 described above, which will not be repeated here.

[0127] According to an embodiment of the present invention, any multiple modules in the target sample pair acquisition module 810 and the performance evaluation module 820 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the target sample pair acquisition module 810 and the performance evaluation module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the target sample pair acquisition module 810 and the performance evaluation module 820 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0128] Fig. 9 A block diagram of an electronic device suitable for implementing a method for generating a disturbance sample pair according to an embodiment of the present invention is shown.

[0129] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0130] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0131] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0133] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0134] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the perturbation sample pair generation method provided by the embodiment of the present invention.

[0135] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0136] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0137] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0138] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0141] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for generating perturbation sample pairs, characterized in that: The method comprises: Acquire a sample pair and a matching label of the sample pair, the sample pair comprising an image and a text, the matching label representing the consistency of the image and the text; For the sample pair, the following operations are repeated until the perturbation strategy meets the preset conditions: For each of the multiple disturbers, adding disturbance data to the image and the text respectively to obtain a disturbed image and a disturbed text of each of the multiple disturbers, wherein the disturbance data is generated based on the disturbance strategy; Inputting the disturbance images and the disturbance texts of the multiple disturbers into a multimodal model respectively to obtain a plurality of disturbance matching data; Adjusting the perturbation strategy based on the plurality of perturbation matching data and the target reward value determined by the matching tags; The disturbed image and the disturbed text when the perturbation strategy satisfies the preset condition are determined as a target perturbation sample pair.

2. The method according to claim 1, characterized in that The disturbance matching data includes similarity and disturbance matching results, wherein the similarity represents the consistency between the image and the text, and the method further includes: For each of the perturbers, Determining a sub-alignment reward value based on the similarity using an alignment reward function; Determining a sub-perturbation reward value based on the perturbation matching result and the matching label; determining a candidate reward value for the perturbator based on the sub-alignment reward value and the sub-disturbance reward value; The target reward value is determined based on candidate reward values ​​for each of the plurality of disturbers.

3. The method according to claim 2, characterized in that The perturbation strategy includes an image perturbation direction, a text perturbation direction and a strategy distribution parameter, wherein the image perturbation direction represents the probability of adding perturbation to a random sub-region in the image, and the text perturbation direction represents the probability of adding perturbation to a random sub-text in the text, wherein the random sub-region and the random sub-text have a corresponding relationship, and the strategy distribution parameter is used to control the consistency difference between the image perturbation caused by the image perturbation direction and the text perturbation caused by the text perturbation direction, and the perturbation strategy is adjusted based on the target reward value, including: Determining the image perturbation direction and the text perturbation direction based on the sub-alignment reward value of the target reward value; The policy distribution parameter is adjusted based on the sub-disturbance reward value of the target reward value to obtain an adjusted policy distribution parameter.

4. The method according to claim 1, characterized in that: Each of the perturbators has a preset perturbation iteration value, and the preset conditions include: The number of iterations reaches the preset disturbance iteration value or the disturbance strategy makes the target reward value converge.

5. The method according to claim 3, characterized in that: The adding disturbance data to the image and the text respectively to obtain the disturbance image and the disturbance text comprises: Adding random noise to the image to obtain the disturbed image; The perturbed text is obtained by performing synonym replacement or word embedding on the sub-text in the text.

6. The method according to claim 5, characterized in that The adding random noise to the image to obtain the disturbed image comprises: determining a target sub-region of the image based on the image disturbance direction; Noise is added to the target sub-region to obtain the disturbed image.

7. A multimodal model performance evaluation method based on reinforcement and complementary space search, characterized in that: The method comprises: Obtaining a target perturbation sample pair, wherein the target perturbation sample pair includes target perturbation image data and target perturbation text data, and the target perturbation sample pair is obtained by using the method described in any one of claims 1 to 6; The target perturbation sample pairs are used to perform a performance evaluation on the multimodal model to obtain a performance evaluation result, and the performance evaluation result is used to adjust the multimodal model.

8. A disturbance sample pair generating device, characterized in that: The device comprises: An acquisition module, used to acquire a sample pair and a matching label of the sample pair, wherein the sample pair includes an image and a text, and the matching label represents the consistency of the image and the text; A disturbance data adding module, used for adding disturbance data to the image and the text respectively for each of the multiple disturbers, so as to obtain the disturbance image and the disturbance text of each of the multiple disturbers, wherein the disturbance data is generated based on the disturbance strategy; An input module, used for inputting the disturbance images and the disturbance texts of the multiple disturbers into the multimodal model to obtain multiple disturbance matching data; An adjustment module, configured to adjust the perturbation strategy based on the plurality of perturbation matching data and a target reward value determined by the matching tags; A determination module is used to determine the disturbed image and the disturbed text as a target disturbance sample pair when the disturbance strategy meets a preset condition.

9. A multimodal model performance evaluation device based on enhanced and complementary space search, characterized in that: The device comprises: A target sample pair acquisition module, used to acquire a target perturbation sample pair, wherein the target perturbation sample pair includes target perturbation image data and target perturbation text data, and the target perturbation sample pair is obtained by using the method described in any one of claims 1 to 6; The performance evaluation module is used to use the target perturbation sample pair to perform performance evaluation on the multimodal model to obtain a performance evaluation result, and the performance evaluation result is used to improve the security of the multimodal model.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

Images