Adversarial sample generation method and device and server

Abstract

The specification provides a method, device and server for generating an adversarial sample. Before implementation, an initial multi-track model containing multiple initial sub-network models with different lengths can be obtained by splitting and combining an initial model matched with a target model; a target multi-track model containing multiple sub-network models with different lengths can be obtained by training the initial multi-track model; and a target model structure for conducting adversarial attack test on the target model can be determined from the target multi-track model, which integrates model blocks for processing low-level features and reduces the influence of model blocks for processing high-level features. In the implementation, the target model structure in the target multi-track model can be used to generate an adversarial sample with good transferability, so that the target model can be tested for security more effectively using the adversarial sample.

Description

TECHNICAL FIELD

[0001] The present specification belongs to the technical field of deep learning, and particularly relates to an adversarial sample generation method, device and server. BACKGROUND

[0002] Although a deep neural network model (DNN) trained by deep learning generally has good model performance, the robustness of the DNN to adversarial samples is often poor.

[0003] For example, if a relatively small and imperceptible adversarial noise is added to the input, the model is likely to misclassify the input with the added adversarial noise with high confidence when processing, resulting in a high security risk of the trained model.

[0004] Therefore, it is necessary to generate and use appropriate adversarial samples to evaluate and improve the robustness of the model by performing black-box attack testing on the model.

[0005] At present, there is an urgent need for a method capable of generating an adversarial sample with good transferability. SUMMARY

[0006] The present specification provides an adversarial sample generation method, device and server, which can generate an adversarial sample with good transferability.

[0007] The adversarial sample generation method, device and server provided by the present specification are implemented as follows:

[0008] An adversarial sample generation method comprises: receiving an adversarial sample generation request for a target model; constructing a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is further connected with a corresponding output module; determining a target model structure from the target multi-track model; wherein the target model structure is a model structure used for adversarial attack testing of the target model; and generating an adversarial sample for the target model using the target model structure.

[0009] A method for obtaining a target multi-track model corresponding to a target model comprises: obtaining and splitting an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models comprise the same input module; combining the plurality of initial sub-network models to obtain an initial multi-track model; and training the initial multi-track model to obtain a target multi-track model corresponding to the target model.

[0010] An apparatus for generating an adversarial sample, comprising: a receiving module configured to receive an adversarial sample generation request for a target model; a constructing module configured to construct a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is further connected with a corresponding output module; a determining module configured to determine a target model structure from the target multi-track model; wherein the target model structure is a model structure for adversarial attack test of the target model; and a generating module configured to generate an adversarial sample for the target model by using the target model structure.

[0011] An apparatus for obtaining a target multi-track model corresponding to a target model, comprising: a splitting module configured to obtain and split an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models comprise a same input module; a combining module configured to combine the plurality of initial sub-network models to obtain an initial multi-track model; and a training module configured to train the initial multi-track model to obtain the target multi-track model corresponding to the target model.

[0012] A server, comprising a processor and a memory for storing processor-executable instructions, the processor implementing relevant steps of the method for generating an adversarial sample and the method for obtaining a target multi-track model corresponding to a target model when executing the instructions.

[0013] A computer-readable storage medium having computer instructions stored thereon, the instructions implementing relevant steps of the method for generating an adversarial sample and the method for obtaining a target multi-track model corresponding to a target model when executed.

[0014] The method, device and server for generating an adversarial sample provided by the specification can first obtain an initial multi-track model comprising a plurality of initial sub-network models with different lengths by splitting and combining an initial model with a series structure matched with a target model before implementation; then obtain a target multi-track model comprising a plurality of sub-network models with different lengths by training the initial multi-track model; further, the target model structure capable of being used for adversarial attack test of the target model can be determined from the target multi-track model, which integrates a model block for processing low-level features and reduces the interference of a model block for processing high-level features. When implemented, the adversarial sample with good migration can be generated by using the target model structure in the target multi-track model, so that the target model can be tested for security more effectively by using the adversarial sample. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the specification, the drawings required in the embodiments will be briefly introduced as follows. The drawings described below are only some embodiments described in the specification, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0016] Figure 1 is a flowchart of the method for generating an adversarial sample provided by an embodiment of the specification;

[0017] Figure 2 is a schematic diagram of one embodiment of the method for generating an adversarial sample provided by the specification in a scene example;

[0018] Figure 3 is a schematic diagram of one embodiment of the method for generating an adversarial sample provided by the specification in a scene example;

[0019] Figure 4 is a schematic diagram of one embodiment of the method for generating an adversarial sample provided by the specification in a scene example;

[0020] Figure 5 is a flowchart of the method for generating an adversarial sample provided by an embodiment of the specification;

[0021] Figure 6 is a structural composition schematic diagram of a server provided by an embodiment of the specification;

[0022] Figure 7 is a structural composition schematic diagram of a device for generating an adversarial sample provided by an embodiment of the specification;

[0023] Figure 8FIG. 1 is a schematic diagram of an embodiment of a method for generating an adversarial sample according to an embodiment of the present specification;

[0024] Figure 9 FIG. 1 is a schematic diagram of an embodiment of a method for generating an adversarial sample according to an embodiment of the present specification;

[0025] Figure 10 FIG. 1 is a schematic diagram of an embodiment of a method for generating an adversarial sample according to an embodiment of the present specification. DETAILED DESCRIPTION

[0026] In order to enable persons skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in the specification below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only some of the embodiments of the specification, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by persons skilled in the art without creative labor should be within the scope of protection of the specification.

[0027] The specification considers that the adversarial samples generated based on the conventional model generally have poor transferability. In actual implementation, when such adversarial samples are used to perform black-box attack (or adversarial attack test) on a target model of interest, the attack is often relatively weak, the test effect is relatively poor, and thus the robustness of the model to the adversarial sample cannot be accurately evaluated.

[0028] Further, the specification analyzes the specific reasons for the above problems. Specifically, the effectiveness of the black-box attack based on transfer depends on the cross-model transferability of the adversarial samples generated by the model. The above transferability depends more on the low-level features (or low-level features) shared between models. In the process of generating adversarial samples based on transfer using the conventional model, it is easy to overfit to the high-level features (or high-level features) of the model, and the shared low-level features are ignored, thereby resulting in poor transferability of the adversarial samples obtained by using the above conventional model, weak attack on the target model, and unsatisfactory test effect.

[0029] Based on the above analysis, the specification further proposes that the model architecture of the model responsible for generating the adversarial samples can be restructured. Specifically, the original serially connected general model can be split into multiple sub-network models with relatively short lengths, wherein the multiple sub-network models contain the same input module. Then, the multiple sub-network models are recombined in a parallel manner, so as to selectively fuse the model blocks for processing low-level features, while reducing the interference and influence of the model blocks for processing high-level features, thereby obtaining a target multi-track model containing multiple sub-network models with different lengths corresponding to the target model. Then, through relevant tests, a single sub-network model or a combination of multiple sub-network models can be determined from the target multi-track model as a target model structure for adversarial attack testing of the target model. Further, the target model structure can be used to generate adversarial samples with good migration and strong attack for the target model, so that subsequent black-box attacks on the target model can be performed using the above adversarial samples to effectively test the security of the target model.

[0030] Referring to Figure 1 The embodiment of the specification provides an adversarial sample generation method. The method is specifically applied to the server side. In specific implementation, the method can include the following contents.

[0031] S101: receiving an adversarial sample generation request for a target model.

[0032] S102: constructing a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model includes multiple sub-network models with different lengths; the multiple sub-network models are connected with a common input module; the sub-network model contains multiple model blocks in series; and the end of the sub-network model is further connected with a corresponding output module.

[0033] S103: determining a target model structure from the target multi-track model; wherein the target model structure is a model structure for adversarial attack testing of the target model.

[0034] S104: generating an adversarial sample for the target model using the target model structure.

[0035] In some embodiments, the above target model can be specifically understood as a model to be tested by a black-box attack (or adversarial attack) and the like for security testing, so as to evaluate and improve the robustness of the model to adversarial samples.

[0036] Specifically, the above target model can be a neural network model that has been trained by deep learning and can realize one or more business functions.

[0037] In most cases, the model parameters of the target model cannot be directly obtained. The user (or the testing party) often regards the target model as a black box and cannot evaluate the robustness of the model to the adversarial samples by performing a white-box attack on the target model.

[0038] In some embodiments, the target model can include at least one of a picture recognition model, a face detection model, a semantic analysis model, an automatic driving model, and the like.

[0039] It should be noted that the above-mentioned target models are only illustrative. In actual implementation, the target model can also include other business type neural network models trained based on deep learning according to specific application scenarios and business requirements. The present specification does not limit this.

[0040] In some embodiments, when the user (or the testing party) needs to obtain the adversarial samples for the target model, the client disposed on the user side can initiate an adversarial sample generation request for the target model to the server.

[0041] In this embodiment, the server can specifically include a background server applied to the business platform side and capable of realizing data transmission, data processing and the like, and responsible for generating adversarial samples. Specifically, the server can be an electronic device with data operation, storage and network interaction functions. Alternatively, the server can be a software program running in the electronic device and providing support for data processing, storage and network interaction. In this embodiment, the number of servers is not specifically limited. The server can be one server, several servers, or a server cluster formed by several servers.

[0042] In this embodiment, the client can specifically include a front-end electronic device applied to the user side and capable of realizing data acquisition, data transmission and the like. Specifically, the client can be a desktop computer, a tablet computer, a notebook computer, a smart phone and the like. Alternatively, the client can be a software application capable of running in the above-mentioned electronic device. For example, it can be an application program running on a computer.

[0043] In some embodiments, after receiving the adversarial sample generation request for the target model, the server can respond to and construct a target multi-track model corresponding to the target model according to the adversarial sample generation request. Then, the target model structure for performing adversarial attack testing on the target model is determined from the target multi-track model. The target model structure can be used to generate an adversarial sample with good effect for the target model.

[0044] Of course, for the case that the corresponding target multi-track model has been constructed before, the server can also not need to re-construct the target multi-track model, but can directly determine the target model structure for the adversarial attack test on the target model from the existing target multi-track model; and then use the target model structure to generate the adversarial samples.

[0045] In some embodiments, the server can feed back the generated adversarial samples to the client, so that the user can use the above-mentioned adversarial samples to test the security of the target model by performing a black-box attack or an adversarial attack on the target model on the client side. Of course, the server can also not feed back the adversarial samples to the client, but directly use the above-mentioned adversarial samples to test the security of the target model by performing a black-box attack on the target model on the server side, and then feed back the obtained test results to the client.

[0046] In some embodiments, the above-mentioned target multi-track model can be specifically understood as a model constructed based on a multi-track model architecture (MMA). The above-mentioned target multi-track model can be used as a proxy model for simulating the target model.

[0047] Specifically, referring to Figure 2 The above-mentioned target multi-track model can include a plurality of sub-network models. Among them, the plurality of sub-network models can be connected with a common input module; the sub-network models respectively contain a plurality of model blocks connected in series; the end of the sub-network model is also respectively connected with the corresponding output module. Each model block can contain one or more model layers.

[0048] In some embodiments, the above-mentioned target multi-track model can also be understood as a network structure containing a plurality of structure rows and a plurality of structure columns arranged by a plurality of model blocks contained in a plurality of sub-network models. Each structure row contains a plurality of model blocks, and each structure column contains a plurality of model blocks.

[0049] Specifically, for example, referring to Figure 2As shown, the target multi-track model constructed based on the multi-track model architecture can include three structure columns, which can be denoted as a first structure column, a second structure column, and a third structure column from left to right. The first structure column, the second structure column, and the third structure column each include a plurality of model blocks (e.g., Block, SoftMax, etc.) connected in series. The first structure column, the second structure column, and the third structure column are further connected with the same starting model block (e.g., Block at the top left corner) as a common input module. Moreover, the first structure column, the second structure column, and the third structure column are further connected with corresponding output modules (e.g., SoftMax at the bottom) at the end of the structure column. In addition, the model blocks at the same position on different structure columns in the target multi-track model can form a structure row. Referring to Figure 2 As shown, the target multi-track model further includes three structure rows.

[0050] In the above target multi-track model, each structure column connected with an input module and an output module can be a sub-network model in the target multi-track model. Accordingly, the above target multi-track model can include a plurality of sub-network models, such as a first sub-network model, a second sub-network model, and a third sub-network model, etc. The sub-network model can be understood as a model structure formed by a plurality of model blocks (including Block, SoftMax, etc.) connected in series. The sub-network model is included in the target multi-track model, has complete input and output, and can complete corresponding data processing. Moreover, the plurality of sub-network models share the same input module.

[0051] Specifically, referring to Figure 2 As shown, for example, the first sub-network model can be composed of the first structure column, including two Block model blocks and a SoftMax model block connected in series on the first structure column. The output of the first Block model block in the first sub-network model is the input of the second Block model block, and the output of the second Block model block is the input of the SoftMax model block. For another example, the second sub-network model can be composed of the second structure column and the model block at the top left corner connected with the second structure column, including the Block model block at the top left corner and two Block model blocks and a SoftMax model block connected in series on the second structure column.

[0052] In some embodiments, referring to Figure 2As shown, the input module in the target multi-track model can be specifically a model block at the top-left corner (i.e., the model block at the first row and the first column). In this case, the feature level of the features processed by the model blocks in the structure row above the target multi-track model is generally lower than the feature level of the features processed by the model blocks in the structure row below.

[0053] Generally, the model block in the lowest structure row (corresponding to the last Block connected by each sub-network model, e.g., SoftMax) is generally responsible for processing relatively high-level features and outputting the corresponding processing results.

[0054] The above model block (e.g., Block or SoftMax) can specifically be understood as a model structure having at least an input and an output. Specifically, the above model can be a feature extractor, a residual block with skip layers, etc. Each model block can specifically include a model layer or a plurality of cascaded model layers. The above model layer can be a convolutional layer or a pooling layer, etc.

[0055] Further, the above model block itself can be a series structure. Specifically, for example, a certain model block can be a model structure composed of a convolutional layer + a batch norm layer + an activation layer + a convolutional layer + a batch norm layer + an activation layer in series.

[0056] It should be noted that the above output module can include different types of structure modules corresponding to different application scenarios and business types. For example, in the image classification scenario, the above output module can be specifically SoftMax. In other application scenarios, the above output module can also be other types of output functions other than SoftMax. The present specification does not limit this.

[0057] It should be noted that the above listed model blocks are only illustrative. In specific implementation, the above model blocks can also include other types of model structures according to specific application scenarios and processing requirements.

[0058] In addition, different model blocks in the same target multi-track model can be different types of model structures. For example, Block1 in the target multi-track model can be a feature extractor, and Block2 in the target multi-track model can be a residual block, etc.

[0059] In the target multi-track model, the model blocks included in each sub-network model can span multiple structural rows. Each sub-network model can be connected with the same model block at the starting position, which can be referred to as an input module. The total number of model layers included in each sub-network model can be less than a preset number threshold. The last model block connected at the end position of each sub-network model can be referred to as an output module, for example, SoftMax, etc. The preset number threshold can be 20. Of course, in actual implementation, other numerical values (for example, 10, etc.) can also be set as the preset number threshold according to specific accuracy requirements and model characteristics.

[0060] In some embodiments, the model blocks in different sub-network models in the target multi-track model can be relatively independent and not connected. Specifically, for example, refer to Figure 9 Based on the model architecture, the model blocks in different sub-network models do not share features.

[0061] In some embodiments, the model blocks in different sub-network models in the target multi-track model can also be connected for sharing processed features between the model blocks in different sub-network models. Specifically, for example, refer to Figure 2 As shown, the model blocks in a sub-network model can be connected with adjacent model blocks in different sub-network models in the right direction and the right-down direction. For another example, refer to Figure 10 As shown, the model blocks in a sub-network model can also be connected with adjacent model blocks in different sub-network models only in the right direction. Based on this model architecture, the features that can be learned by a single sub-network model can be enriched.

[0062] In some embodiments, specifically, for example, refer to Figure 2 As shown, the target multi-track model can include three structural rows. Each structural row includes three model blocks (for example, Block or SoftMax on each structural row).

[0063] Specifically, in the target multi-track model shown in Figure 2 As shown, the model block at the first row and the first column position is used as the starting connection structure, and three sub-network models of different lengths are connected in parallel in order from small to large, which are referred to as the first sub-network model (corresponding to the link of Head1), the second sub-network model (corresponding to the link of Head2), and the third sub-network model (corresponding to the link of Head3).

[0064] The first sub-network model includes three model blocks (corresponding to a length of 3), which are Block at the first row and first column, Block at the second row and first column, and SoftMax at the third row and first column, which are connected in sequence. The second sub-network model includes four model blocks (corresponding to a length of 4), which are Block at the first row and first column, Block at the first row and second column, Block at the second row and second column, and SoftMax at the third row and second column, which are connected in sequence. The third sub-network model includes five model blocks (corresponding to a length of 5), which are Block at the first row and first column, Block at the first row and second column, Block at the first row and third column, Block at the second row and third column, and SoftMax at the third row and third column, which are connected in sequence.

[0065] It should be noted that, unlike the common model (which can be referred to as Figure 3 ), in the target multi-track model, the sub-network models are connected in a non-serial manner, so the length (or the number of model blocks contained) of each sub-network model is relatively shorter than that of the common model. For a deep neural network model, the model layers at the front usually process low-level features, and the model layers at the back process high-level features. Therefore, for the sub-network models contained in the target multi-track model in this embodiment, since the model length is short, it is difficult to reach the model level that starts to process high-level features, and each model block in each sub-network model is mainly used to process low-level features. The model blocks of the part of the sub-network models with shorter model length (for example, the first sub-network model in Figure 2 ) are even used to process low-level features. For the part of the sub-network models with longer model length (for example, the third sub-network model in Figure 2 ), the model blocks at the back (for example, the last Block) may start to process high-level features, but the number of model blocks used to process high-level features is relatively Figure 3 smaller than that of the common model shown in

[0066] Based on the above model architecture, the problem that the generated adversarial samples have poor transferability due to overfitting of high-level features and ignoring of shared low-level features, which is similar to the common model, can be effectively avoided. The adversarial samples generated by the sub-network models in the target multi-track model or the combination of the sub-network models automatically move towards the vulnerable direction common to multiple models, so that the interference and influence of overfitting can be reduced, and adversarial samples with good transferability and strong attackability can be generated.

[0067] In some embodiments, the model block in each sub-network model can also be connected with the model block adjacent thereto in the specified direction in the other sub-network model.

[0068] Specifically, for example, the model block in each sub-network model can be connected with the model block adjacent thereto in the right direction in the other sub-network model, so as to transmit the processed feature to the connected model block for use.

[0069] For another example, the model block in each sub-network model can also be connected with the model block adjacent thereto in the right-down direction in the other sub-network model, so as to transmit the processed feature to the connected model block for use.

[0070] For still another example, the model block in each sub-network model can also be connected with the model block adjacent thereto in the right direction and the model block adjacent thereto in the right-down direction in the other sub-network model, so as to transmit the processed feature to the connected model block for use.

[0071] It should be noted that the output module in the sub-network model is not connected with the output module in the other sub-network model.

[0072] In some embodiments, when being connected, the model block adjacent to the current model block in the same structure column can be found first according to the output parameter of the current model block (for example, the size of the output feature data is 5x5), and then the model block having the input parameter matched with the output parameter of the current model block (for example, the size of the supported input feature data is also 5x5) is connected.

[0073] Based on the above connection mode of the target multi-track model, on the one hand, the total number of model layers contained in each sub-network model is small, which avoids overfitting of high-level features and pays more attention to learning low-level features; on the other hand, the model block in front (for example, the feature extractor responsible for extracting relatively low-level features) can transmit the extracted features to the model block behind, so as to avoid the decline of the model performance caused by the small capacity of the sub-network model and the lack of model blocks responsible for processing high-level features.

[0074] Specifically, for example, referring to Figure 2 , the Block located at the position of the second row and the first column in the first sub-network model (which can be denoted as Block1-2) is also connected with the adjacent Block located at the position of the second row and the second column in the second sub-network model (which can be denoted as Block2-2) and the SoftMax located at the position of the third row and the second column in the second sub-network model (which can be denoted as SoftMax2-3), respectively. Correspondingly, the feature extracted by the Block1-2 in the first sub-network model will be transmitted to the Block2-2 and the SoftMax2-3 in the second sub-network model for use, respectively.

[0075] In some embodiments, the server may, in implementation, first construct a target multi-track model corresponding to the target model according to the request for generating an adversarial sample; then determine a model structure for performing adversarial attack test on the target model as a target model structure from the target multi-track model, and generate an adversarial sample with good migration for the target model using the target model structure.

[0076] The target model structure may, in particular, be understood as a model structure for performing adversarial attack test on the target model.

[0077] Specifically, the target model structure may be the entire target multi-track model, or a specific sub-network model in the target multi-track model, or a combination of multiple sub-network models in the target multi-track model.

[0078] In some embodiments, the server may first construct a target multi-track model corresponding to the target model.

[0079] In some embodiments, the construction of the target multi-track model corresponding to the target model may, in implementation, include the following: obtaining and splitting an initial model to obtain multiple initial sub-network models of different lengths; wherein the initial model is a model matched with the target model; the multiple initial sub-network models contain the same input module; combining the multiple initial sub-network models to obtain an initial multi-track model; training the initial multi-track model to obtain a target multi-track model corresponding to the target model.

[0080] In some embodiments, the business type (or business function, such as face detection, semantic analysis, etc.) of the target model may, in implementation, be first determined; then a common model with the same or similar business type as the target model is obtained or constructed to simulate the target model as the initial model matched with the target model.

[0081] The initial model may be a machine learning model with the same business type as the target model. Specifically, for example, in the case where the target model is a classification model (the input of the model includes one or more feature information, and the output of the model is a classification result), the obtained initial model matched with the target model is also a classification model. Specifically, the initial model may include but is not limited to VGG16, ResNet18, GoogleNet, ShuffleNetV2, and MobileNet, etc.

[0082] In some embodiments, the data type of the data processed by the initial model may be the same as that of the target model. The data type may be text data, audio data, or image data, etc.

[0083] Specifically, for example, if the target model is a text recognition model and the data being processed is text data, then the matching initial model can also be a text recognition model used to process text data.

[0084] In some embodiments, an existing open-source similar model can also be used as the initial model. Specifically, for example, if the target model is an animal recognition model, other existing open-source animal recognition models can be used as the initial model.

[0085] In some embodiments, some models may be pre-selected as candidate models, and it may be determined whether the candidate models are similar to the target model, so as to identify the candidate models similar to the target model as the initial models.

[0086] In some embodiments, when determining whether a candidate model is similar to a target model, the same set of test samples can be input into both the candidate model and the target model, and the output results of both models on the same set of test samples can be obtained and statistically analyzed. The similarity between the candidate model and the target model, and the specific similarity, is then determined based on the proportion of identical output results (e.g., labels). Finally, the candidate model with the highest similarity to the target model can be selected from multiple candidate models as the initial model.

[0087] Of course, it should be noted that the methods listed above for determining whether a candidate model is similar to the target model are merely illustrative. In practice, other suitable methods may be used to determine whether a candidate model is similar to the target model, depending on the specific circumstances. This specification does not limit such methods.

[0088] In some embodiments, when specifically splitting the initial model, refer to Figure 3 As shown, the initial model's model layers can be divided into multiple interconnected model blocks; see reference. Figure 4 As shown, the interconnected model blocks can then be split into multiple initial sub-network models of different lengths according to the corresponding splitting rules. The initial sub-network models contain only a small number of interconnected model blocks. This avoids the presence of model blocks used for processing high-level features in the initial sub-network models, or reduces the number of model blocks used for processing high-level features in the initial sub-network models, thereby reducing the impact of overfitting high-level features on the transferability of the generated adversarial examples.

[0089] Furthermore, the multiple initial sub-network models that are split off also contain the same input module, which serves as the model block for the common initial connection of the multiple initial sub-network models.

[0090] For specific breakdowns, see, for example, [reference needed]. Figure 8As shown, first, the initial model can be divided into 7 model blocks (including 6 Blocks and 1 SoftMax); then, the 7 model blocks connected in series are divided into 3 segments, respectively denoted as segment 1, segment 2 and segment 3. Among them, each segment contains two originally connected Blocks, and the total number of model layers contained in each segment is less than the preset number threshold 20. The model block at the end of segment 1 (Block2) is connected with a SoftMax to obtain an initial sub-network model, denoted as initial sub-network model 1; the model block at the end of segment 2 (Block4) is connected with a SoftMax, and the model block at the start (Block3) is connected with Block1 in segment 1 to obtain an initial sub-network model, denoted as initial sub-network model 2; the model block at the end of segment 3 (Block6) is connected with a SoftMax, and the model block at the start (Block5) is connected with Block3 in segment 2 and Block1 in segment 1 in turn to obtain an initial sub-network model, denoted as initial sub-network model 3. Thus, the splitting is completed, and multiple initial sub-network models are obtained.

[0091] In some embodiments, when the multiple initial sub-network models are specifically combined, according to the corresponding combination rule, the initial multi-track model can be obtained by combining the multiple initial sub-network models in the following manner: taking the input block (for example, Block1) commonly contained in the multiple initial sub-network models as the starting connection structure, and connecting the multiple initial sub-network models in parallel with different lengths to obtain a network structure arranged by multiple model blocks, which contains multiple structure rows and multiple structure columns, as the initial multi-track model; wherein one structure column contains multiple model blocks, and one structure row contains multiple model blocks.

[0092] In some embodiments, in the above initial multi-track model, the feature level of the features processed by the model blocks contained in the previous structure row is generally lower than the feature level of the features processed by the model blocks contained in the next structure row.

[0093] For example, the model blocks located in the first row are often used to process relatively bottom-layer features (for example, the outline, edge, color, texture and the like of an image); and the model blocks located in the lowermost row are often used to process relatively top-layer features (for example, the outline, nose, eyes and the like of a face in an image).

[0094] In some embodiments, the initial sub-network model at least includes a model block for processing low-layer features.

[0095] In some embodiments, the initial sub-network model can further include model blocks for processing high-level features. However, the initial sub-network model includes a relatively small number of model blocks for processing high-level features, and the processed high-level features are relatively low in level, which can effectively reduce overfitting of the model high-level features.

[0096] In some embodiments, when specifically combined, the initial sub-network models can be connected in parallel in the order of length from small to large, starting from the common model blocks, and then connecting the initial sub-network models with smaller length (including fewer model blocks) first, and then connecting the initial sub-network models with larger length (including more model blocks).

[0097] In some embodiments, when specifically combined, according to requirements, the model blocks in different initial sub-network models can be connected to the model blocks adjacent in the right direction and / or the right-down direction in other initial sub-network models in the periphery, so that the model blocks can transmit the processed features to the connected other model blocks. Specifically, the model blocks in different initial sub-network models can be connected to other model blocks matching the input parameters in other initial sub-network models in the periphery.

[0098] In the initial multi-track model, the model blocks can be connected to the model blocks belonging to different initial sub-network models to share the processed features (e.g., feature maps output by the feature extractor) between the model blocks in different initial sub-network models.

[0099] When specifically combined, according to requirements, the model blocks in different initial sub-network models can not be connected to the model blocks adjacent in the periphery in other initial sub-network models to maintain the relative independence of each initial sub-network model.

[0100] In some embodiments, when specifically implemented, the initial multi-track model can be trained using sample data to obtain a trained multi-track model as a target multi-track model. The target multi-track model includes a plurality of trained sub-network models. In this way, only by training the initial multi-track model, the target multi-track model including a plurality of trained sub-network models can be obtained, thereby effectively reducing the training cost.

[0101] When specifically training the initial multi-track model, the corresponding sample labels can be labeled on the sample data to obtain labeled sample data. For example, in the image classification scenario, the animal name in the image can be marked on the sample image as a sample label to obtain labeled sample data. Then, the initial multi-track model is iteratively trained multiple times using the labeled sample data.

[0102] In each specific iteration of training, refer to Figure 2 As shown, the labeled sample data can be input into the initial multi-track model through the Block (input module) located in the upper left corner, and multiple sub-processing results output by each initial sub-network model based on the labeled sample data can be obtained (e.g., from...). Figure 2 The third row contains three SoftMax outputs representing three confidence prediction scores. These scores are then weighted and summed based on the results of multiple sub-processing steps to obtain the overall model processing result. The loss value is calculated based on the model processing result and the sample labels. The corresponding gradient information is then calculated based on the loss value. Based on the gradient information, the model parameters in the initial multi-track model are adjusted to optimize the model towards minimizing the loss value. By performing multiple iterations of training in the above manner, the target multi-track model can be obtained.

[0103] In some embodiments, when specifically training the initial multi-track model, the training set of the target model can also be obtained; using the training set of the target model, the initial multi-track model is trained to obtain the target multi-track model. Since the sample data used to train the target multi-track model is the same as that used in the target model, the two models can have a high degree of similarity.

[0104] In some embodiments, it is also considered that obtaining a training set for the target model may be difficult in certain situations, and machine learning often requires a large amount of sample data during model training. Since the trainer's training samples are limited, data from commonly used image datasets (e.g., ImageNet, CIFAR-10, etc.) are usually used as part of the sample data to participate in model training when training the target model. Therefore, in some embodiments, commonly used image datasets can also be used as sample data to train the initial multitrack model.

[0105] In some embodiments, when training the initial multi-track model, sample data can be acquired; a similar training set can be constructed by testing the target model using the sample data; and the initial multi-track model can be trained using the similar training set to obtain the target multi-track model. Specifically, when constructing the similar training set, multiple sample data can be used to access the target model to obtain its output; the output of the target model can be used as the label of the sample data; and the corresponding similar training set can be obtained by combining the sample data and its label.

[0106] In some embodiments, after obtaining the target multi-track model, the method may further include: determining a target model structure from the target multi-track model; wherein the target model structure is a model structure used for adversarial attack testing of the target model.

[0107] In some embodiments, the determination of the target model structure from the target multi-track model corresponding to the target model may include the following: processing test samples using the target multi-track model to obtain multiple sets of adversarial test samples; wherein, each set of adversarial test samples corresponds to a sub-network model; and selecting a model structure that meets the requirements from the target multi-track model based on the multiple sets of adversarial test samples as the target model structure.

[0108] When generating adversarial test samples, the test samples can be input into the target multi-track model. The target multi-track model then processes the test samples using multiple sub-network models to output corresponding sub-processing results (e.g., the confidence prediction score output by the SoftMax function of the sub-network model). Based on the sub-processing results, the test samples are adjusted accordingly to obtain adjusted test samples. These adjusted test samples are then re-input into the target multi-track model for processing, resulting in updated sub-processing results output by the sub-network models based on the adjusted test samples. Based on the updated sub-processing results, it is determined whether to adjust the test samples again. This process is repeated for multiple rounds of iterative testing until a predetermined stopping condition is met; the test samples used at this point are then used as the adversarial test samples corresponding to that sub-network model. Following this method, multiple sets of adversarial test samples corresponding to multiple sub-network models can be obtained. Each set of adversarial test samples corresponds to one sub-network model.

[0109] In some embodiments, the test samples used in the first round of iterative testing can be the original samples or test samples generated based on the original samples and perturbation noise.

[0110] The original samples mentioned above can be clean samples without added noise or perturbation, and the labels are known. Specifically, for example, the original samples can be labeled text data, audio data, or image data, etc.

[0111] In some embodiments, the original sample can be represented as x, and the test sample x used in the first round of iterative testing. 0 adv It can be generated according to formula (1), the label can be represented as y, and the disturbance noise can be represented as noise0:

[0112] x 0 adv =x or x 0 adv =x + noise0 (1).

[0113] In some embodiments, the difference between the sub-processing result (e.g., the predicted score of the confidence) of the current round of iterative testing and the label y can be used to determine the gradient information of the test sample used in the current round of iterative testing. Then, the test sample is adjusted based on the gradient information to obtain the adjusted test sample after the current round of iterative testing.

[0114] In some embodiments, the adjusted test sample x j adv The following formula can be used to generate the adjusted test sample x

[0115] (2)

[0116] wherein, L represents a loss function, which can reflect the difference between the sub-processing result of each network sub-model and the label y. The loss function can be a cross-entropy loss function or other types of loss functions. The sub-processing result of the test sample used in the current round of iterative testing and the label y. The loss function can be a cross-entropy loss function or other types of loss functions. represents calculating the gradient information of the test sample used in the current round of iterative testing; a is a weight value; and sign(•) is a sign function.

[0117] In some embodiments, the predetermined stop condition can be determined by detecting whether the test sample used in the current round of iterative testing meets the preset requirements, to determine whether the next round of iterative testing is needed or not, or to determine whether the subsequent rounds of iterative testing are ended, and to use the test sample used in the current round of iterative testing as the final test sample, so that the adversarial test sample can be obtained.

[0118] In some embodiments, an iteration threshold number of times can also be set. When the total number of iterations reaches the threshold number of times, it is determined that the predetermined stop condition is met, and the iteration is stopped to obtain the final test sample as the adversarial test sample.

[0119] The final adversarial test sample obtained by the above method can be understood as an adversarial sample.

[0120] The adversarial sample can be a data obtained by adding certain perturbation data to the original sample. Generally, the adversarial sample and the original sample have no obvious difference from the naked eye, but the machine learning model will identify the adversarial sample as a result different from or even opposite to the original label. It can be understood that the greater the perturbation data, the greater the attack success rate of the generated adversarial sample, but this will make the naked eye distinguish it as a "poisoned" sample.

[0121] Therefore, when generating the adversarial sample, the perturbation data of the current round of iterative test can be limited within a perturbation change range, so that there is no large difference between the adversarial sample and the initial sample.

[0122] In some embodiments, based on the above considerations, a perturbation change range ε can also be set, and when it is detected that the perturbation data of the test sample (or adversarial sample) generated by the current round of iterative test exceeds the range, the perturbation data can be automatically reduced or clamped within the maximum value of the perturbation change range ε. Specifically, for example, the adversarial sample x j adv which can be expressed in the following form:

[0123] x j adv = clip x,ϵ (x j adv )(3)

[0124] wherein, clip x,ϵ (•) is a clipping function for clamping data values greater than a preset value in the adversarial sample to the preset value.

[0125] It should be noted that the above method of determining the adversarial test sample is only used for example illustration, and the present application is not limited thereto, and other gradient-based methods or non-gradient-based methods can also be used to determine the adversarial test sample.

[0126] After obtaining a plurality of groups of adversarial test samples in the above manner, the plurality of groups of adversarial test samples can be used to test the target model respectively, and a plurality of groups of test results corresponding thereto are obtained. Then, according to the test results, a target model structure matching the target model is selected from the plurality of sub-network models.

[0127] Specifically, according to the test results, the adversarial test sample with a higher misrecognition rate (or higher attack success rate) of the target model can be selected from the plurality of groups of adversarial test samples, and the sub-network model corresponding to the group of adversarial test samples is determined as the target model structure matching the target model.

[0128] In some embodiments, in specific implementation, the weight coefficients of the adversarial test samples output by each sub-network model in the target multi-track model can also be configured first; then, according to the weight coefficients and the plurality of adversarial test samples, a combined adversarial test sample for testing the target model is generated by weighted summation.

[0129] For example, referring to Figure 2The combined adversarial test sample can be generated according to the following formula: a1xHead1+a2xHead2+a3xHead3, where a1, a2, and a3 are to-be-determined weight coefficients, and Head1, Head2, and Head3 are the adversarial test samples output by the first sub-network model, the second sub-network model, and the third sub-network model, respectively.

[0130] In specific implementation, different combined adversarial test samples corresponding to different combinations of weight coefficients can be generated by continuously adjusting the weight coefficients. Then, different combined adversarial test samples are used to test the target model, and different test results are obtained. According to the test results, the most suitable combination of weight coefficients can be determined as the target combination of weight coefficients. Then, the multiple sub-network models can be combined according to the target combination of weight coefficients to obtain a target model structure with better effect.

[0131] In some embodiments, the above-mentioned filtering of a model structure meeting the requirements from the target multi-track model as the target model structure can include the following content: constructing a combined adversarial test sample according to the multiple groups of adversarial test samples; respectively testing the target model by using the multiple groups of adversarial test samples and the combined adversarial test sample to collect the misrecognition rates of the target model for each group of adversarial test samples and the combined adversarial test sample; and filtering a model structure corresponding to an adversarial test sample or a combined adversarial test sample with the highest misrecognition rate from the target multi-track model as the target model structure.

[0132] In some embodiments, the adversarial test samples corresponding to two or more sub-network models in the target multi-track model can also be combined to obtain multiple groups of combined adversarial test samples. Each group of combined adversarial test samples corresponds to a combined model structure of a sub-network model. In some cases, the adversarial test samples output by all sub-network models can also be combined to obtain a combined adversarial test sample corresponding to the target multi-track model.

[0133] In specific implementation, the adversarial test samples of multiple sub-network models can be added to obtain a corresponding combined adversarial test sample.

[0134] Then, the target model can be tested by using the above multiple groups of adversarial test samples and multiple groups of combined adversarial test samples respectively, so as to collect the misrecognition rates (or attack success rates, which can be denoted as ASR) of the target model for each group of adversarial test samples and each group of combined adversarial test samples as corresponding multiple groups of test results. Each group of test results corresponds to one sub-network model in the target multi-track model or a combination of multiple sub-network models.

[0135] Further, the group of test results with the highest misrecognition rate can be filtered out by comparison, and one sub-network model in the target multi-track model or a combination of multiple sub-network models corresponding to the group of test results is determined as a required model structure as the target model structure. Thus, a required target model structure capable of generating a target model with better migration and stronger attack can be obtained.

[0136] Specifically, for example, referring to Figure 2 According to the multiple groups of adversarial test samples, the third sub-network model in the target multi-track model can be determined as the target model structure matched with the target model.

[0137] In some embodiments, in order to obtain a target model structure with better effect, the training duration can also be controlled to train the initial multi-track model within a preset duration range. In this way, it can be avoided that the target multi-track model or the sub-network model in the target multi-track model starts to learn a large number of high-level features and increases the interference brought by the high-level features due to the excessively long training duration. The preset duration range can be determined according to historical test records.

[0138] In some embodiments, a preset condition can also be set, and when training the initial target multi-track model, whether the preset condition is reached can be detected after each training is completed. In the case where it is detected that the preset condition is reached, the model training can be triggered to stop. In this way, it can also be avoided that the target multi-track model or the sub-network model in the target multi-track model starts to learn a large number of high-level features and increases the interference brought by the high-level features due to the excessively long training duration. The above preset condition can be specifically that the accuracy difference between the model accuracy of the current training and the model accuracy of the last training is less than a preset accuracy threshold.

[0139] In some embodiments, in order to obtain a target model structure with better effect, when the initial sub-network model is split, the number of model blocks contained in the split initial sub-network model can be controlled.

[0140] Specifically, by acquiring and analyzing the historical test records, it can be found that the target model structure usually found contains 5 to 6 model blocks, and the model blocks contained mostly span 3 to 4 structure rows. It can be seen that if a sub-network model contains too many feature structure quantities, it will inevitably develop in the direction of learning higher layer features, leading to the neglect of low layer features and overfitting of high layer features.

[0141] Therefore, in specific implementation, a suitable preset quantity threshold can also be set in combination with the historical test records described above. Correspondingly, in splitting, the quantity of model blocks contained in the initial sub-network model split out can be controlled to be as small as possible compared to the preset quantity threshold, and then the target model structure with better effect can be more easily determined subsequently.

[0142] In some embodiments, after calling the target model structure matching the target model in the target multi-track model to generate the adversarial sample for the target model, the method, in specific implementation, can further include the following content: using the adversarial sample to perform security testing on the target model.

[0143] Specifically, the server or the client can use the adversarial sample described above to initiate a black-box attack on the target model to perform security testing and obtain corresponding security testing results. According to the security testing results, the robustness of the target model to the adversarial sample can be evaluated. Meanwhile, the target model can also be adjusted and modified in a targeted manner to improve the robustness of the target model to the adversarial sample.

[0144] Thus, the security testing on the target model can be relatively more effective, and a relatively more secure and stable target model can be obtained by targeted improvement of the target model.

[0145] As can be seen from the above, based on the generation method of the adversarial sample provided in the embodiments of the present specification, before specific implementation, an initial model matching the business type of the target model can be split and combined to obtain an initial multi-track model; the initial multi-track model is trained to obtain a target multi-track model containing multiple sub-network models with different lengths; and then the target model structure that can fuse model blocks processing low layer features while reducing the influence and interference of model blocks processing high layer features and can perform adversarial attack testing on the target model can be determined based on the target multi-track model. In this way, in specific implementation, the adversarial sample with better migration and stronger attack can be generated by calling the target model structure matching the target model in the target multi-track model, so as to more effectively perform security testing on the target model.

[0146] In one specific scenario example, the server responsible for generating the adversarial sample of the business platform can use the above-mentioned adversarial sample generation method to serve the user to generate and provide an adversarial sample with better effect.

[0147] In specific implementation, when a user plans to perform security testing on a target model and needs to obtain an adversarial sample for the target model, the client can generate and send an adversarial sample generation request for the target model. The adversarial sample generation request can at least carry a model identifier of the target model.

[0148] After receiving the adversarial sample generation request for the target model, the server can first determine the target model according to the model identifier carried in the adversarial sample generation request; then obtain a target model structure matched with the target model; and then call the target model structure to generate an adversarial sample for the target model.

[0149] In specific implementation, the server can first determine the business type of the target model; then filter a model matched with the business type of the target model from a plurality of existing preset models as an initial model. Next, the server can split the initial model into a plurality of sub-network models with shorter lengths, each of which can include a plurality of Blocks connected in series; and the plurality of sub-network models can share a same Block as a starting Block, i.e., an input module; and each of the plurality of sub-networks is further connected with a corresponding output module at the end. Then, the server can take the Block shared by the plurality of sub-network models as a starting connection point, and connect the plurality of sub-network models with different lengths in parallel in order of the lengths of the initial sub-network models from small to large, to obtain an initial multi-track model. The initial multi-track model is trained using sample data to obtain a target multi-track model containing a plurality of trained sub-network models with different lengths.

[0150] Further, the server can process test samples by calling the target multi-track model to obtain a plurality of groups of adversarial test samples corresponding to the plurality of sub-network models, and a plurality of groups of combined adversarial test samples corresponding to combinations of the plurality of sub-network models. Then, the plurality of groups of adversarial test samples and the plurality of groups of combined adversarial test samples can be used to test the target model by performing a black-box attack on the target model to obtain a plurality of groups of test results. Then, the target model structure matched with the target model can be selected from the target multi-track model according to the plurality of groups of test results.

[0151] Further, the server can use the above-mentioned target model structure to generate an adversarial sample for the target model with better migration and stronger attack; and then send the adversarial sample to the client.

[0152] The user can obtain the adversarial sample provided by the server through the client; and use the adversarial sample to perform a black-box attack on the target model, so as to realize the security test of the target model.

[0153] Referring to Figure 5 As shown in the drawings, the embodiments of the present specification also provide a method for obtaining a target multi-track model corresponding to a target model, so as to obtain a target model structure capable of generating an adversarial sample with good effect for the target model. When the method is implemented, the following contents can be included.

[0154] S501: obtaining and splitting an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; and the plurality of initial sub-network models contain the same input module;

[0155] S502: combining the plurality of initial sub-network models to obtain an initial multi-track model;

[0156] S503: training the initial multi-track model to obtain a target multi-track model corresponding to the target model.

[0157] In some embodiments, after obtaining the target multi-track model, when the method is implemented, the following contents can also be included: processing a test sample by using the target multi-track model to obtain a plurality of groups of adversarial test samples corresponding thereto; wherein one group of adversarial test samples corresponds to one sub-network model; and according to the plurality of groups of adversarial test samples, screening a model structure meeting the requirements from the target multi-track model as the target model structure; wherein the target model structure is a model structure used for adversarial attack test of the target model.

[0158] Through the above embodiments, a target model structure capable of generating an adversarial sample with good migration and strong adversarial for the target model can be obtained at a lower processing cost.

[0159] The embodiment of the present specification also provides a server, comprising a processor and a memory for storing processor-executable instructions, the processor, when implemented, can perform the following steps according to the instructions: receiving an adversarial sample generation request for a target model; constructing a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is also connected with a corresponding output module; determining a target model structure from the target multi-track model; wherein the target model structure is a model structure for adversarial attack test of the target model; generating an adversarial sample for the target model by using the target model structure.

[0160] In order to be able to more accurately complete the above instructions, refer to Figure 6 The embodiment of the present specification also provides another specific server, wherein the server comprises a network communication port 601, a processor 602 and a memory 603, and the above structures are connected by internal cables so that each structure can perform specific data interaction.

[0161] The network communication port 601 can be specifically used to receive an adversarial sample generation request for a target model.

[0162] The processor 602 can be specifically used to construct a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is also connected with a corresponding output module; determining a target model structure from the target multi-track model; wherein the target model structure is a model structure for adversarial attack test of the target model; generating an adversarial sample for the target model by using the target model structure.

[0163] The memory 603 can be specifically used to store corresponding instruction programs.

[0164] In the embodiment, the network communication port 601 can be a virtual port bound with different communication protocols, so as to send or receive different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for mail data communication. In addition, the network communication port can also be an entity communication interface or a communication chip. For example, it can be a wireless mobile network communication chip such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.

[0165] In the embodiment, the processor 602 can be implemented in any appropriate manner. For example, the processor can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code (e.g. software or firmware) executable by the (micro)processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. The present specification does not limit the processor.

[0166] In the embodiment, the memory 603 can include multiple levels, and in a digital system, as long as it can save binary data, it can be a memory; in an integrated circuit, a circuit without a physical form and with a storage function is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, a TF card, etc.

[0167] The embodiment of the present specification also provides another server, comprising a processor and a memory for storing processor executable instructions, and the processor, when implemented, can execute the following steps according to the instructions: obtaining and splitting an initial model to obtain a plurality of initial sub-network models; wherein the initial model is a model matched with a target model; the plurality of initial sub-network models contain the same input module; combining the plurality of initial sub-network models to obtain an initial multi-track model; training the initial multi-track model to obtain a target multi-track model corresponding to the target model.

[0168] The embodiment of the present specification also provides a computer storage medium based on the above-mentioned method for generating an adversarial sample, the computer storage medium storing computer program instructions, which, when executed, implement: receiving an adversarial sample generation request for a target model; constructing a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models of different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is also connected with a corresponding output module; determining a target model structure from the target multi-track model; wherein the target model structure is a model structure for adversarial attack testing of the target model; and generating an adversarial sample for the target model by using the target model structure.

[0169] The embodiment of the present specification also provides a computer storage medium based on the above-mentioned method for obtaining a target multi-track model corresponding to a target model, the computer storage medium storing computer program instructions, which, when executed, implement: obtaining and splitting an initial model to obtain a plurality of initial sub-network models of different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models comprise the same input module; combining the plurality of initial sub-network models to obtain an initial multi-track model; and training the initial multi-track model to obtain a target multi-track model corresponding to the target model.

[0170] In the embodiment, the storage medium includes but is not limited to a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk drive (HDD) or a memory card. The storage can be used to store computer program instructions. The network communication unit can be an interface set according to the standard of a communication protocol, used for network connection communication.

[0171] In the embodiment, the program instructions stored in the computer storage medium specifically implement functions and effects, which can be explained by comparing with other embodiments, and will not be described here.

[0172] Referring to Figure 7 As shown in the software layer, the embodiment of the present specification also provides a device for generating an adversarial sample, which specifically can include the following structure modules:

[0173] The receiving module 701 can be specifically used for receiving an adversarial sample generation request for a target model;

[0174] The construction module 702 can be specifically configured to construct a target multi-track model corresponding to the target model according to the request for generating the adversarial sample; the target multi-track model includes a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model includes a plurality of model blocks connected in series; the end of the sub-network model is further connected with a corresponding output module respectively;

[0175] The determination module 703 can be specifically configured to determine a target model structure from the target multi-track model; the target model structure is a model structure used for adversarial attack test of the target model;

[0176] The generation module 704 can be specifically configured to generate an adversarial sample for the target model by using the target model structure.

[0177] In some embodiments, when the construction module 702 is implemented, the target multi-track model can be constructed in the following manner: an initial model is obtained and split to obtain a plurality of initial sub-network models with different lengths; the initial model is a model matched with the target model; the plurality of initial sub-network models include the same input module; the plurality of initial sub-network models are combined to obtain an initial multi-track model; and the initial multi-track model is trained to obtain the target multi-track model corresponding to the target model.

[0178] In some embodiments, when the construction module 702 combines the plurality of initial sub-network models, the following procedure can be performed: the input module commonly included in the plurality of initial sub-network models is taken as a starting connection structure, and the plurality of initial sub-network models with different lengths are connected in parallel to obtain a network structure including a plurality of structure rows and a plurality of structure columns arranged by a plurality of model blocks, as the initial multi-track model; one structure column includes a plurality of model blocks, and one structure row includes a plurality of model blocks.

[0179] In some embodiments, in the initial multi-track model, the feature level of the feature processed by the model block included in the previous structure row is lower than the feature processed by the model block included in the next structure row.

[0180] In some embodiments, in the initial multi-track model, the model block is connected with the model block belonging to a different initial sub-network model to share the processed feature between the model blocks in different initial sub-network models.

[0181] In some embodiments, the construction module 702 can specifically train the initial multi-track model in the following manner: obtaining a training set of a target model; using the training set of the target model as sample data, training the initial multi-track model to obtain a target multi-track model.

[0182] In some embodiments, the construction module 702 can also specifically train the initial multi-track model in the following manner: obtaining sample data; testing the target model using the sample data to construct a similar training set; using the similar training set to train the initial multi-track model to obtain a target multi-track model.

[0183] In some embodiments, the determination module 703 can specifically implement the following procedures: processing test samples using the target multi-track model to obtain a plurality of groups of adversarial test samples; wherein each group of adversarial test samples corresponds to a sub-network model; and selecting a model structure meeting the requirements from the target multi-track model as the target model structure according to the plurality of groups of adversarial test samples.

[0184] In some embodiments, the target model structure includes a sub-network model or a combination of a plurality of sub-network models.

[0185] In some embodiments, the determination module 703 can specifically implement the following procedures to select a model structure meeting the requirements from the target multi-track model as the target model structure: constructing a combined adversarial test sample according to the plurality of groups of adversarial test samples; testing the target model using the plurality of groups of adversarial test samples and the combined adversarial test sample to collect the misidentification rates of the target model for each group of adversarial test samples and the combined adversarial test sample; and selecting a model structure corresponding to the adversarial test sample or the combined adversarial test sample with the highest misidentification rate from the target multi-track model as the target model structure.

[0186] In some embodiments, after the device invokes the target model structure matching the target model in the target multi-track model to generate an adversarial sample for the target model, the device can also be used to test the security of the target model using the adversarial sample.

[0187] In some embodiments, the target model can specifically include at least one of the following: a picture recognition model, a face detection model, a semantic analysis model, an automatic driving model, etc.

[0188] The embodiment of the present specification also provides a device for obtaining a target multi-track model corresponding to a target model, which can specifically include: a splitting module, which can be specifically used for obtaining and splitting an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models contain the same input module; a combination module, which can be specifically used for combining the plurality of initial sub-network models to obtain an initial multi-track model; and a training module, which can be specifically used for training the initial multi-track model to obtain the target multi-track model corresponding to the target model.

[0189] In some embodiments, the device further includes a determination module, which can be specifically used for processing test samples by using the target multi-track model to obtain a plurality of corresponding groups of adversarial test samples; wherein one group of adversarial test samples corresponds to one sub-network model; and a model structure meeting requirements is screened out from the target multi-track model according to the plurality of groups of adversarial test samples as the target model structure; wherein the target model structure is a model structure used for adversarial attack test of the target model.

[0190] It should be noted that the units, devices or modules and the like illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above device is described as various modules respectively described in terms of functions. Of course, in the implementation of the present specification, the functions of each module can be implemented in the same software and / or hardware, or the modules implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The above-described device embodiments are only illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0191] As can be seen from the above, the adversarial sample generation device provided based on the embodiment of the present specification can generate adversarial samples with better migration and stronger attack at a lower cost, so as to more effectively perform security test on the target model.

[0192] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0193] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0194] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0195] Those skilled in the art can clearly understand the present specification can be implemented by means of software and necessary general hardware platforms through the above description of the embodiments. Based on such understanding, the technical solutions of the present specification can essentially be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments of the present specification.

[0196] The various embodiments in the present specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. The present specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, etc.

[0197] Although the present specification is described through the embodiments, those skilled in the art know that the present specification has many modifications and changes without departing from the spirit of the present specification, and it is intended that the appended claims include these modifications and changes without departing from the spirit of the present specification.

Claims

1. A method for generating an adversarial sample, comprising: receiving an adversarial sample generation request for a target model; wherein the target model is an image recognition model; constructing a target multi-track model corresponding to the target model according to the adversarial sample generation request; wherein the target multi-track model comprises a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model comprises a plurality of model blocks connected in series; the end of the sub-network model is further connected with a corresponding output module; wherein the target multi-track model is obtained by training an initial multi-track model; the model blocks in the upper structure row of the initial multi-track model process lower-level image features, and the model blocks in the lower structure row process higher-level image features; determining a target model structure from the target multi-track model; wherein the target model structure is a model structure used for adversarial attack test of the target model; generating an adversarial sample for the target model by using the target model structure and an original sample to test the security of the target model and improve the robustness of the target model; wherein the original sample is image data; generating the adversarial sample of the target model by using the target model structure and the original sample comprises: performing multiple iteration processes according to the following manner until a predetermined stop condition is met to obtain the adversarial sample for the target model: processing the original sample by using the target model structure to obtain a sub-processing result; adjusting the original sample according to the sub-processing result to obtain an adjusted original sample; processing the adjusted original sample by using the target model structure to obtain an updated sub-processing result; and detecting whether the predetermined stop condition is met according to the updated sub-processing result.

2. The method of claim 1, wherein the constructing a target multi-track model corresponding to the target model comprises: obtaining and splitting an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models comprise the same input module; combining the plurality of initial sub-network models to obtain an initial multi-track model; training the initial multi-track model to obtain the target multi-track model corresponding to the target model.

3. The method of claim 2, wherein the combining the plurality of initial sub-network models to obtain an initial multi-track model comprises: parallel connecting the plurality of initial sub-network models with different lengths with the input module commonly contained in the plurality of initial sub-network models as a starting connection structure to obtain a network structure comprising a plurality of structure rows and a plurality of structure columns arranged by a plurality of model blocks, as the initial multi-track model; wherein one structure column comprises a plurality of model blocks, and one structure row comprises a plurality of model blocks.

4. The method of claim 3, wherein in the initial multi-track model, the feature level of the features processed by the model blocks contained in the upper structure row is lower than the features processed by the model blocks contained in the lower structure row.

5. The method of claim 3, in the initial multi-track model, a model block is connected to a model block belonging to a different initial sub-network model to share processed features between model blocks in different initial sub-network models.

6. The method of claim 2, training the initial multi-track model to obtain a target multi-track model corresponding to the target model, comprising: obtaining a training set of the target model; training the initial multi-track model using the training set of the target model to obtain the target multi-track model.

7. The method of claim 2, training the initial multi-track model to obtain a target multi-track model corresponding to the target model, further comprising: obtaining sample data; constructing a similar training set by testing the target model using the sample data; training the initial multi-track model using the similar training set to obtain the target multi-track model.

8. The method of claim 1, determining the target model structure from a target multi-track model corresponding to the target model, comprising: processing test samples using the target multi-track model to obtain a plurality of groups of adversarial test samples; wherein a group of adversarial test samples corresponds to a sub-network model; screening a required model structure from the target multi-track model according to the plurality of groups of adversarial test samples as the target model structure.

9. The method of claim 8, wherein the target model structure comprises a sub-network model or a combination of a plurality of sub-network models.

10. The method of claim 8, screening a required model structure from the target multi-track model according to the plurality of groups of adversarial test samples as the target model structure, comprising: constructing a combined adversarial test sample according to the plurality of groups of adversarial test samples; testing the target model using the plurality of groups of adversarial test samples and the combined adversarial test sample respectively to obtain a misrecognition rate of the target model for each group of adversarial test samples and the combined adversarial test sample; screening a model structure corresponding to an adversarial test sample or the combined adversarial test sample with the highest misrecognition rate from the target multi-track model as the target model structure.

11. A method for obtaining a target multi-track model corresponding to a target model, comprising: obtaining and splitting an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models contain the same input module; and the target model is an image recognition model; combining the plurality of initial sub-network models to obtain an initial multi-track model; wherein a model block in an upper structure row in the initial multi-track model processes lower-level image features, and a model block in a lower structure row processes higher-level image features. training the initial multi-track model to obtain a target multi-track model corresponding to the target model; the target multi-track model is used to generate an adversarial sample for the target model based on an original sample, to test the security of the target model and improve the robustness of the target model; wherein the original sample is image data; wherein the target multi-track model is iteratively processed based on the original sample in the following manner until a predetermined stopping condition is met to obtain the adversarial sample for the target model: processing the original sample using the target model structure to obtain a sub-processing result; adjusting the original sample based on the sub-processing result to obtain an adjusted original sample; processing the adjusted original sample using the target model structure to obtain an updated sub-processing result; and detecting whether the predetermined stopping condition is met based on the updated sub-processing result.

12. The method of claim 11, after obtaining the target multi-track model corresponding to the target model, the method further comprises: processing the test sample using the target multi-track model to obtain a corresponding plurality of sets of adversarial test samples; wherein each set of adversarial test samples corresponds to a sub-network model; screening a required model structure from the target multi-track model based on the plurality of sets of adversarial test samples, as the target model structure; wherein the target model structure is a model structure used for adversarial attack testing of the target model.

13. An adversarial sample generation device, comprising: a receiving module configured to receive an adversarial sample generation request for a target model; wherein the target model is an image recognition model; a construction module configured to construct a target multi-track model corresponding to the target model based on the adversarial sample generation request; wherein the target multi-track model includes a plurality of sub-network models with different lengths; the plurality of sub-network models are connected with a common input module; the sub-network model includes a plurality of model blocks connected in series; the end of the sub-network model is further connected with a corresponding output module; wherein the target multi-track model is obtained by training an initial multi-track model; the model blocks in the upper structure row of the initial multi-track model process lower level image features, and the model blocks in the lower structure row process higher level image features; a determination module configured to determine a target model structure from the target multi-track model; wherein the target model structure is a model structure used for adversarial attack testing of the target model. The generating module is configured to generate an adversarial sample for the target model by using the target model structure and the original sample, so as to test the security of the target model and improve the robustness of the target model; the original sample is image data; the generating of the adversarial sample for the target model by using the target model structure and the original sample comprises: performing multiple iteration processes in the following manner until a predetermined stop condition is met, so as to obtain the adversarial sample for the target model: processing the original sample by using the target model structure to obtain a sub-processing result; adjusting the original sample according to the sub-processing result to obtain an adjusted original sample; processing the adjusted original sample by using the target model structure to obtain an updated sub-processing result; and detecting whether the predetermined stop condition is met according to the updated sub-processing result.

14. An apparatus for obtaining a target multi-track model corresponding to a target model, comprising: a splitting module configured to obtain and split an initial model to obtain a plurality of initial sub-network models with different lengths; wherein the initial model is a model matched with the target model; the plurality of initial sub-network models contain the same input module; and the target model is an image recognition model; a combining module configured to combine the plurality of initial sub-network models to obtain an initial multi-track model; wherein the model blocks in the upper structure row in the initial multi-track model process lower-level image features, and the model blocks in the lower structure row process higher-level image features; a training module configured to train the initial multi-track model to obtain a target multi-track model corresponding to the target model; the target multi-track model is configured to generate an adversarial sample for the target model based on an original sample, so as to test the security of the target model and improve the robustness of the target model; wherein the original sample is image data; and the target multi-track model performs multiple iteration processes based on the original sample in the following manner until a predetermined stop condition is met, so as to obtain the adversarial sample for the target model: processing the original sample by using the target model structure to obtain a sub-processing result; adjusting the original sample according to the sub-processing result to obtain an adjusted original sample; processing the adjusted original sample by using the target model structure to obtain an updated sub-processing result; and detecting whether the predetermined stop condition is met according to the updated sub-processing result.

15. A server comprising a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method of any one of claims 1-10 or 11-12 when executing the instructions.

16. A computer-readable storage medium having computer instructions stored thereon, wherein the instructions implement the steps of the method of any one of claims 1-10 or 11-12 when executed.

Images