Model testing method, device, electronic device and storage medium

Abstract

The present application relates to the field of artificial intelligence technology, and in particular to a model testing method, device, electronic device and storage medium, which are used to improve the accuracy of security testing of a model to be tested. The method includes: obtaining each original image, and the corresponding recognition result label; each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with each original image are different; based on each shape parameter set, the mask result corresponding to the corresponding original image is determined respectively, and based on each mask result, the perturbation parameter is set for each pixel point in the corresponding original image respectively; based on each recognition result label, the perturbation parameter corresponding to the corresponding original image and the associated shape parameter set are updated respectively to obtain an attack image containing an attack area; each attack image is used to perform a security test on the model to be tested. In this way, the accuracy of the test of the model to be tested can be improved.

Description

Background Art

[0002] With the development of deep learning technology, artificial intelligence (AI) models have been widely used in various fields, such as smart payment.

[0003] However, with the rise of attack technology, when the security of AI models is low, if the original image input to the AI ​​model is disturbed, it will affect the output of the AI ​​model. Therefore, it is necessary to conduct security testing on the AI ​​model before it is put into use.

[0004] In the related art, when performing security testing on an AI model, each attack image with added disturbance is input into the AI ​​model respectively, and multiple rounds of testing are performed to obtain the security test results of the AI ​​model.

[0005] However, since the AI ​​model can learn the shape features of the attack area in the attack image during multiple rounds of testing, the subsequent recognition process, based on the learned shape features, will still affect the output results of the AI ​​model.

[0006] Therefore, the accuracy of the model security test in the related art is not high. Summary of the invention

[0007] Embodiments of the present application provide a model testing method, device, electronic device, and storage medium to improve the accuracy of model security testing.

[0008] A model testing method provided in an embodiment of the present application includes:

[0009] Obtaining each original image and its corresponding recognition result label; each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with each original image are different;

[0010] Based on each shape parameter set, a mask result corresponding to the corresponding original image is determined respectively, and based on each mask result, a perturbation parameter is set for each pixel point in the corresponding original image respectively; wherein each mask result represents a setting method of a perturbation parameter for each pixel point in the corresponding original image;

[0011] Based on each recognition result label, the disturbance parameters corresponding to the corresponding original image and the associated shape parameter set are updated respectively to obtain an attack image containing the attack area;

[0012] Use various attack images to perform security tests on the model to be tested.

[0013] A model testing device provided in an embodiment of the present application includes:

[0014] An obtaining unit, used for obtaining each original image and the corresponding recognition result label; each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with each original image are different;

[0015] A first processing unit is used to determine the mask results corresponding to the corresponding original images based on the shape parameter sets, and to set the perturbation parameters for each pixel in the corresponding original images based on the mask results; wherein each mask result represents the setting mode of the perturbation parameters for each pixel in the corresponding original image;

[0016] A second processing unit is used to update the disturbance parameters corresponding to the corresponding original images and the associated shape parameter set based on each recognition result label, so as to obtain an attack image including the attack area;

[0017] The testing unit is used to use various attack images to perform security tests on the model to be tested.

[0018] Optionally, when determining the mask results corresponding to the corresponding original images based on the shape parameter sets, the first processing unit is further configured to:

[0019] For each of the original images, the following operations are performed respectively:

[0020] Based on a set of shape parameters associated with an original image, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained; wherein the pixel point connection length represents the length of the pixel point connection between the center point and the corresponding pixel point, and the intersection point connection length represents the length between the intersection of the vertex connection line and the pixel point connection line of two vertices obtained based on two adjacent shape parameters and the center point;

[0021] Based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the sub-mask results corresponding to the corresponding pixel point are determined respectively; each sub-mask result represents the setting method of the disturbance parameter of the corresponding pixel point;

[0022] The mask result including each sub-mask result is generated.

[0023] Optionally, when obtaining the pixel point connection length and the intersection point connection length corresponding to each pixel point based on the shape parameter set associated with an original image, the first processing unit is further used to:

[0024] For each pixel in an original image, perform the following operations:

[0025] Based on a first shape parameter and a second shape parameter associated with a pixel point, and a preset angle, determining a first vertex coordinate of a first vertex corresponding to the first shape parameter, and a second vertex coordinate of a second vertex corresponding to the first shape parameter;

[0026] Determine, based on the first vertex coordinates, the second vertex coordinates, the pixel point coordinates corresponding to the one pixel point, and the center point coordinates of the one original image, the intersection coordinates corresponding to the intersection of the line between the first vertex and the second vertex and the line between the center point and the pixel point;

[0027] Determine the length of the line connecting the intersection points based on the intersection point coordinates and the center point coordinates;

[0028] And, based on the pixel point coordinates and the center point coordinates, the length of the pixel point connection line is determined.

[0029] Optionally, when determining the sub-mask results corresponding to the corresponding pixel points based on the length of the line connecting the pixel points and the length of the line connecting the intersection points, the first processing unit is further used to:

[0030] For each pixel point, the following operations are performed respectively:

[0031] If it is determined that the length of the pixel point connection line corresponding to a pixel point is not greater than the length of the intersection point connection line, then determining the sub-mask result corresponding to the pixel point is to set the disturbance parameter;

[0032] If it is determined that the length of the line connecting the pixel points is greater than the length of the line connecting the intersection points, then the sub-mask result is determined to be that no disturbance parameter is set.

[0033] Optionally, the second processing unit is further used for:

[0034] For each original image with perturbation parameters set, the following operations are performed:

[0035] Input an original image with perturbation parameters set into a trained parameter update model to obtain a corresponding model recognition result, update the corresponding perturbation parameters and the associated shape parameter set based on the model recognition result and the corresponding recognition result label, and re-input the updated original image with perturbation parameters set into the parameter update model until the cross entropy between the model recognition result and the recognition result label meets the end-of-update condition, thereby obtaining the updated perturbation parameters and the updated shape parameter set;

[0036] Based on the updated disturbance parameters and the updated shape parameter set, an attack image including the attack area is obtained.

[0037] Optionally, when updating the corresponding disturbance parameter and the associated shape parameter set, the second processing unit is further configured to:

[0038] Determining an updated disturbance parameter based on a current disturbance parameter, a first learning rate, and a preset penalty coefficient, wherein the penalty coefficient is used to control the size of the attack area; and

[0039] An updated shape parameter set is determined based on the current shape parameter set, the second learning rate, and the penalty coefficient.

[0040] Optionally, the testing unit is further used for:

[0041] Input each attack image into the model to be tested and obtain the corresponding output result;

[0042] Based on each output result and the corresponding identification result label, the security test result of the model to be tested is determined.

[0043] Optionally, when determining the security test result of the model to be tested based on each output result and the corresponding identification result label, the testing unit is further used to:

[0044] Determine the image test result corresponding to the corresponding attack image based on the output result corresponding to each attack image and the corresponding recognition result label;

[0045] Determine the attack success rate of the model to be tested based on the number of attack images corresponding to unsuccessful attacks as a result of the image test and the total number of the attack images;

[0046] If it is determined that the attack success rate is less than the success rate threshold, then the security test result of the model to be tested is determined to be safe;

[0047] If it is determined that the attack success rate is not less than the success rate threshold, then the security test result of the model to be tested is determined to be unsafe.

[0048] On the one hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned model testing methods.

[0049] On the one hand, an embodiment of the present application provides a computer storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the steps of any one of the above-mentioned model testing methods.

[0050] On the one hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; when a processor of an electronic device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the electronic device performs the steps of any one of the above-mentioned model testing methods.

[0051] The beneficial effects of this application are as follows:

[0052] In the scheme of the embodiment of the present application, since the shape parameter sets associated with each original image are different, the shape of the attack area contained in each generated attack image is also different. When the security test is performed on the model to be tested, the attack intensity and attack accuracy can be improved, thereby improving the accuracy of the model test; and, in the embodiment of the present application, the mask result of the corresponding original image is determined based on the shape parameter set, and the original image is converted into an attack image based on the determined mask result, so that the shape parameters of the attack area can be converted into an optimizable parameter form, so that the disturbance parameters of the original image and the associated shape parameter set can be updated at the same time, so as to achieve a more accurate and effective attack, thereby further improving the accuracy of the model test.

[0053] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 This is a schematic diagram of an application scenario in an embodiment of the present application;

[0056] Figure 2 This is a flow chart of an implementation of a model testing method in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of a process for obtaining a mask result in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a process for obtaining the length of a line connecting pixel points and the length of a line connecting intersection points in an embodiment of the present application;

[0059] Figure 5 This is a schematic diagram of shape parameter modeling in an embodiment of the present application;

[0060] Figure 6 This is a schematic diagram of the deformation of the attack area in the embodiment of the present application.

[0061] Figure 7 This is a first schematic diagram of a triangular area in an embodiment of the present application;

[0062] Figure 8 This is a second schematic diagram of a triangular area in an embodiment of the present application;

[0063] Fig. 9 This is a third schematic diagram of a triangular area in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of a process for determining a sub-mask result corresponding to a pixel point in an embodiment of the present application;

[0065] Fig.11 A flowchart of a method for updating a disturbance parameter and a shape parameter set in an embodiment of the present application;

[0066] Fig.12 Schematic diagram of the parameter updating method in the embodiment of the present application;

[0067] Fig.13 A flowchart of a method for performing security testing on a model to be tested in an embodiment of the present application;

[0068] Fig.14 A schematic diagram of a process for determining a safety test result in an embodiment of the present application;

[0069] Fig.15 This is a schematic diagram of attack effect comparison in the embodiments of this application;

[0070] Fig.16 This is an experimental comparison effect diagram in the embodiments of this application;

[0071] Fig.17 This is an example diagram of the model testing method in the embodiment of the present application;

[0072] Fig.18 A schematic diagram of the structure of a model testing device in an embodiment of the present application;

[0073] Fig.19 A schematic diagram of the hardware structure of an electronic device to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0075] In order to help those skilled in the art better understand the technical solution of the present application, some concepts involved in the present application are introduced below.

[0076] Recognition result label: a label that is pre-configured for the target object contained in the original image and represents the recognition result, and is used to indicate the target object contained in the original image.

[0077] For example, assuming that the target object contained in the original image is a ruler, the recognition result label pre-marked for the original image is a ruler, which is not limited in the embodiments of the present application.

[0078] Shape parameter set: represents a shape parameter set used to generate an attack area. The shape parameter set includes multiple shape parameters, and each shape parameter represents the length of a line between the center point of the attack area and a vertex of the area.

[0079] Attack area: The attack area is generated by adding perturbation parameters to the pixels in the corresponding area of ​​the original image.

[0080] Mask result: It represents the setting method of perturbation parameters for each pixel in the original image. The mask result is generated by multiple sub-mask results.

[0081] Sub-mask result: represents the perturbation parameter setting method for the pixel point, that is, whether the perturbation parameter is set for the pixel point.

[0082] For example, if the sub-mask result of the pixel is determined to be 1, it is determined to set the disturbance parameter for the pixel; for another example, if the sub-mask result of the pixel is determined to be 0, it is determined not to set the disturbance parameter for the pixel.

[0083] Perturbation parameter: The perturbation parameter represents the noise set on the pixel point, which is used to cover the pixel point.

[0084] Pixel line length: represents the length of the pixel line between a pixel of the original image and the center point of the attack area. The pixel line length is determined based on the pixel coordinates corresponding to the pixel and the center coordinates corresponding to the center point.

[0085] Intersection line length: represents the length of the intersection of the vertex line between the first vertex and the second vertex and the pixel line corresponding to two shape parameters associated with a pixel point of the original image, and the intersection line between the center point.

[0086] The following is a brief introduction to the design concept of the embodiment of the present application:

[0087] With the development of deep learning technology, artificial intelligence (AI) products have gradually been put into practical use. Therefore, AI models have also been widely used in various fields, such as smart payment.

[0088] However, with the development of attack technology, when the security of AI models is low, if criminals add imperceptible perturbation parameters to the original images input into the AI ​​models, the output results of the AI ​​models will be affected, which will bring huge security threats to smart driving, smart medical care, smart retail and other fields. Therefore, how to effectively evaluate the security of AI models before they are put into use has become an urgent problem to be solved.

[0089] In related technologies, when evaluating AI models, it can mainly be achieved by adversarial attacks, which can be divided into the following two types:

[0090] The first method: global perturbation attack.

[0091] Global perturbation attack refers to generating an attack image by adding a perturbation that is difficult to be detected by the naked eye to the entire original image. The perturbation can be added to any pixel of the original image. In order to ensure that the perturbation is difficult to be detected by the naked eye, when setting the perturbation parameters, the perturbation parameters are usually limited to L. p The norm is lower than the perturbation parameter threshold. Therefore, the global perturbation attack method is easily affected by the environment during the migration to the physical domain. The slight change in pixel value caused by ambient light will cause the attack to fail. Therefore, the security threat brought by the above method is very limited, which will reduce the accuracy of model testing.

[0092] The second method: local area attack.

[0093] A local area attack means that the attacker only adds perturbation parameters to the pixels in the local area, but does not limit the amplitude of the perturbation parameters. The aforementioned local area attack method can be easily and efficiently migrated to the physical domain. However, in this method in the related technology, the generated attack area is often fixed as a rectangle or a circle. Through the local area attack method, when performing security testing on the AI ​​model, the shape is used as an important discriminant feature, so that the AI ​​model can learn the shape features of the attack area in the attack image. In the subsequent recognition process, recognition based on the learned shape features will limit the upper limit of the attack intensity, which will still affect the output results of the AI ​​model.

[0094] Therefore, the testing accuracy of these two model security testing methods in the related art is not high.

[0095] In view of this, the embodiment of the present application proposes a model testing method, device, electronic device and storage medium, which obtains each original image and the corresponding recognition result label, determines the mask result corresponding to the corresponding original image based on each shape parameter set, and sets the perturbation parameter for each pixel in the corresponding original image based on each mask result, updates the perturbation parameter corresponding to the corresponding original image and the associated shape parameter set based on each recognition result label, obtains the attack image containing the attack area, and uses each attack image to perform security test on the model to be tested. In this way, by modeling each shape parameter in the shape parameter set associated with the original image, each shape parameter is modeled as an optimizable indicator, and the perturbation parameter of the original image and the associated shape parameter set are updated through joint optimization, so as to achieve more accurate and effective attack, improve the attack success rate, and further improve the accuracy of model testing.

[0096] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0097] like Figure 1 As shown, it is a schematic diagram of an application scenario in an embodiment of the present application. The application scenario schematic diagram includes a terminal device 110 and a server 120, and the terminal device 110 and the server 120 can communicate with each other through a communication network.

[0098] In the embodiment of the present application, the terminal device 110 is an electronic device used by the user, and the electronic device may be a personal computer, a mobile phone, a tablet computer, a notebook, an e-book reader, a smart home, a shopping mall gate, etc. Each terminal device 110 may communicate with the server 120 via a communication network. In an optional implementation, the communication network may be a wired network or a wireless network. Therefore, the terminal device 110 and the server 120 may be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application does not impose any specific restrictions here.

[0099] The server 120 can be an independent physical server 120, or an edge device 120 in the field of cloud computing, or a cloud server 120 that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, cloud functions, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0100] It should be noted that the model testing method in the embodiment of the present application can be executed by the server 120 or the terminal device 110 alone, or can be executed by the server 120 and the terminal device 110 together. When the server 110 and the terminal device 120 execute together, for example, the terminal device 110 responds to the test instruction for the model to be tested triggered by the target object, and sends the test instruction to the server 120. The server 120 responds to the received test instruction, determines the mask results corresponding to each original image, and sets the perturbation parameters for each pixel in the corresponding original image based on each mask result, and updates the perturbation parameters and the associated shape parameter set corresponding to the corresponding original image based on each recognition result label, obtains each attack image containing the attack area, and performs model testing based on each attack image to obtain the model test result, and then sends the obtained model test result to the terminal device 110 for display. In the following, the server alone is mainly used as an example for illustration, and no specific limitation is made here.

[0101] The model testing method provided by the exemplary embodiment of the present application is described below in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0102] See also Figure 2 As shown, it is a flowchart of an implementation of a model testing method provided in an embodiment of the present application. Here, the server is used as an example for introduction. The specific implementation process of the method is as follows:

[0103] S20: Obtain each original image and its corresponding recognition result label.

[0104] Each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with each original image are different.

[0105] In an embodiment of the present application, in response to a model test instruction for the model to be tested, each original image associated with the model to be tested is obtained from an image database, and since each original image has a corresponding recognition result label, the recognition result label corresponding to each original image is obtained separately while obtaining each original image.

[0106] In addition, each shape parameter set is obtained. The above shape parameter sets are shape parameter sets associated with the corresponding original image. Each shape parameter set includes multiple shape parameters. The shape parameter sets are used to generate the attack area in the original image.

[0107] For example, assuming that the original image contains a ruler placed on a desk, after the original image is recognized, the output standard recognition result should be a ruler. Therefore, a recognition result label "ruler" is set for the original image.

[0108] S21: Based on each shape parameter set, respectively determine the mask results corresponding to the corresponding original images, and based on each mask result, respectively set a disturbance parameter for each pixel point in the corresponding original images.

[0109] Among them, each mask result represents the setting method of the perturbation parameters of each pixel point in the corresponding original image.

[0110] In the embodiment of the present application, since each shape parameter in the current shape parameter set is a fixed value and each shape parameter cannot be optimized, it is necessary to first model each shape parameter so that each shape parameter is converted into an optimizable form, thereby achieving the optimization of the shape parameter. In the embodiment of the present application, the parameter form of the shape parameter is converted by determining the mask result. Specifically, based on each shape parameter set, the mask result corresponding to the corresponding original image is determined. The determined mask result is used to determine the setting method of the perturbation parameter for each pixel point. Then, based on each determined mask result, in the corresponding original image, the perturbation parameter is set for each pixel point contained in the original image.

[0111] Therefore, in the embodiment of the present application, each pixel point in the original image is represented by a mask parameter, so that attack areas of different shapes can be formed.

[0112] Optionally, in the embodiment of the present application, when executing S21, it is necessary to obtain the mask results corresponding to each original image respectively. Specifically, taking any original image (hereinafter referred to as original image a) as an example, the process of obtaining the mask result of the original image is introduced as follows, see Figure 3 As shown in the figure, it is a schematic diagram of the process of obtaining the mask result in the embodiment of the present application. Figure 3 The process of determining the mask result corresponding to the original image in the embodiment of the present application is described in detail:

[0113] S211: Based on the shape parameter set associated with the original image a, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained.

[0114] The pixel point line length represents the length of the pixel point line between the center point and the corresponding pixel point, and the intersection point line length represents the length between the intersection of the vertex line of two vertices obtained based on two adjacent shape parameters and the pixel point line and the center point.

[0115] In the embodiment of the present application, since each original image is associated with a shape parameter set, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained based on each shape parameter in the shape parameter set associated with the original image a.

[0116] It should be noted that, in the embodiment of the present application, the length of the pixel line corresponding to the pixel point represents the length corresponding to the pixel line between the center point of the attack area and the pixel point, and the length of the intersection line represents the length corresponding to the line between the intersection of the vertex line and the pixel line between the two vertices corresponding to the two shape parameters associated with the pixel point and the center point.

[0117] Optionally, in the embodiment of the present application, when executing S211, it is necessary to obtain the pixel point connection length and the intersection point connection length corresponding to each pixel point respectively. Specifically, taking any pixel point (hereinafter referred to as pixel point b) in an original image as an example, the process of obtaining the pixel point connection length and the intersection point connection length is introduced as follows, see Figure 4 As shown in the figure, it is a schematic diagram of the process of obtaining the length of the line connecting the pixel points and the length of the line connecting the intersection points in the embodiment of the present application. Figure 4 The process of determining the length of the pixel point connection line and the length of the intersection point connection line corresponding to the pixel point in the embodiment of the present application is described in detail:

[0118] S2121: Based on the first shape parameter and the second shape parameter associated with the pixel point b, and a preset angle, determine the first vertex coordinates of the first vertex corresponding to the first shape parameter, and the second vertex coordinates of the second vertex corresponding to the first shape parameter.

[0119] In the embodiment of the present application, first, based on the number of shape parameters included in the shape parameter set, the number of triangles included in the attack area is determined, and based on the determined number of triangles, the angle corresponding to the central angle of the attack area is determined.

[0120] For example, see Figure 5 As shown, it is a schematic diagram of shape parameter modeling in an embodiment of the present application. Since the shape parameter set includes 8 shape parameters, namely r1, r2, r3, r4, r5, r6, r7, and r8, shape parameters r1 and r2 constitute a triangular area △1, shape parameters r2 and r3 constitute a triangular area △2, shape parameters r3 and r4 constitute a triangular area △3, shape parameters r4 and r5 constitute a triangular area △4, shape parameters r5 and r6 constitute a triangular area △5, shape parameters r6 and r7 constitute a triangular area △6, shape parameters r7 and r8 constitute a triangular area △7, shape parameters r8 and r1 constitute a triangular area △8, and there are 8 triangular areas in the attack area. Therefore, the angle corresponding to each central angle of the attack area can be expressed as:

[0121]

[0122] Wherein, Δθ is the angle of the central angle of the circle, and R is the number of triangles, that is, the number of shape parameters included in the shape parameter set.

[0123] Therefore, in the Cartesian coordinate system, using the center point O and a set of rays r i (i=1,2,…,R) represents a polygonal attack area, so that the shape parameter r i By adjusting the length, you can achieve different attack area shapes, see Figure 6 , which is a schematic diagram of the deformation of the attack area in an embodiment of the present application, Figure 6 In the example, the shape parameters associated with the attack area are different.

[0124] Then, since each pixel point is associated with two shape parameters, a first shape parameter associated with pixel point b and a second shape parameter associated with pixel point b are determined from the shape parameter set.

[0125] For example, see Figure 7 As shown in the figure, it is a first schematic diagram of a triangular area in an embodiment of the present application. The center point O and two adjacent rays can form a triangular area △AOB. The triangular area △AOB includes multiple pixel points. For any pixel point b in the area surrounded by two adjacent rays, the pixel point b is Figure 7Represented as C, the mask parameter M(C) can be used to indicate whether the pixel point b is within the area enclosed by △AOB, where the first shape parameter associated with the pixel point b is r1, and the second shape parameter associated with the pixel point b is r2.

[0126] Finally, the first vertex coordinates corresponding to the first vertex are determined based on the determined angle and the first shape parameter associated with the pixel point b, and the second vertex coordinates of the second vertex are determined based on the determined angle and the second shape parameter associated with the pixel point b.

[0127] For example, see Figure 8 As shown in FIG. 1 , it is a second schematic diagram of a triangular region in an embodiment of the present application. Assume that the number of shape parameters included in the shape parameter set is 8, and the first shape parameter r1 is The second shape parameter r2 is The attack area is composed of 8 triangular areas, and the central angle Δθ corresponding to each triangular area is Since the first shape parameter r1 is the length of the line between the first vertex A and the center point O of the attack area, the horizontal coordinate x of the first vertex A is A A'A=OA 2 -OA' 2 , the ordinate y of the first vertex A A OA'=OA 2 -A'A 2 Since the central angle Δθ is 45°, A'A is the same as OA', and the horizontal coordinate x of the first vertex A is A is 1, the ordinate y of the first vertex A A =1, thus determining that the first vertex coordinate of the first vertex A in the Cartesian coordinate system is (-1, -1). Since the second shape parameter r2 is the length of the line between the second vertex B and the center point O of the attack area, the horizontal coordinate x of the second vertex B is B B'B=OB 2 -OB' 2 , the ordinate y of the second vertex B B OB'=OB 2 -B'B 2 Since the central angle Δθ is 45°, B'B is the same as OB', and the horizontal coordinate x of the second vertex A is B is 1, the ordinate y of the second vertex B A is 1, thereby determining that the second vertex coordinates of the second vertex B in the Cartesian coordinate system are (1, -1).

[0128] S2122: Based on the first vertex coordinates, the second vertex coordinates, the pixel point coordinates corresponding to the pixel point b, and the center point coordinates of the original image a, determine the intersection coordinates corresponding to the intersection of the line between the first vertex and the second vertex and the line between the center point and the pixel point.

[0129] In an embodiment of the present application, first, based on the first vertex coordinates corresponding to the first vertex and the second vertex coordinates corresponding to the second vertex, the vertex line between the first vertex and the second vertex, and the corresponding vertex line length are obtained, and then, based on the pixel point coordinates corresponding to pixel point b and the center point coordinates corresponding to the center point of the original image a, the pixel point line between the center point and pixel point b, and the pixel point line length corresponding to the pixel point line are determined, and finally, based on the vertex line length and the pixel point line length, the intersection coordinates corresponding to the intersection between the vertex line and the pixel point line can be determined.

[0130] For example, see Fig. 9 As shown, it is a third schematic diagram of a triangular area in an embodiment of the present application, the first vertex corresponding to the first shape parameter of the attack area is point A, the second vertex corresponding to the second shape parameter of the attack area is point B, the center point of the attack area is point O, the pixel point b is point C, then the vertex line between the first vertex and the second vertex is AB, and the vertex line length corresponding to the vertex line AB is expressed as:

[0131] A1x+B1y+C1=0

[0132] The pixel line between pixel point C and center point O is OC, and the pixel line length corresponding to pixel line OC is expressed as:

[0133] A0x+B0y+C0=0

[0134] Therefore, based on the first vertex coordinates of the first vertex A, the second vertex coordinates of the second vertex B, the center point coordinates of the center point O and the pixel point coordinates of the pixel point C, as well as the equation corresponding to the length of the vertex line and the equation corresponding to the length of the pixel point line, the intersection coordinates corresponding to the intersection point D between the vertex line and the pixel point line can be determined.

[0135] S2123: Determine the length of the line connecting the intersection points based on the intersection point coordinates and the center point coordinates.

[0136] In the embodiment of the present application, after the intersection coordinates are determined, the intersection line length corresponding to the intersection line between the center point and the intersection is determined based on the intersection coordinates and the center point coordinates of the attack area.

[0137] For example, Fig. 9 As shown, the intersection point is point D, and the center point is point O, then the length of the intersection line corresponding to the intersection line between the center point and the intersection point is determined to be DO.

[0138] S2124: Determine the length of the line connecting the pixel points based on the pixel point coordinates and the center point coordinates.

[0139] In an embodiment of the present application, after obtaining the pixel point coordinates of pixel point b in the Cartesian coordinate system, the pixel point line length corresponding to the pixel point line between the center point and the pixel point is determined based on the pixel point coordinates and the center point coordinates of the attack area.

[0140] For example, assuming that the pixel point coordinates corresponding to pixel point b are (-2, -3), and the center point coordinates corresponding to the center point are (0, 0), then the pixel point line between the center point and the pixel point is determined to be OC, and the pixel point line length corresponding to the pixel point line OC is determined to be

[0141] It should be noted that in the embodiment of the present application, there is no restriction on the execution order between S1121-S1123 and S1124. S1121-S1123 can be executed first, and then S1124, that is, the length of the intersection line is determined first, and then the length of the pixel line is determined; S1124 can also be executed first, and then S1121-S1123, that is, the length of the pixel line is determined first, and then the length of the intersection line is determined; of course, S1121-S1123 and S2234 can also be executed at the same time, that is, the length of the intersection line and the length of the pixel line are determined at the same time.

[0142] Through the above method, the pixel point connection length and the intersection point connection length can be determined based on two adjacent shape parameters, so that the shape parameters can be converted into a differentiable form, the shape parameters can be optimized, and the accuracy of the model test can be further improved.

[0143] S212: Based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, respectively determine the sub-mask results corresponding to the corresponding pixel points.

[0144] Among them, each sub-mask result represents the setting method of the perturbation parameters of the corresponding pixel point.

[0145] In the embodiment of the present application, based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the sub-mask results corresponding to the corresponding pixel points are determined respectively.

[0146] Optionally, in the embodiment of the present application, the sub-mask result corresponding to the corresponding pixel point can be obtained by the ratio between the length of the line connecting each pixel point and the length of the line connecting the corresponding intersection points.

[0147] Optionally, in the embodiment of the present application, when executing S212, it is necessary to determine the sub-mask result corresponding to each pixel point. Specifically, taking the above-mentioned pixel point b as an example, the process of determining the sub-mask result corresponding to the pixel point is described as follows. Fig.10 As shown in FIG. 1 , a schematic diagram of a process for determining a sub-mask result corresponding to a pixel point in an embodiment of the present application is shown in FIG. Fig.10 The process of determining the sub-mask result corresponding to the pixel point in the embodiment of the present application is described in detail:

[0148] S2121: If it is determined that the length of the line connecting the pixel points corresponding to the pixel point b is not greater than the length of the line connecting the intersection points, then the sub-mask result corresponding to the pixel point b is determined to be the set disturbance parameter.

[0149] In the embodiment of the present application, in order to ensure the differentiability of the judgment, it can be implemented based on the activation function, where the activation function can be expressed as:

[0150]

[0151] Among them, x is a variable and λ is a hyperparameter for adjusting the ratio.

[0152] When the variable is When , the sub-mask result can be expressed as:

[0153]

[0154] Among them, CO represents the length of the line connecting the pixels, and DO represents the length of the line connecting the intersections.

[0155] Therefore, it is determined whether the length of the pixel line corresponding to the pixel point b is greater than the length of the intersection line. If it is determined that the length of the pixel line corresponding to the pixel point b is not greater than the length of the intersection line, that is, the ratio of the pixel line length to the intersection line length is not greater than 1, then the sub-mask result corresponding to the pixel point b is determined to be the set disturbance parameter.

[0156] For example, Figure 7 As shown, assuming that the pixel point connection length OC corresponding to pixel point b is 1.7 and the intersection connection length OD is 2.2, then determine the ratio between the pixel point connection length and the intersection connection length Then the sub-mask result corresponding to the pixel b is determined by setting the perturbation parameter, that is, the intersection D satisfies When , the pixel point C is within the triangle ΔAOB, and the sub-mask result corresponding to the pixel point b is determined to be 1, that is, the disturbance parameter is set.

[0157] Among them, the sub-mask result can be expressed as:

[0158]

[0159] 0 means not setting the disturbance parameter, and 1 means setting the disturbance parameter.

[0160] S2122: If it is determined that the length of the line connecting the pixel points is greater than the length of the line connecting the intersection points, then the sub-mask result is determined to be that the disturbance parameters are not set.

[0161] In an embodiment of the present application, it is determined whether the length of the pixel line corresponding to the pixel point b is greater than the length of the intersection line. If it is determined that the length of the pixel line corresponding to the pixel point b is less than the length of the intersection line, that is, the ratio between the pixel line length and the intersection line length is less than 1, then the sub-mask result corresponding to the pixel point b is determined to be without setting the disturbance parameter.

[0162] For example, Figure 7 As shown, assuming that the pixel point connection length OC corresponding to pixel point b is 2.2 and the intersection connection length OD is 1.7, the ratio between the pixel point connection length and the intersection connection length is determined as Then the sub-mask result corresponding to the pixel point b is determined as not setting the perturbation parameter, that is, the intersection point D satisfies When , the pixel point C is not within the triangle ΔAOB, and the sub-mask result corresponding to the pixel point b is determined to be 0, that is, the disturbance parameter is not set.

[0163] Therefore, when the sub-mask result is determined based on the shape parameter in the above manner, the sub-mask result can be converted into a differentiable form, and then the shape parameter can be converted into a differentiable form. Therefore, when the attack image is generated based on the shape parameter, the attack image can be optimized, thereby improving the attack success rate of the attack image.

[0164] S213: Generate a mask result including each sub-mask result.

[0165] In the embodiment of the present application, after each sub-mask result is obtained, a corresponding mask result can be generated based on each sub-mask result.

[0166] The generated mask result is a mask result including each sub-mask result.

[0167] In an embodiment of the present application, after obtaining each sub-mask result, all pixels in the entire original image can be judged accordingly, thereby obtaining a binary mask image of the same size as the original image, and generating a mask result containing the sub-mask results. Therefore, the mask result is a set of sub-mask results containing each sub-mask result.

[0168] Therefore, based on the above method, all pixels in the original image are represented by mask parameters, which can generate attack areas of different shapes, thereby improving the success rate of the attack.

[0169] S22: Based on each recognition result label, the disturbance parameters corresponding to the corresponding original image and the associated shape parameter set are updated respectively to obtain an attack image including the attack area.

[0170] In an embodiment of the present application, based on each recognition result label, the disturbance parameters corresponding to the corresponding original image are updated respectively, and the associated shape parameter set is updated, so as to obtain the updated disturbance parameters and the updated shape parameter set corresponding to the corresponding original image, and based on the updated disturbance parameters corresponding to each original image and the updated associated shape parameter set, an attack image containing an attack area is generated.

[0171] Optionally, in the embodiment of the present application, when executing S22, it is necessary to update the disturbance parameters corresponding to each original image and the associated shape parameter set. Specifically, taking the above original image a as an example, the process of updating the disturbance parameters corresponding to the original image and the associated shape parameter set is introduced as follows. Fig.11 As shown, it is a flow chart of a method for updating a set of disturbance parameters and shape parameters in an embodiment of the present application:

[0172] S221: Input the original image a with perturbation parameters into the trained parameter update model to obtain the corresponding model recognition result, update the corresponding perturbation parameters and the associated shape parameter set based on the model recognition result and the corresponding recognition result label, and re-input the updated original image with perturbation parameters into the parameter update model until the cross entropy between the model recognition result and the recognition result label meets the update termination condition, and obtain the updated perturbation parameters and the updated shape parameter set.

[0173] In an embodiment of the present application, an original image b with perturbation parameters is input into a trained parameter update model. Based on the parameter update model, the original image b with perturbation parameters is identified, and the probability that the original image b belongs to each object type is determined respectively, and the object type corresponding to the highest probability is used as the model recognition result output by the parameter update model. Then, according to the model recognition result determined by the parameter update model and the recognition result label corresponding to the original image b, the cross entropy between the model recognition result and the recognition result label is determined, and it is judged whether the determined cross entropy meets the end update condition. If it is determined that the cross entropy does not meet the end update condition, the perturbation parameters and the associated shape parameter set are updated accordingly to obtain the updated perturbation parameters and the associated shape parameter set. Based on the updated perturbation parameters and the associated shape parameter set, the updated original image b with perturbation parameters is obtained. Then, the updated original image b with perturbation parameters is re-input into the parameter update model until the cross entropy between the model recognition result and the recognition result label meets the end update condition, and the updated perturbation parameters and the updated shape parameter set are obtained.

[0174] It should be noted that the original image with perturbation parameters can be expressed as:

[0175] x adv =δθM O +xΘ(1-M O )

[0176] Among them, x adv is the original image with perturbation parameters, δ is the perturbation parameter, M O is the sub-mask result, and x is the original image.

[0177] Optionally, in the embodiment of the present application, a possible implementation method is provided for updating the corresponding disturbance parameters and shape parameters, see Fig.12 FIG. 1 is a flow chart of a parameter updating method in an embodiment of the present application, which specifically includes:

[0178] S2211: Determine an updated disturbance parameter based on the current disturbance parameter, the first learning rate and the preset penalty coefficient.

[0179] Among them, the penalty coefficient is used to control the size of the attack area.

[0180] In an embodiment of the present application, the gradient of the loss function between the model recognition result and the recognition result label to the perturbation parameter is calculated, and the gradient is clipped to obtain the clipped gradient, and the updated perturbation parameter is determined based on the current perturbation parameter, gradient, first learning rate and preset penalty coefficient.

[0181] For example, the updated disturbance parameter in the embodiment of the present application can be expressed as:

[0182]

[0183] Among them, δ is the perturbation parameter, k is the number of iterations, α is the first learning rate, and L is the penalty coefficient.

[0184] It should be noted that in the embodiment of the present application, in order to prevent the attack area from expanding infinitely, a penalty coefficient L for the size of the attack area is added to the loss function. shape =mean(M k ), subject to the penalty coefficient, the attack area will be controlled within a suitable range.

[0185] S2212: Determine an updated shape parameter set based on the current shape parameter set, the second learning rate, and the penalty coefficient.

[0186] In an embodiment of the present application, in an embodiment of the present application, the gradient of the loss function between the model recognition result and the recognition result label for each shape parameter is calculated, and the gradient is clipped to obtain the clipped gradient, and based on the shape parameters, gradient, second learning rate and preset penalty coefficient in the current shape parameter set, the updated shape parameter set is determined.

[0187] For example, the updated shape parameters in the embodiment of the present application can be expressed as:

[0188]

[0189] Among them, r is the shape parameter, k is the number of iterations, β is the second learning rate, and L is the penalty coefficient.

[0190] It should be noted that in the embodiment of the present application, there is no restriction on the execution order between S2211 and S2212. S2211 may be executed first, and then S2212, that is, the disturbance parameters are updated first, and then the shape parameter set is updated; S2212 may be executed first, and then S2211, that is, the shape parameter set is updated first, and then the disturbance parameters are updated; of course, S2211 and S2212 may also be executed at the same time, that is, the disturbance parameters and the shape parameter set are updated at the same time.

[0191] In this way, when the shape parameter set and the perturbation parameter are updated, the penalty coefficient is introduced, so that the size of the attack area can be controlled within an appropriate range, avoiding the attack area from expanding infinitely, and further improving the attack success rate of the attack image.

[0192] S222: Based on the updated disturbance parameter and the updated shape parameter set, an attack image including the attack area is obtained.

[0193] In an embodiment of the present application, each shape parameter in a shape parameter set is updated through a parameter updating model to obtain an updated shape parameter set, so that the mask result of the original image b is also updated accordingly. Therefore, based on the updated mask result, an updated disturbance parameter is set for the corresponding pixel points of the original image b, thereby generating an attack image containing an attack area.

[0194] In this way, by adopting the gradient back propagation method to jointly optimize the shape parameter set and the perturbation parameter, it is possible to achieve simultaneous optimization of the perturbation parameter and the shape parameter set, thereby improving the attack success rate of the attack image.

[0195] S23: Use each attack image to perform security test on the model to be tested.

[0196] In the embodiment of the present application, each attack image is input into the model to be tested respectively, and the security test of the model to be tested is performed.

[0197] It should be noted that the security testing method in the embodiments of the present application can be used for modules built on the basis of AI models in the face authentication system. The AI ​​model can be a face detection model, a face recognition model, a face liveness detection model, a text recognition model, a pedestrian detection model, an object recognition model, etc., and this is not limited in the embodiments of the present application.

[0198] Among them, the structure of the AI ​​model can be a multi-layer perceptron, a convolutional neural network, a transformer, etc., which is not limited in the embodiments of the present application.

[0199] Optionally, in the embodiment of the present application, a possible implementation method is provided for performing security testing on the model to be tested, see Fig.13 As shown, it is a flow chart of the method for performing security testing on the model to be tested in an embodiment of the present application, which specifically includes:

[0200] S231: Input each attack image into the model to be tested to obtain the corresponding output result.

[0201] In the embodiment of the present application, each attack image is input into the model to be tested respectively to obtain the output result corresponding to each attack image.

[0202] S232: Based on each output result and the corresponding recognition result label, determine the security test result of the model to be tested.

[0203] In an embodiment of the present application, after obtaining each output result, the security test result of the model to be tested is determined based on each output result and the corresponding recognition result label.

[0204] Optionally, in the embodiments of the present application, a possible implementation method is provided for determining the safety test results. Fig.14 FIG. 1 is a flow chart of a method for determining a safety test result in an embodiment of the present application, which specifically includes:

[0205] S2321: Determine the image test result corresponding to the corresponding attack image based on the output result corresponding to each attack image and the corresponding recognition result label.

[0206] In the embodiment of the present application, for each attack image, the following operations are performed respectively: based on the output result corresponding to an attack image and the corresponding recognition result label, the image test result corresponding to the attack image is determined.

[0207] Specifically, if it is determined that the output result corresponding to an attack image is the same as the corresponding recognition result label, then the image test result corresponding to the attack image is determined to be an unsuccessful attack; if it is determined that the output result corresponding to the attack image is different from the corresponding recognition result label, then the image test result corresponding to the attack image is determined to be a successful attack.

[0208] S2322: Determine the attack success rate of the model to be tested based on the number of attack images corresponding to unsuccessful attacks as a result of the image test and the total number of attack images.

[0209] In an embodiment of the present application, the number of attack images whose image test results are unsuccessful attacks is determined, and based on the total number of attack images and the number of attack images whose image test results are unsuccessful attacks, the attack success rate of the model to be tested is determined.

[0210] Specifically, in an embodiment of the present application, the ratio between the number of attack images corresponding to unsuccessful attacks in the image test results and the total number of images can be used as the attack success rate of the model to be tested.

[0211] For example, assuming that the number of attack images corresponding to the image test result that the attack was unsuccessful is 90 and the total number of images is 100, then the attack success rate of the model to be tested is 90%.

[0212] S2323: If it is determined that the attack success rate is less than the success rate threshold, the security test result of the model to be tested is determined to be safe.

[0213] In an embodiment of the present application, it is determined whether the attack success rate is less than the success rate threshold. If it is determined that the attack success rate is less than the success rate threshold, it is determined that the model to be tested meets the security requirements, and a test result indicating that the model to be tested is safe and can be put online is returned to the client.

[0214] S2324: If it is determined that the attack success rate is not less than the success rate threshold, the security test result of the model to be tested is determined to be unsafe.

[0215] In an embodiment of the present application, it is determined whether the attack success rate is less than the success rate threshold value. If it is determined that the attack success rate is not less than the success rate threshold value, it is determined that the model to be tested does not meet the security requirements, and a test result indicating that the model to be tested is unsafe and needs to be optimized is returned to the client.

[0216] Through the above method, the test results of the model to be tested can be returned to the client in a timely manner, so that relevant staff can make adaptive adjustments to the model.

[0217] For example, in the embodiment of the present application, a physical domain attack experiment was conducted by printing and posting. Fig.15As shown in the figure, it is a schematic diagram of the attack effect comparison in the embodiment of the present application. When the attack area is a square, the attack success rate of the attack image is 71.64%, when the attack area is a circle, the attack success rate of the attack image is 63.94%, and when the attack area is a polygon, the attack success rate of the attack image is 92.11%. Fig.15 It can be seen that when the attack area is a deformable polygon, the attack success rate of the attack image is the highest, which can improve the accuracy of the model security test.

[0218] For another example, in the embodiments of the present application, comparative experiments are set up based on different model structures. Fig.16 As shown in FIG. , it is an experimental comparison effect diagram in the embodiment of the present application, such as Fig.16 It can be seen that compared with the current mainstream attack methods, such as GAP and LaVAN, the model testing method in the embodiment of the present application has obvious advantages in attack success rate.

[0219] Optionally, in an embodiment of the present application, for example, an attack image can be generated based on an attack method of Project Gradient Descent (PGD). For another example, an attack image can be generated based on a confused deep learning classification model (DeepFool) or Carlini and Wagner Attacks (C&W). This is not limited in the embodiments of the present application.

[0220] Optionally, in the embodiment of the present application, the attack effect can be improved by optimizing the position of the attack area in the attack image, thereby improving the accuracy of the security test of the model to be tested.

[0221] In the embodiments of the present application, it is possible to simulate well the attacks that the model to be tested may suffer in reality, thereby effectively evaluating the security of the model and providing a good security warning function for AI products.

[0222] Based on the above embodiment, the model testing method in the embodiment of the present application is described in detail using a specific example. Fig.17 As shown, it is an example diagram of the model testing method in the embodiment of the present application, which specifically includes:

[0223] First, original image 1, original image 2 and original image 3 are obtained. Original image 1 is associated with shape parameter set 1, original image 2 is associated with shape parameter set 2, and original image 3 is associated with shape parameter set 3.

[0224] Then, based on shape parameter set 1, the mask result corresponding to the original image 1 is determined to be 0, 0, 0, 1, ..., 0, 0; based on shape parameter set 2, the mask result corresponding to the original image 2 is determined to be 0, 1, 01, 1, 0, 0, ..., 0, 0; based on shape parameter set 3, the mask result corresponding to the original image 3 is determined to be 0, 0, 0, 0, 1, ..., 0, 0.

[0225] Among them, 0 represents setting perturbation parameters for pixels, and 1 represents not setting perturbation parameters for pixels.

[0226] Then, based on the mask result 0,0,0,1,,…,0,0 corresponding to the original image 1, the perturbation parameters are set for each pixel of the original image 1, and based on the mask result 0,1,01,1,0,0,…,0,0 corresponding to the original image 2, the perturbation parameters are set for the original image 2, and based on the mask result 0,0,0,0,1,…,0,0 corresponding to the original image 3, the perturbation parameters are set for the original image 3.

[0227] Then, based on the recognition result label “envelope” corresponding to the original image 1, the perturbation parameter 1 corresponding to the original image 1 and the associated shape parameter set 1 are updated to obtain the updated perturbation parameter 1 and the updated shape parameter set 1, and based on the updated perturbation parameter 1 and the updated shape parameter set 1, an attack image 1 including the attack area 1 and the “envelope” is generated; based on the recognition result label “pen” corresponding to the original image 2, the perturbation parameter 2 corresponding to the original image 2 and the associated shape parameter set 2 are updated to obtain the updated perturbation parameter 2 and the updated shape parameter set 2, and based on the updated perturbation parameter 2 and the updated shape parameter set 2, an attack image 2 including the attack area 2 and the “pen” is generated; based on the recognition result label “ruler” corresponding to the original image 3, the perturbation parameter 3 corresponding to the original image 3 and the associated shape parameter set 3 are updated to obtain the updated perturbation parameter 3 and the updated shape parameter set 3, and based on the updated perturbation parameter 3 and the updated shape parameter set 3, an attack image 3 including the attack area 3 and the “ruler” is generated.

[0228] Finally, based on attack image 1, attack image 2, and attack image 3, a security test is performed on the model to be tested, thereby obtaining a security test result of the model to be tested.

[0229] Based on the same inventive concept, the present application also provides a model testing device. Fig.18 As shown, it is a schematic diagram of the structure of the model testing device 1800, which may include:

[0230] The obtaining unit 1801 is used to obtain each original image and the corresponding recognition result label; each original image is associated with a shape parameter set for generating an attack area, and the shape parameter set associated with each original image is different;

[0231] The first processing unit 1802 is used to determine the mask results corresponding to the corresponding original images based on the shape parameter sets, and set the disturbance parameters for each pixel in the corresponding original image based on the mask results; wherein each mask result represents the setting mode of the disturbance parameters for each pixel in the corresponding original image;

[0232] The second processing unit 1803 is used to update the disturbance parameters corresponding to the corresponding original images and the associated shape parameter set based on each recognition result label, and obtain an attack image including the attack area;

[0233] The testing unit 1804 is used to use various attack images to perform security tests on the model to be tested.

[0234] Optionally, when determining the mask results corresponding to the corresponding original images based on the shape parameter sets, the first processing unit 1802 is further configured to:

[0235] For each original image, perform the following operations:

[0236] Based on a set of shape parameters associated with an original image, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained; wherein the pixel point connection length represents the length of the pixel point connection between the center point and the corresponding pixel point, and the intersection point connection length represents the length between the intersection of the vertex connection line of two vertices obtained based on two adjacent shape parameters and the pixel point connection line and the center point;

[0237] Based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the sub-mask results corresponding to the corresponding pixel point are determined respectively; each sub-mask result represents the setting method of the disturbance parameter of the corresponding pixel point;

[0238] Generates a mask result that contains each sub-mask result.

[0239] Optionally, when obtaining the pixel point connection length and the intersection point connection length corresponding to each pixel point based on a shape parameter set associated with an original image, the first processing unit 1802 is further configured to:

[0240] For each pixel in an original image, perform the following operations:

[0241] Based on a first shape parameter and a second shape parameter associated with a pixel point, and a preset angle, determining a first vertex coordinate of a first vertex corresponding to the first shape parameter, and a second vertex coordinate of a second vertex corresponding to the first shape parameter;

[0242] Based on the first vertex coordinates, the second vertex coordinates, the pixel point coordinates corresponding to a pixel point, and the center point coordinates of an original image, determine the intersection coordinates corresponding to the intersection of the line between the first vertex and the second vertex and the line between the center point and the pixel point;

[0243] Based on the intersection point coordinates and the center point coordinates, determine the length of the intersection point connection line;

[0244] And, based on the pixel point coordinates and the center point coordinates, the length of the pixel point connection line is determined.

[0245] Optionally, when determining the sub-mask results corresponding to the corresponding pixel points based on the length of the line connecting the pixel points and the length of the line connecting the intersection points, the first processing unit 1802 is further configured to:

[0246] For each pixel, perform the following operations:

[0247] If it is determined that the length of the line connecting the pixels corresponding to a pixel point is not greater than the length of the line connecting the intersection points, then the sub-mask result corresponding to the pixel point is determined to be the set disturbance parameter;

[0248] If it is determined that the length of the line connecting the pixel points is greater than the length of the line connecting the intersection points, the sub-mask result is determined to be that the disturbance parameter is not set.

[0249] Optionally, the second processing unit 1803 is further configured to:

[0250] For each original image with perturbation parameters set, the following operations are performed:

[0251] Input an original image with perturbation parameters into the trained parameter update model to obtain the corresponding model recognition result, update the corresponding perturbation parameters and the associated shape parameter set based on the model recognition result and the corresponding recognition result label, and re-input the updated original image with perturbation parameters into the parameter update model until the cross entropy between the model recognition result and the recognition result label meets the end-of-update condition, and obtain the updated perturbation parameters and the updated shape parameter set;

[0252] Based on the updated disturbance parameters and the updated shape parameter set, an attack image including the attack area is obtained.

[0253] Optionally, when updating the corresponding disturbance parameter and the associated shape parameter set, the second processing unit 1803 is further configured to:

[0254] Determining updated disturbance parameters based on current disturbance parameters, a first learning rate, and a preset penalty coefficient, wherein the penalty coefficient is used to control the size of the attack area; and

[0255] Based on the current shape parameter set, the second learning rate, and the penalty coefficient, an updated shape parameter set is determined.

[0256] Optionally, the testing unit 1804 is further configured to:

[0257] Input each attack image into the model to be tested and obtain the corresponding output result;

[0258] Based on each output result and the corresponding recognition result label, the security test result of the model to be tested is determined.

[0259] Optionally, when determining the security test result of the model to be tested based on each output result and the corresponding identification result label, the testing unit 1804 is further used to:

[0260] Determine the image test result corresponding to the corresponding attack image based on the output result corresponding to each attack image and the corresponding recognition result label;

[0261] Determine the attack success rate of the model to be tested based on the number of attack images corresponding to unsuccessful attacks as a result of the image test and the total number of images of each attack image;

[0262] If it is determined that the attack success rate is less than the success rate threshold, the security test result of the model to be tested is determined to be safe;

[0263] If it is determined that the attack success rate is not less than the success rate threshold, then the security test result of the model to be tested is determined to be unsafe.

[0264] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0265] In some possible implementations, the model testing device according to the present application may include at least a processor and a memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the model testing method according to various exemplary implementations of the present application described in this specification. For example, the processor may execute the following steps: Figure 2 Follow the steps shown in .

[0266] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Fig.19 As shown, it includes a memory 1901 , a communication module 1903 and one or more processors 1902 .

[0267] The memory 1901 is used to store computer programs executed by the processor 1902. The memory 1901 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0268] The memory 1901 may be a volatile memory, such as a random-access memory (RAM); the memory 1901 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1901 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1901 may be a combination of the above memories.

[0269] The processor 1902 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1902 is used to implement the above model testing method when calling the computer program stored in the memory 1901 .

[0270] The communication module 1903 is used to communicate with terminal devices and other servers.

[0271] The specific connection medium between the memory 1901, the communication module 1903 and the processor 1902 is not limited in the embodiment of the present application. Fig.19 In the embodiment, the memory 1901 and the processor 1902 are connected via a bus 1904. The bus 1904 is Fig.19 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1904 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.19The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0272] The memory 1901 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the model testing method of the embodiment of the present application. The processor 1902 is used to execute the above-mentioned model testing method, such as Figure 2 shown.

[0273] In some possible implementations, various aspects of the model testing method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the model testing method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the following steps: Figure 2 Follow the steps shown in .

[0274] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0275] The program product of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a computing device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, device or device.

[0276] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0277] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0278] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user equipment, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0279] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0280] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0281] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0282] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0283] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A model testing method, characterized in that: include: Obtain each original image and its corresponding recognition result label; Each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with the original images are different; Based on each shape parameter set, a mask result corresponding to the corresponding original image is determined respectively, and based on each mask result, a perturbation parameter is set for each pixel point in the corresponding original image respectively; wherein each mask result represents a setting method of a perturbation parameter for each pixel point in the corresponding original image; Based on each recognition result label, the disturbance parameters corresponding to the corresponding original image and the associated shape parameter set are updated respectively to obtain an attack image containing the attack area; Use various attack images to perform security tests on the model to be tested.

2. The method according to claim 1, characterized in that The step of determining mask results corresponding to the corresponding original images based on the shape parameter sets includes: For each of the original images, the following operations are performed respectively: Based on a set of shape parameters associated with an original image, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained; wherein the pixel point connection length represents the length of the pixel point connection between the center point and the corresponding pixel point, and the intersection point connection length represents the length between the intersection of the vertex connection line and the pixel point connection line of two vertices obtained based on two adjacent shape parameters and the center point; Based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the sub-mask results corresponding to the corresponding pixel point are determined respectively; each sub-mask result represents the setting method of the disturbance parameter of the corresponding pixel point; The mask result including each sub-mask result is generated.

3. The method according to claim 2, characterized in that The method of obtaining the pixel point connection length and the intersection point connection length corresponding to each pixel point based on a shape parameter set associated with an original image includes: For each pixel in an original image, perform the following operations: Based on a first shape parameter and a second shape parameter associated with a pixel point, and a preset angle, determining a first vertex coordinate of a first vertex corresponding to the first shape parameter, and a second vertex coordinate of a second vertex corresponding to the first shape parameter; Determine, based on the first vertex coordinates, the second vertex coordinates, the pixel point coordinates corresponding to the one pixel point, and the center point coordinates of the one original image, the intersection coordinates corresponding to the intersection of the line between the first vertex and the second vertex and the line between the center point and the pixel point; Determine the length of the line connecting the intersection points based on the intersection point coordinates and the center point coordinates; And, based on the pixel point coordinates and the center point coordinates, the length of the pixel point connection line is determined.

4. The method according to claim 3, characterized in that The determining of the sub-mask results corresponding to the corresponding pixel points based on the length of the line connecting each pixel point and the length of the line connecting each intersection point respectively includes: For each pixel point, the following operations are performed respectively: If it is determined that the length of the pixel point connection line corresponding to a pixel point is not greater than the length of the intersection point connection line, then determining the sub-mask result corresponding to the pixel point is to set the disturbance parameter; If it is determined that the length of the line connecting the pixel points is greater than the length of the line connecting the intersection points, then the sub-mask result is determined to be that no disturbance parameter is set.

5. The method according to any one of claims 1 to 4, characterized in that: The method of updating the disturbance parameters and the associated shape parameter set corresponding to the corresponding original image based on each recognition result label to obtain an attack image containing the attack area includes: For each original image with perturbation parameters set, the following operations are performed: Input an original image with perturbation parameters set into a trained parameter update model to obtain a corresponding model recognition result, update the corresponding perturbation parameters and the associated shape parameter set based on the model recognition result and the corresponding recognition result label, and re-input the updated original image with perturbation parameters set into the parameter update model until the cross entropy between the model recognition result and the recognition result label meets the end-of-update condition, thereby obtaining the updated perturbation parameters and the updated shape parameter set; Based on the updated disturbance parameters and the updated shape parameter set, an attack image including the attack area is obtained.

6. The method according to claim 5, characterized in that The updating of the corresponding disturbance parameters and the associated shape parameter set includes: Determining an updated disturbance parameter based on a current disturbance parameter, a first learning rate, and a preset penalty coefficient, wherein the penalty coefficient is used to control the size of the attack area; and An updated shape parameter set is determined based on the current shape parameter set, the second learning rate, and the penalty coefficient.

7. The method according to claim 1, characterized in that The security test of the model to be tested is performed using each attack image, including: Input each attack image into the model to be tested and obtain the corresponding output result; Based on each output result and the corresponding identification result label, the security test result of the model to be tested is determined.

8. The method according to claim 7, characterized in that The step of determining the security test result of the model to be tested based on each output result and the corresponding identification result label includes: Determine the image test result corresponding to the corresponding attack image based on the output result corresponding to each attack image and the corresponding recognition result label; Determine the attack success rate of the model to be tested based on the number of attack images corresponding to unsuccessful attacks as a result of the image test and the total number of the attack images; If it is determined that the attack success rate is less than the success rate threshold, then the security test result of the model to be tested is determined to be safe; If it is determined that the attack success rate is not less than the success rate threshold, then the security test result of the model to be tested is determined to be unsafe.

9. A model testing device, characterized in that: include: An obtaining unit, used to obtain each original image and the corresponding recognition result label; Each original image is associated with a shape parameter set for generating an attack area, and the shape parameter sets associated with the original images are different; A first processing unit is used to determine the mask results corresponding to the corresponding original images based on the shape parameter sets, and to set the perturbation parameters for each pixel in the corresponding original images based on the mask results; wherein each mask result represents the setting mode of the perturbation parameters for each pixel in the corresponding original image; A second processing unit is used to update the disturbance parameters corresponding to the corresponding original images and the associated shape parameter set based on each recognition result label, so as to obtain an attack image including the attack area; The testing unit is used to use various attack images to perform security tests on the model to be tested.

10. The device according to claim 9, characterized in that When determining the mask results corresponding to the corresponding original images based on the shape parameter sets, the first processing unit is further used to: For each of the original images, the following operations are performed respectively: Based on a set of shape parameters associated with an original image, the pixel point connection length and the intersection point connection length corresponding to each pixel point are obtained; wherein the pixel point connection length represents the length of the pixel point connection between the center point and the corresponding pixel point, and the intersection point connection length represents the length between the intersection of the vertex connection line and the pixel point connection line of two vertices obtained based on two adjacent shape parameters and the center point; Based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the sub-mask results corresponding to the corresponding pixel point are determined respectively; each sub-mask result represents the setting method of the disturbance parameter of the corresponding pixel point; The mask result including each sub-mask result is generated.

11. The device according to claim 10, characterized in that When obtaining the pixel point connection length and the intersection point connection length corresponding to each pixel point based on the shape parameter set associated with an original image, the first processing unit is further used for: For each pixel in an original image, perform the following operations: Based on a first shape parameter and a second shape parameter associated with a pixel point, and a preset angle, determining a first vertex coordinate of a first vertex corresponding to the first shape parameter, and a second vertex coordinate of a second vertex corresponding to the first shape parameter; Determine, based on the first vertex coordinates, the second vertex coordinates, the pixel point coordinates corresponding to the one pixel point, and the center point coordinates of the one original image, the intersection coordinates corresponding to the intersection of the line between the first vertex and the second vertex and the line between the center point and the pixel point; Determine the length of the line connecting the intersection points based on the intersection point coordinates and the center point coordinates; And, based on the pixel point coordinates and the center point coordinates, the length of the pixel point connection line is determined.

12. The device according to claim 11, characterized in that When determining the sub-mask results corresponding to the corresponding pixel points based on the length of the line connecting each pixel point and the length of the line connecting each intersection point, the first processing unit is further used to: For each pixel point, the following operations are performed respectively: If it is determined that the length of the pixel point connection line corresponding to a pixel point is not greater than the length of the intersection point connection line, then determining the sub-mask result corresponding to the pixel point is to set the disturbance parameter; If it is determined that the length of the line connecting the pixel points is greater than the length of the line connecting the intersection points, then the sub-mask result is determined to be that no disturbance parameter is set.

13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of any one of the methods of claims 1-8.

14. A computer-readable storage medium, characterized in that: It includes program code. When the storage medium is run on an electronic device, the program code is used to enable the electronic device to execute the steps of any method described in claims 1-8.

15. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions implement the steps of any method described in claims 1-8 when executed by a processor.

Images