A method, device, storage medium and electronic device for defending against model attacks

Abstract

In the method for defending against model attacks provided in the specification, according to a determined target service, a target model is determined from among preset candidate models that can execute the target service to execute the target service, a first output result is obtained, the first output result is transformed according to a transformation function preset by the target model to obtain a second output result, a final output result of the target model is determined according to the second output result, and the target service is executed according to the final output result. From the above method, it can be seen that by randomly determining a target model from multiple candidate models to execute a service, the difficulty of cracking the model structure through multiple attacks is increased, and by transforming the first output result output by the target model, the output results of different candidate models are unified, the attack difficulty is further improved, and the model security is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and in particular to a method and device for defending against model attacks, a storage medium, and an electronic device. BACKGROUND

[0002] With the development of Internet technology, people pay more and more attention to the protection of user privacy data. Online deployed models are vulnerable to attacks, and a common attack method is the approximate gradient attack. Attackers send requests carrying input data to the online model, obtain the output result of the online model, and continuously adjust the input data to ultimately make the online model output the result that the attacker wants.

[0003] To solve this problem, existing technologies use methods such as enhancing the robustness of online models or using adversarial training when training models to enhance the ability of online models to resist approximate gradient attacks. Although the above methods can reduce the probability of model being attacked by increasing the number of requests that need to be sent, in theory, the attacker can still attack the online model by continuously trying.

[0004] Therefore, the present application proposes a method for defending against model attacks to resist attacks by attackers. SUMMARY

[0005] The present application provides a method and device for defending against model attacks, a storage medium, and an electronic device to at least partially solve the above problems.

[0006] The present application uses the following technical solutions:

[0007] The present application provides a method for defending against model attacks, the method comprising:

[0008] receiving a service request carrying service data, determining a target service to be executed according to the service request;

[0009] determining a target model from a plurality of candidate models for executing the target service;

[0010] inputting the service data into the target model, and determining a first output result output by the target model;

[0011] transforming the first output result according to a preset transformation function corresponding to the target model to determine a second output result corresponding to the first output result; wherein the target value range of the second output result after transformation of the first output result output by different candidate models is consistent;

[0012] According to the second output result, a final output result of the target model is determined, and the target service is executed according to the final output result.

[0013] Optionally, a target model is determined from preset candidate models for executing the target service, and specifically includes:

[0014] A service type of the target service is determined.

[0015] Pre-set candidate models for executing services of the service type are determined.

[0016] A target model is determined from the candidate models.

[0017] The model structures of the candidate models are not completely identical, and / or the training samples used in training the candidate models are not completely identical.

[0018] Optionally, the method further includes:

[0019] An original value range of a first output result of the target model is determined.

[0020] According to a preset target value range, a linear mapping relationship between the original value range of the first output result and the target value range is determined.

[0021] According to the linear mapping relationship, a transformation function of the target model is generated.

[0022] Optionally, the method further includes:

[0023] Business data is determined, and according to the business data, a value range of an output result of the candidate models is determined.

[0024] For each candidate model, according to a preset value range, a linear mapping relationship between a value range of an output result of the candidate model and the preset value range is determined.

[0025] According to the linear mapping relationship, a transformation function of the candidate model is generated.

[0026] Optionally, according to the second output result, a final output result of the target model is determined, and specifically includes:

[0027] The second output result is quantized, and the quantized second output result is taken as the final output result of the target model.

[0028] Optionally, the second output result is quantized, and specifically includes:

[0029] According to a preset number of digits, low-bit values lower than the number of digits in the second output result are determined.

[0030] determining a quantized value corresponding to the low-bit value according to a preset quantization rule;

[0031] determining a quantized second output result according to the high-bit value not lower than the number of bits in the value in the second output result and the quantized value.

[0032] Optionally, the method further comprises:

[0033] determining a service type of the target service, and determining, according to the service type, preset candidate models for executing services of the service type;

[0034] determining service data of historical services of the service type as test samples, and determining execution results of the historical services as labels of the test samples;

[0035] inputting the test samples into the candidate models respectively to obtain third output results output by the candidate models;

[0036] for each candidate model, determining a service execution result of executing a service based on the third output result output by the candidate model as a verification output result of the candidate model;

[0037] comparing the verification output results with the labels of the test samples to determine comparison results, and determining, according to the comparison results, probabilities of the candidate models being selected as the target model; wherein the comparison results are negatively correlated with the probabilities of the candidate models being selected as the target model;

[0038] determining a target model from preset candidate models for executing the target service, specifically comprising:

[0039] determining the target model from the candidate models according to the determined probabilities of the candidate models being selected as the target model.

[0040] Optionally, the method further comprises:

[0041] for each candidate model, determining whether the probability of the candidate model being selected as the target model is less than a preset probability;

[0042] if yes, the candidate model is taken offline;

[0043] if no, the online state of the candidate model is maintained.

[0044] The specification provides a device for defending against model attacks, the device comprising:

[0045] a receiving device configured to receive a service request carrying service data, and determine a target service to be executed according to the service request;

[0046] determining means for determining a target model from preset candidate models that can execute the target service;

[0047] first output means for inputting the service data into the target model to determine a first output result output by the target model;

[0048] second output means for transforming the first output result according to a preset transformation function corresponding to the target model to determine a second output result corresponding to the first output result; wherein the target value range of the second output result after transformation of the first output result output by different candidate models is consistent;

[0049] final output means for determining a final output result of the target model according to the second output result and executing the target service according to the final output result.

[0050] The specification provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for defending against model attacks.

[0051] The specification provides an electronic device, which includes a storage, a processor, and a computer program stored in the storage and executable on the processor, and the processor implements the method for defending against model attacks when executing the program.

[0052] The above at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0053] In the method for defending against model attacks provided by the specification, after a target model is determined from preset candidate models that can execute a target service to execute the target service and a first output result is obtained, the first output result is transformed according to a preset transformation function of the target model to obtain a second output result, a final output result of the target model is determined according to the second output result, and the target service is executed according to the final output result.

[0054] As can be seen from the above method, by randomly determining a target model from multiple candidate models to execute a service, the difficulty of cracking the model structure through multiple attacks is increased, and by transforming a first output result output by the target model, the output results of different candidate models are unified, the attack difficulty is further improved, and the model security is improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are included to provide a further understanding of the present description and constitute a part of the present description, illustrate the illustrative embodiments of the present description and the description thereof serve to explain the present description, and do not constitute an improper limitation on the present description. In the drawings:

[0056] Figure 1 A flowchart of a defense model attack provided in the present description;

[0057] Figure 2 A schematic diagram of determining a second output result provided in the present description;

[0058] Figure 3 A schematic diagram of a whole process of a defense model attack provided in the present description;

[0059] Figure 4 A schematic diagram of an apparatus of a defense model attack provided in the present description;

[0060] Figure 5 A schematic diagram of an electronic device corresponding to Figure 1 provided in the present description. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present description clearer, the technical scheme of the present description will be described in detail below in combination with the specific embodiments of the present description and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present description, not all the embodiments. Based on the embodiments in the present description, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0062] The technical scheme provided by each embodiment of the present description will be described in detail below in combination with the drawings.

[0063] Figure 1 A flowchart of a method of a defense model attack provided in the present description, specifically comprising the following steps:

[0064] S100: receiving a service request carrying service data, and determining a target service to be executed according to the service request.

[0065] The execution subject of the defense model attack provided in the present description can be a server or an electronic device such as a personal computer (PC). Since the attack usually comes from the outside, a server is generally used to call the model to execute the service and resist the attack of the attacker. Therefore, the method of the defense model attack provided in the present description will be described below taking the server as the execution subject.

[0066] In the embodiments of the present specification, the server can receive a service request, which can come from a user, and the source of the service request is not specifically limited in the present specification. The server determines a target service server to be executed according to the received service request, inputs the service data into the corresponding model as input, and executes the target service according to the output result of the model. In the embodiments of the present specification, the attack refers to that an attacker sends a service request to the model for the purpose of mastering the model structure of the model. The attacker can continuously adjust the service data (i.e., the service data input into the model) carried by the service request according to the output result of the model until the attacker determines the model structure, so that the attacker can initiate a service request according to the demand, and the output of the model is the result expected by the attacker. Wherein, each service request of the attacker can be regarded as an attack.

[0067] S101: determining a target model from preset candidate models for executing the target service.

[0068] Generally, after receiving a service request, the server inputs the service data into a preset single model to obtain an output result. However, since the model structure and the model parameters of the single model are fixed, the attacker can continuously initiate a service request and adjust the input according to the output result of the single model until the model structure of the single model is tried out. It can be seen that when only one model is preset to execute the same type of service, the ability to defend against attacks is low and the model is easy to be attacked by the attacker.

[0069] Therefore, in the method for resisting model attacks provided in the specification, multiple candidate models are preset for the same type of service. When the server receives a service request and determines a target service to be executed, the server can determine, according to the category of the target service, the candidate models that can execute the target service from the preset models. Then, the server determines a target model from the multiple candidate models. After the target model is determined, the server inputs service data of the target service into the target model to obtain an output result. For example, 30 models for executing various types of services are preset in the server. According to the received service data, the service type of the target service is determined, and it is determined that the number of models that can execute the target service is 7. Then, the 7 models are used as candidate models, and one of the candidate models is randomly selected as the target model to execute the target service. Since the target model for executing the service is uncertain each time, although the output result may always be the output result of the same model from the perspective of an attacker, the output result is actually the output result of different models, which greatly enhances the ability of the model to resist attacks. An attacker usually adjusts the input according to the output result of the model and then obtains the output result of the model again. However, each candidate model has different sensitivities to data of different dimensions, and the adjusted input may cause a large change in the output result of the candidate model a, but the adjusted input may cause a small change in the output result of the candidate model b. However, by transforming the first output result of the target model and quantifying the second output result obtained after the transformation, the same final output result can be obtained for the same input by each candidate model.

[0070] In the embodiments of the specification, each candidate model is applied to the same type of service, and to increase the security of the model, the model structures of the candidate models can be different. When an attacker wants to master the model structure of the target model through the output result, since the target model for executing the target service is uncertain each time and the model structures of the candidate models are different, the difficulty for the attacker to master the model structure of the target model is greatly increased.

[0071] In addition to making the model structures of the candidate models different, different training samples can be used to train each candidate model when training the candidate models. The use of different training samples will make the model parameters of each candidate model different, which can also enhance the resistance of the model to attacks.

[0072] Of course, in order to maximize the ability of the model to resist attacks, different training samples can be used when training each candidate model, and each candidate model uses a different model structure. If the model structures of each candidate model are the same, and the training samples used when training each candidate model are also the same, although the number of models increases, it is equivalent to selecting one of the models in the same model, and there is no difference in essence from only setting one model to perform the target business, and the security of the model is not increased.

[0073] S102: input the service data into the target model, and determine a first output result output by the target model.

[0074] After the target model is determined, the service data carried by the service request is input into the target model to obtain a first output result. In order to increase the security of the model, the output result of the model needs to be processed, and therefore, the first output result can be in the form of a numerical value, and the score is easier to transform to resist model attacks. The score can be an integer or a decimal number, which is not specifically limited in the specification, and the value range of the score can also be set according to actual conditions.

[0075] Generally, the model output result can be a classification result or a numerical value (for example, a probability or a score), and in the method provided in the specification, in order to increase the attack cost of the attacker, the model is randomly selected to perform the business. Therefore, it is also necessary to avoid the problem that the same business execution result is inconsistent due to model differences, and therefore, in order to unify the results of performing the business based on each candidate model, the first output result of the candidate model is a numerical value.

[0076] S103: according to a preset transformation function corresponding to the target model, transforming the first output result to determine a second output result corresponding to the first output result; wherein the target value range of the second output result after the first output result of different candidate models is transformed is consistent.

[0077] In the embodiment of the specification, because the model structures of each candidate model performing the same type of business are different, or the training samples used when training each candidate model are different. Therefore, for the same target business, the output results of each candidate model are different. If the first output result of the target model is directly used as the final output result, although the target model is randomly selected in each candidate model, as long as the attacker increases the number of inputs, that is, increases the number of attacks, the model structure of each candidate model can be easily mastered, that is, only the number of preset models is increased, and the output result of the model is not protected, and the resistance ability of the model cannot be enhanced.

[0078] Therefore, after obtaining the first output result, it is necessary to transform the first output result. For each candidate model, a transformation function is preset according to the value range of the candidate model. When the candidate model is used as the target model, the transformation function can be used to transform the first output result to obtain the second output result, as shown in Figure 2 Figure 2 The target model outputs the first output result, and the first output result is transformed by using the preset transformation function corresponding to the target model (the part indicated by the dashed arrow in the figure) to obtain the second output result. For the same target service, when each candidate model is used as the target model, the differences between the first output results of each candidate model are large due to different value ranges, and the model structure is easy to be determined by an attacker. However, the value ranges of the second output results obtained after transformation are the same, and the differences between the second output results are small. When an attacker attacks multiple times, the output results obtained each time are not easy to be found by the attacker as the results output by different target models, and the security of the target model is increased.

[0079] For example, there are three candidate models that can perform a certain target service, which are candidate model a, candidate model b, and candidate model c. When the attacker attacks for the first time, a is used as the target model, and the first output result is 7.1. When the attacker attacks for the second time, b is used as the target model, and the first output result is 22.1. When the attacker attacks for the third time, c is used as the target model, and the first output result is 0.1. It can be seen that the differences between the first output results are large. If the first output result is directly used as the final output result, as long as the number of attacks by the attacker increases, the total number of candidate models can be easily determined, and the model structure of each candidate model can be determined. However, after the first output result is transformed, when a is used as the target model, the second output result is 5.13; when b is used as the target model, the second output result is 5.12; and when c is used as the target model, the second output result is 5.11. It can be seen that the differences between the second output results are small, and it is not easy for the attacker to find that the target model for each attack is not necessarily the same.

[0080] S104: determining the final output result of the target model according to the second output result, and performing the target service according to the final output result.

[0081] ​In the embodiments of the present disclosure, the first output result and the second output result are in the form of scores, and the second output result can be directly used as the final output result. Alternatively, the final output result of the target model can be determined based on the second output result. For example, if the target service is to determine whether there is a risk, a preset risk threshold is 5.0, and if the second output result exceeds the preset risk threshold, it is determined that the target service has a risk; if the second output result does not exceed the preset risk threshold, it is determined that the target service does not have a risk. For the target service, the second output result of the target model is 3.9, and thus it is determined that the target service does not have a risk, and the result of the determination is used as the final output result of the target model, that is, the final output result is "no".

[0082] Based on Figure 1 As shown in the method for defending against model attacks, in the embodiments of the present disclosure, a target model is determined from a plurality of candidate models that can perform a target service according to the target service, a first output result output by the target model is determined, the first output result is transformed to obtain a second output result, a final output result of the target model is determined based on the second output result, and the target service is performed based on the final output result. By determining a target model from a plurality of candidate models, the target model for performing a service each time can be different from the candidate models, thereby increasing the security of the model. Meanwhile, the first output result of the target model is transformed, thereby further enhancing the defense capability of the model.

[0083] Even for models performing the same type of service, because there are differences in model structures between the models, the same dimension data in the service data has different contribution degrees to the same output result of different models. Therefore, by setting a plurality of candidate models with different structures and selecting a target model from the plurality of candidate models, an attacker cannot determine the correspondence between each dimension data of the input service data and the output result through a large number of attacks, thereby effectively defending against model attacks. For example, the service data includes a and b two dimensions of data, and for two models x and y that output the same result that a user has a risk, the model x outputs that the user has a risk mainly because of the a dimension data, and the model y outputs that the user has a risk mainly because of the b dimension data.

[0084] Similarly, models trained by different training samples have differences between model parameters, and thus different models have different "sensitivity" to different dimensions of data, that is, the same dimension data has different contribution degrees to the same output result of different models. For example, the b dimension data in each training sample for training the model x is consistent, and thus the model x is less likely to determine whether a user has a risk based on the b dimension data, and is more likely to determine the output result based on the a dimension data.

[0085] Of course, in the present specification, the model structures of the various candidate models can not be completely consistent, or the training samples can not be completely consistent, or both, and the present specification does not limit this, as long as the data dimensions of the various candidate models are not completely consistent, the effect of defending against model attacks can be achieved.

[0086] In the embodiments of the present specification, a plurality of models are preset, when the server receives a service request, determines a target service, and determines the service type of the target service. According to the service type, the server can filter out each candidate model that can execute the target service from the plurality of preset models, and randomly determine a candidate model as a target model from each candidate model, and determine a final output result, and execute the target service according to the final output result, as shown in Figure 3 Figure 3 In the embodiments of the present specification, a plurality of models are preset, when the server receives a service request, determines a target service, and determines the service type of the target service. According to the service type, the server can filter out each candidate model that can execute the target service from the plurality of preset models, and randomly determine a candidate model as a target model from each candidate model, and determine a final output result, and execute the target service according to the final output result, as shown in Figure 3 In the embodiments of the present specification, a plurality of models are preset, when the server receives a service request, determines a target service, and determines the service type of the target service. According to the service type, the server can filter out each candidate model that can execute the target service from the plurality of preset models, and randomly determine a candidate model as a target model from each candidate model, and determine a final output result, and execute the target service according to the final output result, as shown in

[0087] After obtaining the first output result output by the target model, the first output result needs to be transformed according to the preset transformation function of the target model. The method for determining the transformation function is: determining a set of input data, determining the original value range of the first output result of the target model according to the input data; determining the linear mapping relationship between the original value range of the first output result and the target value range according to the preset target value range; and generating the transformation function of the target model according to the linear mapping relationship. The input data can be random data, but in order to improve the efficiency of determining the original value range of the target model, specific input data can also be determined according to the interval of the historical first output result of the target model, and the efficiency of determining the original value range of the target model is improved.

[0088] Since the target model is determined from each candidate model, the above method for determining the transformation function is a method for determining the transformation function of each candidate model. After the training of each candidate model is completed, the transformation function of each candidate model is determined by the above method, and the transformation function corresponding to each candidate model is stored. When the first output result of the target model is obtained, the transformation function corresponding to the target model is called to obtain the second output result. ​

[0089] In order to further enhance the attack resistance of the model, when the second output result after transformation of the first output result is obtained, the second output result can also be quantized, and the quantization method is: determining low bit values in the second output result lower than a preset number of bits according to the number of bits; determining a quantized value corresponding to the low bit values according to a preset quantization rule; and determining a quantized second output result according to high bit values in the second output result not lower than the number of bits and the quantized value.

[0090] For example, the second output result obtained by the first attack is 4.114, and the second output result obtained by the second attack is 4.144. The attacker can determine the adjustment direction of the input data used for attack according to the two second output results. If the second output result is quantized, and the preset number of bits is 2, the second output result of the first attack is kept as 4.1, and the second output result of the second attack is also kept as 4.1. In this way, the attacker cannot determine how the adjustment direction of the input data will affect the output result of the target model, thereby increasing the attack resistance of the model.

[0091] Suppose the second output result is 8.3265, and the preset number of bits is 3. The quantization rule can take 0 for the low bit values lower than the preset number of bits, that is, the quantized result of the second output result is 8.32; or according to the rounding rule, it is determined whether the low bit values resisting the preset number of bits are greater than 5. If greater than 5, the previous bit value of the low bit value is increased by 1, and the quantized result of the second output result is 8.33. Different quantization rules result in different results, and the quantization rule is not limited in this specification.

[0092] Since the accuracy of the model will gradually decrease over time, the model can be automatically put on and off line by setting a probability for the model. The initial selection probability of each candidate model can be set according to the situation, and the probability of the candidate model being selected as the target model can be increased or decreased, so that the accuracy of the business based on each candidate model is ensured while the security of the model is ensured. The initial selection probability refers to the probability of the candidate model being selected as the target model. The initial selection probability is the probability set when each candidate model is trained and just put on line. At this time, the performance of each candidate model is good, so the initial selection probability of each candidate model is uniform.

[0093] Firstly, the service type of the target service is determined, and according to the service type, preset each candidate model for executing the service of the service type is determined; the service data of the historical service of the service type is determined as a test sample, and the execution result of the historical service is determined as the label of the test sample; the test sample is input into each candidate model respectively, and the third output result output by each candidate model is obtained; for each candidate model, the service execution result based on the third output result output by the candidate model is determined as the verification output result of the candidate model; each verification output result is compared with the label of the test sample, a comparison result is determined, and according to the comparison result, the probability of each candidate model being selected as the target model is determined. Wherein, the smaller the difference of the comparison result is, the higher the probability of the candidate model being selected as the target model is; the larger the difference of the comparison result is, the lower the probability of the candidate model being selected as the target model is.

[0094] The server can determine the target model from each candidate model according to the probability of each candidate model. The probability of each candidate model can also be monitored at any time. When the probability is less than a preset probability, it proves that the performance of the model has decreased and the model is not suitable for continuous operation, and then the state of the candidate model is marked as an offline state. At this time, the candidate model will no longer have the opportunity to be determined as the target model until the state of the candidate model is marked as an online state again. When the probability is not less than the preset probability, it proves that the model can continue to operate, and then the state of the candidate model is kept as the online state. At this time, the candidate model still has the opportunity to be determined as the target model. The candidate model marked as the offline state can wait to be recycled by the server according to a preset recycling period.

[0095] The above is the method for defending model attack provided by one or more embodiments of the present specification. Based on the same idea, the present specification also provides a corresponding device for defending model attack, as shown in Figure 4 .

[0096] Figure 4 The device for defending model attack provided by the present specification is shown in the figure, which specifically includes:

[0097] The receiving module 401 is configured to receive a service request carrying service data, and determine a target service to be executed according to the service request;

[0098] The determining module 402 is configured to determine a target model from each candidate model for executing the target service;

[0099] The first output module 403 is configured to input the service data into the target model, and determine a first output result output by the target model;

[0100] The second output module 404 is configured to transform the first output result according to a preset transformation function corresponding to the target model, to determine a second output result corresponding to the first output result; wherein the target value ranges of the second output results of the first output results output by different candidate models are consistent after transformation.

[0101] The final output module 405 is configured to determine a final output result of the target model according to the second output result, and to execute the target service according to the final output result.

[0102] Optionally, the determining module 402 is specifically configured to determine a service type of the target service, to determine each candidate model preset for executing the service of the service type, and to determine a target model from the each candidate model; wherein the model structures of the each candidate model are not completely same, and / or the training samples used for training the each candidate model are not completely same.

[0103] Optionally, the second output module 404 is further configured to determine an original value range of the first output result of the target model, to determine a linear mapping relationship between the original value range of the first output result and a target value range according to the preset target value range, and to generate the transformation function of the target model according to the linear mapping relationship.

[0104] Optionally, the final output module 405 is specifically configured to quantize the second output result, and to take the quantized second output result as the final output result of the target model.

[0105] Optionally, the final output module 405 is specifically configured to determine a low-bit value lower than a preset number of bits in the second output result, to determine a quantized value corresponding to the low-bit value according to a preset quantization rule, and to determine a quantized second output result according to a high-bit value not lower than the number of bits in the second output result and the quantized value.

[0106] Optionally, the final output module 405 is further configured to determine a service type of the target service, and determine preset candidate models for performing services of the service type according to the service type; determine service data of historical services of the service type as test samples, and determine execution results of the historical services as labels of the test samples; input the test samples into the candidate models respectively to obtain third output results output by the candidate models; for each candidate model, determine a service execution result of performing a service based on the third output result output by the candidate model as a verification output result of the candidate model; compare the verification output results with the labels of the test samples to determine comparison results, and determine probabilities of the candidate models being selected as the target model according to the comparison results; and the comparison results are negatively correlated with the probabilities of the candidate models being selected as the target model.

[0107] The determination module 402 is specifically configured to determine the target model from the candidate models according to the probabilities of the candidate models being selected as the target model.

[0108] Optionally, the final output module 405 is further configured to determine, for each candidate model, whether the probability of the candidate model being selected as the target model is less than a preset probability; if yes, the candidate model is offline; and if no, the candidate model is kept online.

[0109] The specification also provides a computer readable storage medium storing a computer program, and the computer program can be used to execute the above Figure 1 Provided is a defense model attack method.

[0110] The specification also provides Figure 5 The structure diagram of the electronic device is shown. As Figure 5 At the hardware level, the interface matching device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and can also include other hardware required by services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The defense model attack method. Of course, in addition to the software implementation, the specification does not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0111] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0112] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0113] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0114] For the sake of description, the above apparatuses are described in various units with functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in implementing the present specification.

[0115] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage etc.) containing computer usable program code.

[0116] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0117] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0118] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. ​ one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0119] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0120] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer-readable media.

[0121] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0122] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0123] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0125] The various embodiments in the specification are described in progressive manner, and the same or similar parts among the various embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0126] The above only describes the embodiments of the specification, and is not used to limit the specification. The specification can have various changes and variations for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the specification shall be included in the scope of claims of the application.

Claims

1. A method for defending against model attacks, the method comprising: Receive a service request carrying service data, and determine the target service to be executed based on the service request; A target model is randomly determined from a set of candidate models for executing the target service. The business data is input into the target model, and the first output result of the target model is determined. Based on the preset transformation function corresponding to the target model, the first output result is transformed to determine the second output result corresponding to the first output result; wherein, the target value range of the second output result after the transformation of the first output result of different candidate models is consistent; Based on the second output result, the final output result of the target model is determined, and the target service is executed based on the final output result.

2. The method as described in claim 1, wherein randomly determining a target model from a preset pool of candidate models for executing the target service, specifically includes: Determine the business type of the target business; Determine the candidate models for executing the business of the aforementioned business type; The target model is randomly selected from the candidate models. The model structures of the candidate models are not completely the same, and / or the training samples used to train the candidate models are not completely the same.

3. The method of claim 1, further comprising: Determine the original value range of the first output result of the target model; Based on the preset target value range, determine the linear mapping relationship between the original value range of the first output result and the target value range; Based on the linear mapping relationship, the transformation function of the target model is generated.

4. The method of claim 1, further comprising: Determine the business data, and based on the business data, determine the value range of the output results of each candidate model; For each candidate model, a linear mapping relationship is determined between the value range of the output result of the candidate model and the preset value range, based on the preset value range. Based on the linear mapping relationship, the transformation function of the candidate model is generated.

5. The method as described in claim 1, wherein determining the final output result of the target model based on the second output result specifically includes: The second output result is quantized, and the quantized second output result is used as the final output result of the target model.

6. The method as described in claim 5, wherein quantizing the second output result specifically includes: Based on the preset number of digits, determine the lower digits of the second output result that are lower than the preset number of digits; According to the preset quantization rules, determine the quantized value corresponding to the lower-order value; The quantized second output result is determined based on the high-order bits of the second output result that are not less than the number of bits of the numerical value and the quantization value.

7. The method of claim 1, further comprising: Determine the business type of the target business, and based on the business type, determine the preset candidate models for executing the business of the business type; The historical business data of the business type is determined as test samples, and the execution results of the historical business are used as labels for the test samples; The test samples are input into each of the candidate models to obtain the third output result of each candidate model; For each candidate model, the business execution result based on the third output result of the candidate model is determined as the verification output result of the candidate model; Each verification output result is compared with the label of the test sample to determine the comparison result, and based on the comparison result, the probability of each candidate model being selected as the target model is determined; wherein, the difference in the comparison result is negatively correlated with the probability of being selected as the target model; From a preset pool of candidate models for executing the target service, a target model is randomly determined, specifically including: Based on the probability that each candidate model is selected as the target model, the target model is randomly selected from the candidate models.

8. The method of claim 7, further comprising: For each candidate model, determine whether the probability of the candidate model being selected as the target model is less than a preset probability; If so, then the candidate model will be taken offline; If not, then the candidate model will remain online.

9. A device for defending against model attacks, the device comprising: A receiving device is used to receive a service request carrying service data and determine the target service to be executed based on the service request. A determining device is used to randomly determine a target model from a preset pool of candidate models for performing the target service; A first output device is used to input the business data into the target model and determine the first output result output by the target model; The second output device is used to transform the first output result according to the preset transformation function corresponding to the target model, and determine the second output result corresponding to the first output result; wherein the target value range of the second output result after the transformation of the first output result of different candidate models is the same. The final output device is used to determine the final output result of the target model based on the second output result, and to execute the target service based on the final output result.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any one of claims 1 to 8.

Images