Neural network model security improvement method and device, and computing device cluster

Abstract

The invention discloses a security improvement method for a neural network model. The method comprises the following steps: converting the neural network model into a first executable file; performing attack surface scanning of bit flipping attack on the first executable file to obtain potential bit flipping attack points in the first executable file; and inserting a detection function at the potential bit flipping attack point in the first executable file to obtain a second executable file, the detection function being used for detecting whether the bit at the bit flipping attack point is tampered or not. Thus, in the subsequent execution process of executing the second executable file on the computing device, the instruction sequence executed by the CPU in the execution process can be detected, the control flow of the second executable file of the neural network model is verified, and defense of the bit flipping attack of the executable file of the neural network model is completed. The attack cost of bit flipping attacks is remarkably improved, and the attack success rate is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a method, device and computing device cluster for improving the security of a neural network model. Background Art

[0002] With the rapid development of computer technology, AI neural network models are increasingly widely used in production and life. For example, deep neural networks (DNN) are widely needed in audio and video processing, autonomous driving, augmented reality and other fields. At present, in order to use neural network models on low-computing devices (such as IoT devices, smart wearable devices and other devices that do not have graphics processing units (GPUs)), deep learning compilers can be used to compile neural network models to generate optimized high-performance machine code (i.e., binary executable files of neural network models) for different back-end device types, and deploy the machine code on low-computing devices. Of course, executable files of neural network models can also be deployed on high-computing devices. However, after the executable file of the neural network model is deployed on the device, the executable file can often be attacked by hardware vulnerability attacks (such as cache side channel attacks, Rowhammer bit flip attacks, etc.), resulting in the theft of neural network models or reduced accuracy. Summary of the invention

[0003] The present application provides a method, apparatus, computing device cluster, computer storage medium and computer product for improving the security of a neural network model, which can improve the security of the executable file of the neural network model and reduce the probability of the neural network model being stolen or the accuracy being reduced due to attacks on the executable file of the neural network model.

[0004] In a first aspect, the present application provides a method for improving the security of a neural network (NN) model, comprising: converting the NN model into a first executable file; performing an attack surface scan of a bit flip attack on the first executable file to obtain potential bit flip attack points in the first executable file; inserting a detection function into the potential bit flip attack points in the first executable file to obtain a second executable file, wherein the detection function is used to detect whether the bits at the bit flip attack points have been tampered with.

[0005] In this way, during the subsequent execution of the second executable file on the computing device, the instruction sequence executed by the CPU during the execution process can be detected, the control flow of the second executable file of the NN model can be verified, and the defense against the bit flip attack on the executable file of the NN model can be completed, which significantly improves the attack cost of the bit flip attack and reduces the success rate of the attack.

[0006] In a possible implementation, after obtaining the second executable file, the method further includes: deploying the second executable file in a computing device. In this way, the NN model can be used on the computing device.

[0007] In a possible implementation, the detection function is further used to interrupt the reasoning of the NN model and / or re-record the NN model when the bit at the bit flip attack point is tampered, so that the attacker cannot achieve the expected goal.

[0008] In a possible implementation, before converting the NN model into the first executable file, it also includes: inserting a comparison function at the output point of at least one intermediate layer in the NN model, and the comparison function is used to determine the legitimacy of the activation value of the intermediate layer. In this way, when the second executable file is subsequently executed on the computing device, the intermediate data value generated during the execution process can be detected to verify the correctness of the data of the second executable file of the NN model, complete the defense against the bit flip attack of the executable file of the NN model, significantly improve the attack cost of the bit flip attack, and reduce the success rate of the attack.

[0009] In a possible implementation, the comparison function is also used to refuse to return the prediction output of the NN model or reload the NN model when the activation value of the intermediate layer is illegal. In this way, the attacker cannot achieve the expected goal.

[0010] In the second aspect, the present application provides a method for improving the security of a neural network NN model, including: inserting a comparison function at the output point of at least one intermediate layer in the NN model, the comparison function being used to determine the legitimacy of the activation value of the intermediate layer; converting the NN model with the comparison function inserted into a first executable file. In this way, during the subsequent execution of the first executable file on the computing device, the intermediate data values ​​generated during the execution process can be detected to verify the correctness of the data of the first executable file of the NN model, and the defense against the bit flip attack of the executable file of the NN model is completed, which significantly improves the attack cost of the bit flip attack and reduces the success rate of the attack.

[0011] In a possible implementation, after obtaining the first executable file, the method further includes: deploying the first executable file in a computing device. In this way, the NN model can be used on the computing device.

[0012] In a possible implementation, the comparison function is also used to refuse to return the prediction output of the NN model or reload the NN model when the activation value of the intermediate layer is illegal. In this way, the attacker cannot achieve the expected goal.

[0013] In a third aspect, the present application provides a method for improving the security of a neural network (NN) model, comprising: obtaining at least one record fragment from an access record cached on a computing device, an executable file of the NN model being deployed on the computing device, the access record being generated by the executable file executing the NN model, a record fragment being used to record access information of an operator in the NN in the cache; obtaining a representation vector of a record fragment contained in at least one record fragment; and determining whether the executable file of the NN model has a cache side channel vulnerability based on the representation vector of the record fragment related to the same operator in the NN model.

[0014] In this way, without relying on any shared memory or any prior analysis, end-to-end fully automated cache side-channel detection can be achieved, thereby detecting whether there are cache side-channel vulnerabilities in the executable file of the NN model that may leak the model structure, thereby improving the security of the NN model.

[0015] In a possible implementation, the NN model includes a first operator, and at least one record segment includes a first record segment and a second record segment, and the first record segment and the second record segment are both related to the first operator. At this time, based on the characterization vectors of the record segments related to the same operator in the NN, determining whether the executable file of the NN model has a cache side channel vulnerability includes: obtaining the similarity between the characterization vector of the first record segment and the characterization vector of the second record segment; when the similarity is greater than a preset threshold, determining that the executable file of the NN model has a cache side channel vulnerability. In this way, it is possible to determine whether the executable file of the NN model has a cache side channel vulnerability.

[0016] In a possible implementation, the method further includes: outputting warning information when it is determined that the executable file of the NN model has a cache side channel vulnerability. In this way, the administrator can be informed of the vulnerability information and take timely remedial measures.

[0017] In a fourth aspect, the present application provides a NN model security enhancement device including: a model conversion module, a bit flip attack point scanning module and a patch module. Among them, the model conversion module can be used to convert the NN model into a first executable file. The bit flip attack point scanning module can be used to scan the attack surface of the first executable file for bit flip attacks to obtain potential bit flip attack points in the first executable file. The patch module can be used to insert a detection function into the potential bit flip attack point in the first executable file to obtain a second executable file, and the detection function is used to detect whether the bit at the bit flip attack point has been tampered with.

[0018] In a possible implementation, the NN model security improvement device further includes: a deployment module. The deployment module can be used to deploy the second executable file in the computing device after the second executable file is obtained through the patch module.

[0019] In a possible implementation, the detection function is also used to interrupt the reasoning of the NN model and / or re-record the NN model when the bits at the bit flip attack point are tampered with.

[0020] In a possible implementation, the NN model security enhancement device further includes: an insertion module. The insertion module is used to insert a comparison function at an output point of at least one intermediate layer in the NN model before the model conversion module converts the NN model into a first executable file, and the comparison function is used to determine the legitimacy of the activation value of the intermediate layer.

[0021] In a possible implementation, the comparison function is also used to refuse to return the prediction output of the NN model or to reload the NN model when the activation value of the intermediate layer is illegal.

[0022] In a fifth aspect, the present application provides a security improvement device for a NN model, including: an insertion module and a conversion module. The insertion module is used to insert a comparison function at the output point of at least one intermediate layer in the NN model. The comparison function is used to determine the legitimacy of the activation value of the intermediate layer. The conversion module is used to convert the NN model with the comparison function inserted into a first executable file.

[0023] In a possible implementation, the NN model security improvement device further includes: a deployment module. The deployment module can be used to deploy the first executable file in the computing device after the first executable file is obtained through the conversion module.

[0024] In a possible implementation, the comparison function is also used to refuse to return the prediction output of the NN model or to reload the NN model when the activation value of the intermediate layer is illegal.

[0025] In a sixth aspect, the present application provides a security improvement device for a NN model including: a fragment collection module, a vector acquisition module and a vulnerability detection module. Among them, the fragment collection module is used to obtain at least one record fragment from the access record cached on a computing device, on which an executable file of the NN model is deployed, and the access record is generated by the executable file that executes the NN model, and a record fragment is used to record the access information of an operator in the NN in the cache. The vector acquisition module is used to obtain a characterization vector of a record fragment contained in at least one record fragment. The vulnerability detection module is used to determine whether the executable file of the NN model has a cache side channel vulnerability based on the characterization vector of the record fragment related to the same operator in the NN model.

[0026] In a possible implementation, the NN model includes a first operator, and at least one record segment includes a first record segment and a second record segment, and the first record segment and the second record segment are both related to the first operator. At this time, when the vulnerability detection module determines whether the executable file of the NN model has a cache side channel vulnerability based on the characterization vector of the record segment related to the same operator in the NN, it is specifically used to: obtain the similarity between the characterization vector of the first record segment and the characterization vector of the second record segment; when the similarity is greater than a preset threshold, determine that the executable file of the NN model has a cache side channel vulnerability.

[0027] In a possible implementation, the NN model security enhancement device further includes: an early warning module. The early warning module is used to output early warning information when it is determined that the executable file of the NN model has a cache side channel vulnerability.

[0028] In a seventh aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect, or executes the method described in the second aspect or any possible implementation of the second aspect, or executes the method described in the third aspect or any possible implementation of the third aspect.

[0029] In an eighth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect, or executes the method described in the second aspect or any possible implementation of the second aspect, or executes the method described in the third aspect or any possible implementation of the third aspect. The computing device cluster may include one or more computing devices.

[0030] In a ninth aspect, the present application provides a computer program product including instructions, which, when executed by a computing device cluster, causes the computing device cluster to execute the method described in the first aspect or any possible implementation of the first aspect, or the method described in the second aspect or any possible implementation of the second aspect, or the method described in the third aspect or any possible implementation of the third aspect. The computing device cluster may include one or more computing devices.

[0031] It can be understood that the beneficial effects of the fourth to ninth aspects mentioned above can be found in the relevant descriptions of the first to third aspects mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of a method for improving the safety of a neural network model provided in an embodiment of the present application;

[0033] Figure 2 It is a flowchart of another method for improving the safety of a neural network model provided in an embodiment of the present application;

[0034] Figure 3 It is a structural schematic diagram of a safety lifting device of a neural network model provided in an embodiment of the present application;

[0035] Figure 4 It is a structural schematic diagram of another safety lifting device of a neural network model provided in an embodiment of the present application;

[0036] Figure 5 It is a structural schematic diagram of another safety lifting device of a neural network model provided in an embodiment of the present application;

[0037] Figure 6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0038] Figure 7 It is a structural diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0040] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0041] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0043] First, some technical terms involved in this application are introduced.

[0044] (1) Deep learning (DL) compiler

[0045] A deep learning compiler is a programming tool or framework that converts a DNN model into an efficient executable file. A deep learning compiler can optimize the DNN model so that it can run on different hardware platforms to meet the needs of different application scenarios.

[0046] (2) DNN executable file

[0047] The executable file of DNN is an application or code generated by the deep learning compiler, which can execute the reasoning process that is exactly the same as the original DNN model function. The executable file of DNN can be run directly on the central processing unit (CPU) or other computing units, and does not rely on third-party computing libraries.

[0048] (3) Cache side-channel

[0049] Cache side channel is a computer security attack method that analyzes the cache behavior of the target system to infer the user's privacy behavior pattern or privacy data, such as encryption keys, access patterns, etc. Cache side channel attacks may include: Flush+Reload attack, Prime+Probe attack, Rowhammer attack, etc. Among them, Flush+Reload attack is a cache side channel attack that relies on shared memory between the attacker and the victim. It determines the victim's memory access by detecting whether the data in the shared memory is loaded into the cache. Prime+Probe attack is a cache side channel attack that does not rely on shared memory. It determines the victim's memory access by detecting whether the attacker's data is cleared from the cache. Rowhammer attack uses the hardware characteristics of random access memory (RAM) to access the adjacent data of the target data in RAM multiple times. Without directly accessing the target data, it causes the voltage of the target data to change, resulting in a bit flip of the corresponding data.

[0050] (4) Operator

[0051] An operator is a computing unit that can perform simple or complex calculations such as addition, subtraction, multiplication, and division.

[0052] Next, the technical solution provided by this application is introduced.

[0053] For example, Figure 1 The flowchart of a neural network (NN) model security improvement method provided by the embodiment of the present application is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 1 As shown, the security improvement method of the NN model may include the following steps:

[0054] S101. Obtain at least one record fragment from the access records cached on a computing device, on which an executable file of an NN model is deployed. The access record is generated by executing the executable file of the NN model. A record fragment is used to record the access information of an operator in the NN in the cache.

[0055] In this embodiment, an executable file of the NN model may be deployed (i.e., installed or configured) on the computing device. The executable file may be obtained by processing the NN model through a DL compiler. The executable file may be, but is not limited to, binary code. In the cache on the computing device, access records generated by the executable file executing the NN may be stored. Exemplarily, the NN model may be, but is not limited to, a DNN model.

[0056] In this embodiment, a cache side channel attack (such as Prime+Probe attack, etc.) method can be used to collect record fragments from the access records generated by the executable file executing the NN in the cache on the computing device. Among them, a record fragment can be used to record the access information of an operator in the NN in the cache. An operator can be associated with multiple record fragments, and a record fragment is associated with an operator. Exemplarily, a record fragment can be composed of one or more cache lines.

[0057] S102: Obtain a representation vector for each record segment.

[0058] In this embodiment, the acquired record segments may be input into a pre-trained NN (eg, a convolutional neural network (CNN)), so as to obtain a representation vector of each record segment.

[0059] S103. Determine whether the executable file of the NN model has a cache side channel vulnerability based on the representation vector of the record fragment related to the same operator in the NN model.

[0060] In this embodiment, after obtaining the characterization vector of the record fragment, the similarity between the characterization vectors of each record fragment related to the same operator can be calculated. Then, based on the calculated similarity, it is determined whether there is a cache side channel vulnerability. Among them, for the record fragments related to any operator, when the similarity between the characterization vectors of any two record fragments is greater than a preset threshold, it can be determined that there is a cache side channel vulnerability in the executable file of the NN model. Exemplarily, when there is a cache side channel vulnerability in the executable file of the NN model, warning information can be output to the user, such as outputting the location of the vulnerable code and so on.

[0061] In this way, without relying on any shared memory or any prior analysis, end-to-end fully automated cache side-channel detection can be achieved, thereby detecting whether there are cache side-channel vulnerabilities in the executable file of the NN model that may leak the model structure, thereby improving the security of the NN model.

[0062] For example, Figure 2 The flowchart of another NN model security improvement method provided by the embodiment of the present application is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 2 As shown, the security improvement method of the NN model may include the following steps:

[0063] S201. Insert a comparison function at an output point of at least one intermediate layer in the NN model, where the comparison function is used to determine whether the activation value of the intermediate layer conforms to the expected distribution.

[0064] In this embodiment, a comparison function can be inserted into the output point of at least one intermediate layer in the NN model, that is, the NN model is plugged. The comparison function can be used to determine whether the activation value of the intermediate layer conforms to the expected distribution, that is, whether it is legal. When the comparison function determines that the activation value of the intermediate layer does not meet the expected distribution, it can refuse to return the predicted output of the NN model or reload the NN model, so that the attacker cannot achieve the expected goal and attract the attention of the administrator. Exemplarily, the comparison function can be called a data protection module or a data flow integrity (DFI) checker. Exemplarily, the model architecture of the NN model can be obtained through the model description of the NN model. For the model description of the NN model, the NN model can be processed by tools such as a DL compiler to obtain the model description of the NN model. Exemplarily, the format of the model description of the NN model can be ONNX (open neural network exchange) format. Exemplarily, the NN model can be but not limited to a DNN model.

[0065] S202 . Convert the NN model with the comparison function inserted into it into a first executable file of the NN model through a DL compiler.

[0066] S203: Perform a bit-flip attack (BFA) attack surface scan on the first executable file to obtain potential bit-flip attack points in the first executable file.

[0067] In this embodiment, the first executable file may be subjected to a BFA attack surface scan, such as by simulating a Rowhammer attack, to obtain potential bit flip attack points in the first executable file, such as flippable bit positions and directions.

[0068] S204: Insert a detection function into each bit flip attack point in the first executable file to obtain a second executable file, where the detection function is used to detect whether the bit at the bit flip attack point is tampered with.

[0069] In this embodiment, a detection function can be inserted into each bit flip attack point in the first executable file, that is, the first executable file is patched to obtain a second executable file. The detection function can be used to detect whether the bit at the bit flip attack point has been tampered with. Exemplarily, the detection function can compare the bit at the bit flip attack point with the expected value (ground truth) of the bit at the flip attack point stored in advance. When the two are inconsistent, it indicates that the bit at the bit flip attack point has been tampered with. At this time, the NN model can be interrupted and reloaded to prevent the attacker from achieving the expected goal and attract the attention of the administrator. Exemplarily, the detection function can be called a control flow protection module or a control flow integrity (CFI) checker.

[0070] After obtaining the second executable file, the second executable file can be deployed on the computing device. In this way, during the subsequent execution of the executable file on the computing device, the intermediate data values ​​generated during the execution process and the instruction sequence executed by the CPU can be detected, so as to verify the control flow and data correctness of the executable file of the NN model, complete the defense against the bit flip attack of the executable file of the NN model, significantly improve the attack cost of the bit flip attack, and reduce the success rate of the attack.

[0071] It should be noted that in Figure 2 In the security improvement method of the NN model shown, in addition to simultaneously inserting the NN model and patching the executable file of the NN model, it is also possible to only insert the NN model without patching the executable file of the NN model, or, instead of inserting the NN model, directly patch the executable file of the NN model. The specific method can be determined according to the actual situation, and both methods are within the scope of this application.

[0072] It is understandable that the order of execution of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the scope of protection of the present application.

[0073] Based on the method in the above embodiment, an embodiment of the present application provides a problem handling device.

[0074] For example, Figure 3 The schematic diagram of the structure of a safety lifting device of a NN model provided in an embodiment of the present application is shown. Figure 3As shown, the security improvement device 300 of the NN model includes: a model conversion module 301, a bit flip attack point scanning module 302 and a patch module 303. Among them, the model conversion module 301 can be used to convert the NN model into a first executable file. The bit flip attack point scanning module 302 can be used to scan the attack surface of the bit flip attack on the first executable file to obtain potential bit flip attack points in the first executable file. The patch module 303 can be used to insert a detection function into the potential bit flip attack point in the first executable file to obtain a second executable file, and the detection function is used to detect whether the bit at the bit flip attack point has been tampered with.

[0075] In some embodiments, the NN model security enhancement device 300 further includes: a deployment module (not shown in the figure). The deployment module can be used to deploy the second executable file in the computing device after the second executable file is obtained through the patch module 303.

[0076] In some embodiments, the detection function is also used to interrupt the reasoning of the NN model and / or re-record the NN model in the event that the bits at the bit flip attack point are tampered with.

[0077] In some embodiments, the safety improvement device 300 of the NN model further includes: a plugging module (not shown in the figure). The plugging module is used to insert a comparison function at the output point of at least one intermediate layer in the NN model before the model conversion module 301 converts the NN model into the first executable file, and the comparison function is used to judge the legitimacy of the activation value of the intermediate layer.

[0078] In some embodiments, the comparison function is also used to refuse to return the predicted output of the NN model or to reload the NN model when the activation value of the intermediate layer is illegal.

[0079] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0080] For example, Figure 4 A schematic diagram of the structure of another NN model safety lifting device provided in an embodiment of the present application is shown. Figure 4 As shown, the safety improvement device 400 of the NN model includes: an insertion module 401 and a conversion module 402. The insertion module 401 is used to insert a comparison function at the output point of at least one intermediate layer in the NN model. The comparison function is used to judge the legitimacy of the activation value of the intermediate layer. The conversion module 402 is used to convert the NN model with the comparison function inserted into a first executable file.

[0081] In some embodiments, the NN model safety enhancement device 400 further includes: a deployment module (not shown in the figure). The deployment module can be used to deploy the first executable file in the computing device after obtaining the first executable file through the conversion module.

[0082] In some embodiments, the comparison function is also used to refuse to return the predicted output of the NN model or to reload the NN model when the activation value of the intermediate layer is illegal.

[0083] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0084] For example, Figure 5 A schematic diagram of the structure of another NN model safety lifting device provided in an embodiment of the present application is shown. Figure 5 As shown, the security improvement device 500 of the NN model includes: a fragment collection module 501, a vector acquisition module 502 and a vulnerability detection module 503. Among them, the fragment collection module 501 is used to obtain at least one record fragment from the access record cached on the computing device, the executable file of the NN model is deployed on the computing device, the access record is generated by the executable file executing the NN model, and a record fragment is used to record the access information of an operator in the NN in the cache. The vector acquisition module 502 is used to obtain the characterization vector of the record fragment contained in at least one record fragment. The vulnerability detection module 503 is used to determine whether the executable file of the NN model has a cache side channel vulnerability based on the characterization vector of the record fragment related to the same operator in the NN model.

[0085] In some embodiments, the NN model includes a first operator, and at least one record segment includes a first record segment and a second record segment, and the first record segment and the second record segment are both related to the first operator. At this time, when the vulnerability detection module 503 determines whether the executable file of the NN model has a cache side channel vulnerability based on the characterization vector of the record segment related to the same operator in the NN, it is specifically used to: obtain the similarity between the characterization vector of the first record segment and the characterization vector of the second record segment; when the similarity is greater than a preset threshold, determine that the executable file of the NN model has a cache side channel vulnerability.

[0086] In some embodiments, the NN model security enhancement device 500 further includes: an early warning module (not shown in the figure). The early warning module is used to output early warning information when it is determined that the executable file of the NN model has a cache side channel vulnerability.

[0087] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0088] The present application embodiment also provides a computing device 600. Figure 6 As shown, computing device 600 includes: bus 602, processor 604, memory 606 and communication interface 608. Processor 604, memory 606 and communication interface 608 communicate through bus 602. Computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 600.

[0089] The bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 604 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 604 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 606, the processor 604, and the communication interface 608).

[0090] The processor 604 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0091] The memory 606 may include a volatile memory, such as a random access memory (RAM). The processor 604 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0092] The memory 606 stores executable codes, and the processor 604 executes the executable codes to implement the functions of the safety enhancement device of any of the NN models described above, thereby implementing the safety enhancement method of the NN model related to the safety enhancement device of the NN model described in the above embodiment. That is, the memory 606 stores instructions for executing the safety enhancement method of the NN model described in the above embodiment.

[0093] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0094] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0095] like Figure 7 As shown, the computing device cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the digital human lip synchronization evaluation method described in the above embodiment.

[0096] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also respectively store partial instructions for executing the security enhancement method of any one of the NN models described in the above embodiments. In other words, the combination of one or more computing devices 600 can jointly execute instructions for executing the security enhancement method of any one of the NN models described in the above embodiments.

[0097] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the safety enhancement device of any of the aforementioned NN models.

[0098] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network.

[0099] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 7 The connection mode of the computing device cluster is different in that the memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the method in the above embodiment.

[0100] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store some instructions for executing the method in the above embodiment. In other words, the combination of one or more computing devices 600 can jointly execute the instructions for executing the method in the above embodiment.

[0101] Based on the method in the above embodiment, the embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computing device cluster including at least one computing device, the computing device cluster executes the method described in the above embodiment. Exemplarily, the computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0102] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computing device cluster including at least one computing device, the computing device cluster executes the method in the above embodiment.

[0103] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0104] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0106] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for improving the safety of a neural network (NN) model, characterized in that: include: Converting the NN model into a first executable file; Performing a bit flip attack attack surface scan on the first executable file to obtain potential bit flip attack points in the first executable file; A detection function is inserted into a potential bit flip attack point in the first executable file to obtain a second executable file, wherein the detection function is used to detect whether the bit at the bit flip attack point is tampered with.

2. The method according to claim 1, characterized in that After obtaining the second executable file, the method further includes: The second executable file is deployed in a computing device.

3. The method according to claim 1 or 2, characterized in that: The detection function is also used to interrupt the reasoning of the NN model and / or re-record the NN model when the bits at the bit flip attack point are tampered with.

4. The method according to any one of claims 1 to 3, characterized in that: Before converting the NN model into a first executable file, the method further includes: A comparison function is inserted into an output point of at least one intermediate layer in the NN model, and the comparison function is used to judge the legitimacy of the activation value of the intermediate layer.

5. The method according to claim 4, characterized in that The comparison function is also used to refuse to return the predicted output of the NN model or reload the NN model when the activation value of the intermediate layer is illegal.

6. A method for improving the security of a neural network (NN) model, characterized in that: include: Inserting a comparison function at an output point of at least one intermediate layer in the NN model, wherein the comparison function is used to determine the legitimacy of the activation value of the intermediate layer; The NN model into which the comparison function is inserted is converted into a first executable file.

7. The method according to claim 6, characterized in that After obtaining the first executable file, the method further includes: The first executable file is deployed in a computing device.

8. The method according to claim 6 or 7, characterized in that: The comparison function is also used to refuse to return the predicted output of the NN model or reload the NN model when the activation value of the intermediate layer is illegal.

9. A method for improving the safety of a neural network (NN) model, characterized in that: include: Acquire at least one record fragment from the access record cached on the computing device, the computing device having an executable file of the NN model deployed thereon, the access record being generated by executing the executable file of the NN model, one of the record fragments being used to record access information of an operator in the NN in the cache; Obtaining a representation vector of a record segment included in the at least one record segment; Based on the representation vectors of the record fragments related to the same operator in the NN model, it is determined whether the executable file of the NN model has a cache side channel vulnerability.

10. The method according to claim 9, characterized in that The NN model includes a first operator, the at least one record segment includes a first record segment and a second record segment, and the first record segment and the second record segment are both related to the first operator; The determining, based on the representation vector of the record fragments related to the same operator in the NN, whether the executable file of the NN model has a cache side channel vulnerability comprises: Obtaining a similarity between a representation vector of the first record segment and a representation vector of the second record segment; When the similarity is greater than a preset threshold, it is determined that the executable file of the NN model has a cache side channel vulnerability.

11. The method according to claim 9 or 10, characterized in that: Also includes: When it is determined that the executable file of the NN model has a cache side channel vulnerability, a warning message is output.

12. A safety lifting device for a neural network (NN) model, characterized in that: include: A model conversion module, used for converting the NN model into a first executable file; A bit flip attack point scanning module, used for performing a bit flip attack attack surface scan on the first executable file to obtain potential bit flip attack points in the first executable file; The patch module is used to insert a detection function into a potential bit flip attack point in the first executable file to obtain a second executable file, wherein the detection function is used to detect whether the bit at the bit flip attack point is tampered with.

13. A safety lifting device for a neural network (NN) model, characterized in that: include: An insert module, used for inserting a comparison function into an output point of at least one intermediate layer in the NN model, wherein the comparison function is used for judging the legitimacy of the activation value of the intermediate layer; A conversion module is used to convert the NN model inserted with the comparison function into a first executable file.

14. A safety lifting device for a neural network (NN) model, characterized in that: include: A fragment collection module, which obtains at least one record fragment from the access record cached on the computing device, the executable file of the NN model is deployed on the computing device, the access record is generated by executing the executable file of the NN model, and one record fragment is used to record the access information of an operator in the NN in the cache; A vector acquisition module, used to acquire a representation vector of a record segment contained in the at least one record segment; A vulnerability detection module is used to determine whether the executable file of the NN model has a cache side channel vulnerability based on the characterization vector of the record fragment related to the same operator in the NN model.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-5, or executes the method according to any one of claims 6-8, or executes the method according to any one of claims 9-11.

16. A computer-readable storage medium storing a computer program, which, when executed on a computing device cluster comprising at least one computing device, enables the computing device cluster to execute the method described in any one of claims 1 to 5, or the method described in any one of claims 6 to 8, or the method described in any one of claims 9 to 11.

17. A computer program product, characterized in that When the computer program product runs on a computing device cluster including at least one computing device, the computing device cluster executes the method according to any one of claims 1 to 5, or executes the method according to any one of claims 6 to 8, or executes the method according to any one of claims 9 to 11.

Images