Adding adversarial robustness to trained machine learning models

Abstract

Methods and systems are provided for protecting trained machine learning models by one or more processors in a computing system. By adding adversarial protection to one or more trained machine learning models, one or more hardened machine learning models can resist adversarial attacks.

Description

Technical Field

[0001] The present invention relates generally to computing systems and, more particularly, to various embodiments for adding adversarial robustness against adversaries to trained machine learning models in computing systems using computing processors. Background Art

[0002] Computing systems can be found in the workplace, at home, or at school. Due to recent advances in information technology and the increasing popularity of the Internet, a wide variety of computer systems have been used for machine learning. Machine learning is a form of artificial intelligence that allows computers to evolve behavior based on empirical data. Machine learning can use training examples to capture interesting features of unknown underlying probability distributions. Training data can be viewed as examples that illustrate the relationships between observed variables. A major focus of machine learning research is to automatically learn to recognize complex patterns and make intelligent decisions based on data. Therefore, there is a need in the art to address the above issues. Summary of the invention

[0003] From a first aspect, the present invention provides a method for protecting a trained machine learning model in a computing environment through one or more processors, the method comprising: providing one or more enhanced machine learning models to resist adversarial attacks by adding adversarial protection to one or more trained machine learning models.

[0004] From another aspect, the present invention provides a system for protecting trained machine learning models in a computing environment, including: one or more computers having executable instructions, which, when executed, cause the system to: provide one or more enhanced machine learning models to resist adversarial attacks by adding adversarial protection to one or more trained machine learning models.

[0005] Viewed from another aspect, the present invention provides a computer program product for protecting a trained machine learning model in a computing environment via one or more processors, the computer program product comprising a computer-readable storage medium that can be read by a processing circuit and stores instructions for execution by the processing circuit to execute a method for performing the steps of the present invention.

[0006] From another aspect, the present invention provides a computer program stored on a computer readable medium and loadable into the internal memory of a digital computer, comprising software code portions for executing the steps of the present invention when said program is run on a computer.

[0007] From another aspect, the present invention provides a computer program product for protecting trained machine learning models in a computing environment through a processor, the computer program product including a non-transitory computer-readable storage medium having a computer-readable program code portion stored therein, the computer-readable program code portion including: an executable portion that provides one or more enhanced machine learning models to resist adversarial attacks by adding adversarial protection to one or more trained machine learning models.

[0008] Various embodiments are provided for protecting trained machine learning models by one or more processors in a computing system. In one embodiment, by way of example only, a method is provided for adding adversarial robustness to a trained machine learning model against an adversary in a computing system, again by a processor. By adding adversarial protection to one or more trained machine learning models, one or more hardened machine learning models can resist adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to easily understand the advantages of the present invention, the invention briefly described above will be described in more detail by reference to specific embodiments shown in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the present invention and are therefore not to be considered limiting of its scope, the present invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0010] Figure 1 is a block diagram depicting an exemplary cloud computing node according to an embodiment of the present invention;

[0011] Figure 2 is an additional block diagram depicting an exemplary cloud computing environment according to an embodiment of the present invention;

[0012] Figure 3 is an additional block diagram describing an abstract model layer according to an embodiment of the present invention;

[0013] Figure 4 is a flow chart describing an exemplary method for protecting a trained machine learning model against an adversary / adversarial system in a computing environment by a processor in which aspects of the present invention may be implemented;

[0014] Figure 5 is a block diagram describing exemplary systems and functionality for protecting a trained machine learning model against an adversary / adversarial system in a computing environment via a processor in which various aspects of the present invention may be implemented, according to an embodiment of the present invention;

[0015] Figure 6is a block diagram describing another exemplary system and functionality for protecting a trained machine learning model against an adversary / adversarial system for images in a computing environment by a processor in which various aspects of the present invention may be implemented, according to another embodiment of the present invention;

[0016] Figure 7 is a block diagram describing additional exemplary operations for protecting a trained machine learning model against an adversary / adversarial system in a computing environment by a processor in which various aspects of the present invention may be implemented, according to another embodiment of the present invention;

[0017] Figure 8 is a flow chart describing an exemplary method for protecting a trained machine learning model against an adversary / adversarial system in a computing environment by a processor in which various aspects of the present invention may be implemented. DETAILED DESCRIPTION

[0018] The present invention generally relates to the field of artificial intelligence ("AI"), such as machine learning and / or deep learning. Deep learning refers to a class of machine learning algorithms based on learning multiple levels of features or representations of a set of data. Current deep learning methods include feature extraction and transformation using cascaded layers of nonlinear processing units. Feature extraction refers to the process of receiving an initial set of measurement data and constructing derived values ​​(or features) to facilitate subsequent learning and generalization steps. In many cases, higher-level features are derived from lower-level features to produce a hierarchical representation of the measurement data and derived features.

[0019] Furthermore, deep learning algorithms are based on distributed representations. Distributed representations operate under the assumption that the observed (or measured) data are the result of the interaction of one or more factors organized into one or more layers. Conceptually, deep learning introduces another assumption that the layers of factors that interact to provide the measured data represent levels of abstraction or combination. Under this assumption, multiple layers and layer sizes correspond to different amounts of abstraction.

[0020] Overall, deep learning has made significant progress in optimizing decision making and extracting business value from unstructured data such as images, audio, video, and text. However, while deep learning has achieved higher computational performance on various data types and tasks, there are several challenges, weaknesses, and / or security issues with deep neural networks (DNNs). For example, DNNs are vulnerable to adversarial attacks, where an adversary may completely change the behavior of a DNN by making imperceptible changes to its inputs. In addition, adversaries / adversarial systems (e.g., that may implement adversarial attacks) may also be installed in the physical world, posing a real threat to the deployment of artificial intelligence and deep learning, especially in safety-critical applications such as biometrics, cybersecurity, autonomous vehicles, robotics, etc.

[0021] Currently, the deployment of machine learning models in mission-critical environments (such as business environments) is inhibited and trust in AI systems is undermined due to the lack of adversarial robustness of trained machine learning models to adversarial examples. On the one hand, "adversarial samples" can be synonymous with adversarial instances. Adversarial samples / instances can be inputs to machine learning models that are deliberately designed by attackers / adversaries to make machine learning models fail or cause errors. Although "white box scenarios" can defend against adversarial samples, defending against white box attacks remains challenging. Defending against adversarial threats requires a deep understanding of adversarial samples, attacks, and defenses, and only a few experts have the knowledge and experience required to apply such defenses. Therefore, protecting and securing trained machine learning / deep learning models from adversarial samples is critical to ensuring the safe deployment of AI and DL in safety-critical real-world applications, and more broadly, to maintaining trust in AI systems.

[0022] Therefore, various embodiments are provided herein for protecting trained machine learning models (e.g., neural network models and any machine learning model that can be trained with a gradient-based optimization algorithm). In one embodiment, by way of example only, one or more hardened machine learning models are provided to prevent adversarial attacks by adding adversarial protection to one or more trained machine learning models.

[0023] In another aspect, by way of example only, one or more reinforced machine learning models are provided to prevent adversarial attacks by adding adversarial protection to one or more trained machine learning models. The retrained machine learning model may be extended by a preprocessing layer to create an extended machine learning model. If the machine learning model is a neural network, the present invention may add additional layers to the machine learning model and retrain the trained machine learning model to prevent accuracy loss due to the preprocessing layer. The "trained learning model" used herein may indicate that the model "M" has been trained to a desired state. In one aspect, the model "M" (e.g., an extended machine learning model) may be extended by adding one or more additional layers so that the structure of the model M is changed to indicate "M+K". Then, a new model "M+K" needs to be trained again, referred to herein as retraining.

[0024] One or more rollback strategies may be applied to maintain / keep intermediate model states (e.g., model weights) before some or all of them (e.g., maintain / save models M2, M4, and M6 instead of saving models M1, M2, M3, and / or M5 to save maintenance costs), and recover from these model states when a training crash is detected. Training a machine learning model "M" means changing the state of model M through a cycle. At the beginning, M is in state M1, the next iteration is state M2, and it may continue until the last iteration with Mn. It can be expected that with each iteration, the model will become more accurate / better. Therefore, model state Mn may be the most optimized / best state for prediction. Models M1, M2, Mn-1 may be called intermediate models. However, in practice, it may not be possible to know exactly the number of iterations required (i.e., "n"). For example, perhaps model M10 is ideal and optimal, and it may be necessary to stop when "n" equals ten ("10"), but this is unknown, and the system continues to train for M11, etc., which may be worse. Therefore, some intermediate model states need to be maintained / kept in case they are better than the last one.

[0025] In addition, a post-processing output layer can be added to the retrained extended machine learning model, thereby producing a protected machine learning model. One or more performance reports can be provided, illustrating the performance of the protected machine learning model on benign and adversarial samples (e.g., under black-box and / or white-box attacks). The protected machine learning model can then be returned to a user of the system.

[0026] Therefore, the present invention strengthens previously trained machine learning models to cope with adversarial challenges and enables the deployment of secure, trained machine learning models, especially in safety-critical applications (e.g., healthcare, insurance, finance, etc.). On the other hand, the machine learning model can be strengthened against an adversary / adversarial system consisting of computers, CPUs, GPUs, servers, preprocessors, learners, and neural networks. It should be noted that the strengthening of the machine learning model can also be provided as a service in a cloud-based environment.

[0027] On the other hand, by way of example only, adversarial robustness can be added to a trained machine learning model by retraining the trained machine learning model. The present invention can add post-processing functions to the trained machine learning model and / or add an input preprocessing layer to the neural network and adjust the defense strength of the preprocessing layer.

[0028] The present invention can extend a trained neural network with additional neural network layers. The present invention can request user or system input for optional selection steps of the protection process. In addition, the present invention can determine machine learning model performance on benign samples and / or determine machine learning model performance on adversarial samples. The performance of benign and / or adversarial examples can be reported / transmitted back to a user or another computing system. The present invention can track intermediate states of a trained machine learning model during adversarial retraining, and can detect training crashes and enable rollback strategies.

[0029] In addition, it should be noted that one or more calculations may be performed using various mathematical operations or functions that may involve one or more mathematical operations (e.g., performing rate of change / calculus operations, analytically or computationally solving differential or partial differential equations, using addition, subtraction, division, multiplication, standard deviation, mean, average, percentages, statistical modeling using statistical distributions, by finding minimum, maximum or similar thresholds of combined variables, etc.).

[0030] In general, as used herein, "optimization" may refer to and / or be defined as "maximizing," "minimizing," or achieving one or more specific goals, objectives, or intentions. Optimization may also refer to maximizing the benefit of a user (e.g., maximizing the benefit of a trained machine learning model). Optimization may also refer to making the most efficient or functional use of a situation, opportunity, or resource.

[0031] Furthermore, “optimize” need not refer to an optimal solution or result, but rather may refer to a solution or result that is “good enough” for a particular application, for example. In some implementations, the goal is to recommend an “optimal” combination of preprocessing operations (“preprocessors”) and / or machine learning models, but there may be a variety of factors that cause alternative recommendations for preprocessing operations (“preprocessors”) and / or machine learning model combinations to produce better results. Herein, the term “optimize” may refer to such results based on minimum values ​​(or maximum values, depending on the parameters considered in the optimization problem). On the other hand, the terms “optimize” and / or “performing optimization” may refer to operations performed to achieve improved results, such as reduced execution costs or improved resource utilization, regardless of whether the optimal result is actually achieved. Similarly, the term “optimize” may refer to a component used to perform such an improved operation, and the term “optimized” may be used to describe the results of such an improved operation.

[0032] It should be understood that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings given herein is not limited to cloud computing environments. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0033] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0034] Features are as follows:

[0035] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed, without requiring human interaction with the provider of the service.

[0036] Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin-client platforms or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0037] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence, as consumers typically do not have control or knowledge of the exact location of the provided resources, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0038] Rapid elasticity: The ability to quickly and elastically provision capacity, in some cases automatically scaling down quickly and releasing quickly to scale up quickly. To the consumer, the capacity available for provisioning generally appears unlimited and can be purchased in any quantity at any time.

[0039] Measured services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0040] The service model is as follows:

[0041] Software as a Service (SaaS): The capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications can be accessed from different client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0042] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications created or acquired by consumers using programming languages ​​and tools supported by the provider onto the cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure including networks, servers, operating systems or storage, but have control over the deployed applications and possible configuration of the application hosting environment.

[0043] Infrastructure as a Service (IaaS): The capabilities provided to consumers are the provision of processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0044] The deployment model is as follows:

[0045] Private Cloud: Cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0046] Community Cloud: Cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0047] Public cloud: Cloud infrastructure is made available to the public or large industry groups and is owned by an organization that sells cloud services.

[0048] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0049] Cloud computing environments are service-oriented and focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure that includes a network of interconnected nodes.

[0050] See now Figure 1 , describes an illustrative cloud computing environment 50. Cloud computing node 10 is only one example of a suitable cloud computing node and is not intended to impose any limitation on the scope of use or functionality of the embodiments of the present invention described herein. Regardless, cloud computing node 10 is capable of implementing and / or performing any of the functions described above.

[0051] In cloud computing node 10 there is a computer system / server 12, which may operate with many other general or special purpose computing system environments or configurations. Examples of known computing systems, environments, and / or configurations that may be suitable for use with computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, and the like.

[0052] Computer system / server 12 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally speaking, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 12 may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located in local and remote computer system storage media including memory storage devices.

[0053] like Figure 1 As shown, the computer system / server 12 in the cloud computing node 10 is shown in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 to the processor 16.

[0054] Bus 18 represents one or more of several bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0055] Computer system / server 12 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system / server 12, and it includes volatile and nonvolatile media, removable and non-removable media.

[0056] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer system / server 12 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be provided for reading from or writing to a non-removable, non-volatile magnetic medium (not shown, commonly referred to as a "hard drive"). Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In this case, each may be connected to the bus 18 via one or more data media interfaces. As will be further shown and described below, the system memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of an embodiment of the present invention.

[0057] A program / utility 40 having a set (at least one) of program modules 42 may be stored in system memory 28, as may an operating system, one or more application programs, other program modules, and program data, by way of example and not limitation. Each or some combination of the operating system, one or more application programs, other program modules, and program data may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of embodiments of the present invention as described herein.

[0058] The computer system / server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with the computer system / server 12; and / or any device that enables the computer system / server 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the computer system / server 12 may also communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other components of the computer system / server 12 through a bus 18. It should be understood that, although not shown, other hardware and / or software components may be used in conjunction with the computer system / server 12. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems.

[0059] Reference now Figure 2, an illustrative cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N used by cloud consumers can communicate. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks such as, for example, private, community, public, or hybrid clouds or combinations thereof as described above. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as a service without the cloud consumer needing to maintain resources on local computing devices. It should be understood that Figure 2 The types of computing devices 54A-N shown in FIG. 5 are exemplary only, and computing node 10 and cloud computing environment 50 may communicate with any type of computerized device via any type of network and / or network addressable connection (eg, using a web browser).

[0060] See now Figure 3 , showing the cloud computing environment 50 ( Figure 2 ) provides a set of functional abstraction layers. It should be understood in advance that Figure 3 The components, layers and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As shown in the figure, the following layers and corresponding functions are provided:

[0061] The device layer 55 includes physical and / or virtual devices embedded with independent electronic devices, sensors, actuators and other objects to perform various tasks in the cloud computing environment 50. Each device in the device layer 55 can be networked to other functional abstraction layers so that information obtained from the device can be provided to other functional abstraction layers, and / or information can be provided to the device from other abstraction layers. In one embodiment, the various devices comprising the device layer 55 may include a network of entities collectively referred to as the "Internet of Things" (IoT). As will be understood by those of ordinary skill in the art, such a network of entities allows for mutual communication, collection and dissemination of data to achieve a variety of purposes.

[0062] As shown, the device layer 55 includes sensors 52, actuators 53, a "learning" thermostat with integrated processing, sensors and network electronics, cameras 57, controllable household outlets 58, and controllable electrical switches 59, as shown. Other possible devices may include, but are not limited to, various additional sensor devices, network devices, electronic devices (such as remote controls), additional actuator devices, so-called "smart" devices (such as refrigerators or washing machines / dryers), and various other possible interconnected objects.

[0063] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; servers based on RISC (Reduced Instruction Set Computer) architecture 62; servers 63; blade servers 64; storage devices 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0064] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0065] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when resources are utilized within a cloud computing environment, and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-scheduling and procurement of cloud computing resources in anticipation of future demand according to the SLA.

[0066] The workload layer 90 provides examples of functions that can utilize a cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom teaching 93; data analysis processing 94; transaction processing 95; and, in the context of the embodiment illustrated in the present invention, workloads and functions 96 for protecting trained machine learning models against adversaries / adversarial systems in a cloud computing environment. In addition, the workloads and functions 96 for protecting trained machine learning models against adversaries / adversarial systems in a cloud computing environment may include operations such as analysis, deep learning, and user and device management functions as will be further described. A person of ordinary skill in the art will understand that the workloads and functions 96 for protecting trained machine learning models against adversaries / adversarial systems in a cloud computing environment may also work in conjunction with other parts of the various abstraction layers—such as the hardware and software layer 60 virtualization layer 70, the management layer 80, and those parts of the other workload layers 90 (e.g., data analysis processing 94)—to achieve the various purposes of the embodiments illustrated in the present invention.

[0067] As previously described, the present invention provides novel solutions for providing a reinforced machine learning model protected against adversaries / adversarial systems in a computing system. By applying one or more of a plurality of combinations of selected preprocessing operations in one or more machine learning models, a dataset for reinforcing one or more machine learning models, a list of preprocessors, and a selected number of learners, one or more reinforced machine learning models protected against adversarial challenges are provided.

[0068] Now go to Figure 4 , describes a method 400 of protecting a trained machine learning model against an adversary / adversarial system using a processor in which various aspects of the illustrated embodiments may be implemented. Function 400 may be implemented as a method executed on a machine as instructions, where the instructions are included on at least one computer-readable medium or a non-transitory machine-readable storage medium. It should be noted that one or more of the following operations / steps may or may not be applied / selected (e.g., a user may select which step to employ). Function 400 may begin at block 402.

[0069] As shown in block 401, a trained machine learning model may be received and input data may be requested. That is, a user or computing system provides a trained machine learning model and / or may receive a machine learning model. In addition, a request may be received from a user as to what protections to apply, or in an alternative operation, function 400 may automatically select one or more protections to apply.

[0070] As shown in box 402, one or more trained machine learning models can be retrained adversarially. That is, function 400 can retrain the trained neural network using one or more adversarial protocols (e.g., Madry protocols) that have been modified by additional adversarial measures, such as by gradually increasing the proportion of adversarial samples in each mini-batch (e.g., according to a small cosine schedule or "small data", where only a small portion of the data or "mini-batch" of data may be used in each training iteration). This retraining step can change all or only a subset of the model weights. In one aspect, "adversarial retraining" means that the machine learning model can be retrained using adversarial examples and training data. The adversarial examples are not generated by a real adversary, but by the owner of the machine learning model (e.g., the machine learning model is "vaccinated" against future adversarial attacks due to adversarial retraining). Once the machine learning model is trained with specified adversarial examples, then in the future, if the machine learning model is attacked by these adversarial examples, the machine learning model will not make errors or cause errors.

[0071] One or more preprocessing layers may be added and / or calibrated, as shown in block 403. In one aspect, adding preprocessing operations to a trained machine learning model and calibrating a trained machine learning model (e.g., adjusting JPEG compression strength, Gaussian noise variance, etc.) improves the tradeoff between benign and adversarial testing accuracy.

[0072] As shown in block 404, one or more robustification layers may be added. In one aspect, "robustification layers" may mean that one or more layers have been added to the trained machine learning model to cause and / or enable the trained machine learning model to be more robust and to protect against future adversary attacks. In one aspect, one or more neural network layers (e.g., convolutional, dense, etc.) may be added to extend and robustify the retrained machine learning model (e.g., the retrained neural network). It should be noted that, for example, the operations of block 404 may be specific to a neural network model.

[0073] As shown in box 405, the retrained machine learning model can be adversarially retrained (e.g., retrained with adversarial examples and training data). In one aspect, one or more machine learning model parameters of the extended machine learning model can be updated (e.g., following Madry's protocol) to create an adversarially protected machine learning model. In one aspect, "extending the machine learning model" means adding more layers to the current machine learning model, for example, from machine learning model M to M+K. Even if the machine learning model M has been trained, the trained knowledge now does not work well with the added layers, so a new machine learning model "M+K" needs to be retrained.

[0074] The adversarial-protected machine learning model can be returned to a user or system (e.g., deployed into production, as shown in block 406), and information about adversarial robustness performance information can be optionally provided (e.g., test performance on benign and adversarial examples, such as under projected gradient descent (“PGD”) (n,ε) white-box attack, fast gradient sign method (“FGSM”) (ε) transfer attack, and zero-order optimization (“ZOO”) based black-box attack, etc., minimum l∞-norm perturbation required to change classification, etc.).

[0075] Now go to Figure 5 , a block diagram of an exemplary system 500 and functionality for protecting a trained machine learning model against an adversary / adversarial system for images in a computing environment.

[0076] In one aspect, Figure 1-4 One or more of the components, modules, services, applications, and / or functions shown may be used to Figure 5As shown, various functional blocks are depicted with arrows that indicate the relationship between the functional blocks in system 500 and show the process flow. In addition, descriptive information associated with each functional block of system 500 can be seen. As will be seen, many of these functional blocks can also be viewed as "modules" of functionality, with the same descriptive meaning as previously described in Figure 4 The same as described in. Considering the foregoing, the module blocks of system 500 can also be incorporated into various hardware and software components of the system for image enhancement according to the present invention. Many of the functional blocks 500 can be executed as background processes on various components, which can be distributed computing components, on user devices, or elsewhere.

[0077] As shown, system 500 includes a protected neural network 510 (e.g., a reinforced machine learning model system) that may be located at Figure 1 The cloud computing node 10 and / or computing system 12 is internal and / or external.

[0078] The protected neural network 510 may include performing an adversarial retraining operation in a first stage (e.g., “stage 1”) and performing an adversarial retraining operation in a second stage (e.g., “stage 2”).

[0079] The protected neural network 510 may receive one or more images (e.g., “input images”) as input data, as shown in block 507. As shown in block 503, the protected neural network 500 may add an input pre-processing layer to modify the input image (of block 507), such as JPEG compression, Gaussian noise and spatial smoothing, feature compression, etc.

[0080] As shown in block 501, a trained neural network (e.g., a convolutional neural network "CNN", a long-short term memory "LSTM", etc.) may be provided by a user or may be created by an automated system (e.g., via an automated neural network model). As shown in block 502, in a first stage, a protected neural network 510 may be resistant to retraining the trained neural network, for example, by following Madry's protocol, using a projected gradient descent white-box attack with an infinite norm ("I∞ norm" or "uniform norm") adversarial sample constraint and using adversarial samples that are gradually increased according to a cosine schedule.

[0081] In addition, as shown in block 504, one or more additional neural network layers (e.g., dense layers, convolutional layers, etc.) can be added after the trained model. As shown in block 505, a second adversarial retraining operation (e.g., adversarial retraining phase 2) can be performed (similar to block 502). The protected neural network 510 can add an output post-processing layer that produces a non-monotonic, non-differentiable transformation of the output probabilities. As shown in block 508, the protected neural network 500 can provide retrained machine learning model predictions (e.g., neural network predictions (e.g., classification, regression, etc.)).

[0082] Now go to Figure 6 , a block diagram depicting an exemplary system for protecting a trained machine learning model against an adversary / adversarial system on text data in a computing environment. In one aspect, as Figure 1-5 One or more of the components, modules, services, applications and / or functions shown may be used to Figure 6 For the sake of brevity, repeated descriptions of similar steps / frames, elements, components, modules, services, applications, and / or functions used in other embodiments described herein are omitted.

[0083] As shown, system 600 includes a protected neural network 610 (e.g., a reinforced machine learning model system) that may be located at Figure 1 The cloud computing node 10 and / or computing system 12 is internal and / or external.

[0084] Protected neural network 610 may include performing an adversarial retraining operation in a first stage (e.g., “stage 1”) and performing an adversarial retraining operation in a second stage (e.g., “stage 2”).

[0085] As shown in block 607, the protected neural network 610 may receive text data (e.g., "text" such as a short message service) as input data (e.g., "input image"). As shown in block 603, the protected neural network 610 may add input pre-processing layers and text embeddings. For example, in block 607, the protected neural network 610 may add a spell-checking operation as a potential input pre-processing defense for adversarial robustness / protection, such as lower-casing, stemming, lemmatization, stop word removal, normalization, noise removal, text enrichment / enhancement, etc. to defeat character-level attacks. In addition, text embeddings may be added to convert text data into numerical data, such as by using pre-trained word embeddings (e.g., global vectors "GloVe" and / or Word2Vec for word representation, etc.).

[0086] As shown in box 601, a trained neural network (e.g., a convolutional neural network "CNN", a long-short-term memory "LSTM", etc.) can be provided by a user or can be created by an automated system (e.g., via an automated neural network model).

[0087] As shown in block 602, in a first stage, the protected neural network 610 may adversarially retrain the trained neural network. For example, the protected neural network 610 may adversarially retrain the trained neural network by following Miyato's protocol, using the fast gradient method with adversarial training samples with I2-norm ε, and adding an additional hidden layer before the output, which may be modified by gradually increasing the proportion of adversarial samples in each mini-batch (e.g., according to fractional cosine scheduling, etc.) or other measures.

[0088] In addition, as shown in block 604, one or more additional neural network layers (e.g., dense layers, convolutional layers, etc.) can be added after the trained model. As shown in block 605, a second adversarial retraining operation (e.g., adversarial retraining phase 2) can be performed (similar to block 602). The protected neural network 610 can add an output post-processing layer that produces a non-monotonic, non-differentiable transformation of the output probabilities. As shown in block 608, the protected neural network 610 can provide retrained machine learning model predictions (e.g., neural network predictions (e.g., classification, regression, etc.)).

[0089] Now go to Figure 7 , a block diagram of an exemplary system for protecting a trained machine learning model against a text data adversary / adversarial system in a computing environment. In one aspect, as Figure 1-6 One or more of the components, modules, services, applications, and / or functions shown may be used to Figure 7 For the sake of brevity, repeated descriptions of similar steps / frames, elements, components, modules, services, applications, and / or functions used in other embodiments described herein are omitted.

[0090] As shown, system 700 includes a protected neural network 710 (e.g., a reinforced machine learning model system) that may be located at Figure 1 The cloud computing node 10 and / or computing system 12 is internal and / or external.

[0091] As shown in block 705, protected neural network 710 may receive input data. As shown in block 703, protected neural network 710 may add an input preprocessing layer to modify the input data. For example, at block 707, protected neural network 710 may modify the input data using compression, Gaussian noise, smoothing, and / or feature compression, etc.

[0092] As shown in block 701 , a trained neural network (eg, a support vector machine “SVM,” etc.) may be provided by a user or may be created by an automated system.

[0093] As shown in block 702, the protected neural network 710 may adversarially retrain (e.g., "adversarially retrain") the trained neural network. For example, the protected neural network 710 may adversarially retrain the trained neural network using a fast gradient method with adversarial training samples having an I2-norm ε, and add an additional hidden layer before the output, which may be modified by gradually increasing the proportion of adversarial samples in each mini-batch (e.g., according to a fractional cosine schedule, etc.), among other measures.

[0094] Furthermore, as shown in block 704, one or more additional neural network layers may be added after the trained model. That is, the protected neural network 710 may add an output post-processing layer that produces a non-monotonic, non-differentiable transformation of the output probabilities. As shown in block 706, the protected neural network 710 may provide a retrained machine learning model prediction (e.g., a neural network prediction (e.g., classification, regression, etc.)).

[0095] Now go to Figure 8 , describes a method 800 for strengthening learning input preprocessing of a machine learning model against adversarial attacks using a processor in which various aspects of the illustrated embodiments may be implemented. Function 800 may be implemented as a method executed on a machine in the form of instructions, where the instructions are included on at least one computer-readable medium or a non-transitory machine-readable storage medium. Function 800 may begin at block 802.

[0096] As shown in block 804, one or more trained machine learning models may be received. As shown in block 806, the one or more trained machine learning models may be retrained to include adversarial protection based on one or more adversarial protection protocols. As shown in block 808, one or more hardened machine learning models against adversarial attacks are provided by adding adversarial protection to the one or more trained machine learning models. As shown in block 810, functionality 800 may end.

[0097] In one aspect, in combination with and / or as Figure 8 The operations of 800 may include each of the following: The operations of 800 may receive one or more trained machine learning models and retrain the one or more trained machine learning models to include adversarial protection based on one or more adversarial protection protocols.

[0098] The operations of 800 may add one or more pre-processing layers to one or more trained machine learning models, wherein the degree of adversarial protection strength is adjusted; add one or more neural network layers to one or more trained machine learning models, and / or add one or more post-processing output layers to one or more trained machine learning models that are retrained with adversarial protection based on one or more adversarial protection protocols.

[0099] The operations of 800 may automatically implement one or more adversarial protection protocols for providing one or more enhanced machine learning models, and / or receive one or more adversarial protection protocols for providing one or more enhanced machine learning models from a user.

[0100] The operations of 800 may monitor and track each state of one or more trained machine learning models during retraining, detect training crashes of one or more trained machine learning models during retraining, and / or enable one or more rollback strategies for one or more trained machine learning models during retraining. The operations of 800 may determine a security score of one or more reinforced machine learning models indicating a security level against adversarial attacks.

[0101] The present invention may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute various aspects of the present invention.

[0102] Computer readable storage medium can be a tangible device that can retain and store instructions for use by instruction execution devices. Computer readable storage medium can be, for example but not limited to, electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage medium includes the following: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device such as punch card or protruding structure in the groove with instructions recorded thereon, and any suitable combination of the above. Computer readable storage medium as used herein should not be interpreted as a temporary signal itself, such as radio waves or other free propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (e.g., light pulses passing through fiber optic cables) or electrical signals emitted by wires.

[0103] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device through a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or downloaded to an external computer or external storage device. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network, and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0104] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, etc.) and conventional process programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (including local area network (LAN) or wide area network (WAN)), or can be connected to an external computer (for example, using an Internet service provider through the Internet). In certain embodiments, the electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can be personalized to perform computer-readable program instructions by utilizing the state information of the computer-readable program instructions to perform the electronic circuit, so as to perform various aspects of the present invention.

[0105] The present invention is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which causes the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium in which the instructions are stored includes a manufactured product containing instructions for implementing aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in or in multiple boxes in the flowchart and / or block diagram.

[0108] The flow charts and block diagrams in the accompanying drawings show the architecture, functions and operations of the possible implementations of the system, method and computer program product according to various embodiments of the present invention. To this end, each box in the flow chart or block diagram may represent a module, segment or part of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the box may not occur in the order marked in the figure. For example, depending on the functions involved, the two boxes shown in succession can actually be executed substantially simultaneously, or these boxes can sometimes be executed in reverse order. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

Claims

1. A method for protecting a trained machine learning model in a computing environment by one or more processors, the method comprising: Providing one or more fortified machine learning models to resist adversarial attacks by adding adversarial protection to one or more previously trained machine learning models, wherein the one or more fortified machine learning models are generated by iteratively retraining, and the proportion of adversarial samples is iteratively increased during each retraining iteration; Monitor and track each state of one or more previously trained machine learning models while they are being retrained; Detecting training crashes of one or more previously trained machine learning models during retraining; as well as Enable one or more rollback strategies for one or more previously trained machine learning models during retraining.

2. The method according to claim 1, further comprising: receiving one or more previously trained machine learning models; and Retraining of one or more previously trained machine learning models is performed to include adversarial protection based on one or more adversarial protection protocols.

3. The method according to claim 1 also includes adding one or more pre-processing layers to one or more previously trained machine learning models, wherein the degree of adversarial protection strength is adjusted in each of the one or more pre-processing layers.

4. The method of claim 1 , further comprising adding one or more neural network layers to one or more previously trained machine learning models.

5. The method of claim 1 further comprises adding one or more post-processing output layers to one or more previously trained machine learning models, wherein the one or more post-processing output layers have been retrained with adversarial protection based on one or more adversarial protection protocols.

6. The method according to claim 1, further comprising: automatically implementing one or more adversarial protection protocols to be used to provide one or more hardened machine learning models; or One or more adversarial protection protocols are received from a user to be used to provide one or more reinforced machine learning models.

7. A system for protecting a trained machine learning model in a computing environment, comprising: One or more computers having executable instructions that, when executed, cause the system to: Providing one or more fortified machine learning models to resist adversarial attacks by adding adversarial protection to one or more previously trained machine learning models, wherein the one or more fortified machine learning models are generated by iteratively retraining, and the proportion of adversarial samples is iteratively increased during each retraining iteration; Monitor and track each state of one or more previously trained machine learning models while they are being retrained; Detecting training crashes of one or more previously trained machine learning models during retraining; as well as Enable one or more rollback strategies for one or more previously trained machine learning models during retraining.

8. The system of claim 7, wherein the executable instructions: receiving one or more previously trained machine learning models; and Retraining of one or more previously trained machine learning models is performed to include adversarial protection based on one or more adversarial protection protocols.

9. The system of claim 7, wherein the executable instructions add one or more pre-processing layers to one or more previously trained machine learning models, wherein a degree of adversarial protection strength is adjusted in each of the one or more pre-processing layers.

10. The system of claim 7, wherein the executable instructions add one or more neural network layers to one or more previously trained machine learning models.

11. The system of claim 7, wherein the executable instructions add one or more post-processing output layers to one or more previously trained machine learning models, the one or more post-processing output layers having been retrained with adversarial protection based on one or more adversarial protection protocols.

12. The system of claim 7, wherein the executable instructions: automatically implementing one or more adversarial protection protocols to be used to provide one or more hardened machine learning models; or One or more adversarial protection protocols are received from a user to be used to provide one or more reinforced machine learning models.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images