Provable editing of deep neural networks using parametric linear relaxation

Abstract

Systems and methods for editing neural networks, and in particular deep neural networks (DNNs) to satisfy a given property. Different from existing approaches, it supports quantified linear formulas on the input and output as the specification, which scales to infinite sets of data points in high-dimensional input and output spaces (like text and images). Implementations of these systems and methods may use efficient polynomial-time linear programming solvers instead of gradient descent to optimize the editing, and do not require modification of the architecture of the underlying DNN.

Description

PROVABLE EDITING OF DEEP NEURAL NETWORKS USINGPARAMETRIC LINEAR RELAXATIONSTATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0001] This invention was made with Government support under grant no. 2048123 awarded by the National Science Foundation (NSF); and grant no. DE-SC0022285 awarded by the Department of Energy (DoE). The Government has certain rights in this invention.CROSS-REFERNCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 684,040, filed August 16, 2024, the disclosure of which is herein incorporated by reference in its entirety for all purposes.TECHNICAL FIELD

[0003] The present embodiments generally relate to systems and methods for data processing. In particular, the present embodiments relate to systems and methods for editing of neural network algorithms.BACKGROUND

[0004] Computational models such as neural networks may be utilized to recognize patterns and solve complex problems. Such models may comprise interconnected nodes, or neurons, organized in layers such as an input layer, one or more hidden layers, and an output layer.Each neuron may process input data and pass the result to one or more neurons in the next layer through weighted connections. Adjusting these weights based on the error of the output compared to the desired result may allow for evolutionary learning and improvement of the model, using algorithms such as backpropagation.

[0005] Deep neural networks (DNNs) are increasingly used in safety-critical applications such as self-driving cars, health care, etc. However, these models may be prone to producing errors in which they output apparently reasonable data that is nonetheless incorrect. Some attempts to guarantee accuracy of a DNN include DNN verifiers; however, even when successful at identifying an inaccurate DNN, they provide no guidance for how to edit or correct the DNN beyond simply retraining it from the beginning. This may be an unacceptable waste of time, processor resources, memory, power, etc. SUMMARY

[0006] In some aspects, the present disclosure is directed to implementations of systems and methods for editing neural networks, and in particular deep neural networks (DNNs) to satisfy a given property. Different from existing approaches, it supports quantified linear formulas on the input and output as the specification, which scales to infinite sets of data points in high-dimensional input and output spaces (like text and images). Implementations of these systems and methods may use efficient polynomial-time linear programming solvers instead of gradient descent to optimize the editing, and do not require modification of the architecture of the underlying DNN.

[0007] The systems and methods discussed herein provide implementations of an efficient technique for provable editing of DNNs. Specifically, given a DNN N with parameters θ,implementations of the systems and methods discussed herein are able to provably edit the DNN to satisfy this property within a reasonable runtime (e.g. less than an hour, less than 10 minutes, less than a minute, etc., depending on implementation and resources). The systems and methods discussed herein are efficient relative to other systems, because they relax the NP-hard provable editing problem to solving a linear program; specifically, by using Parametric Linear Relaxation, the system may construct tight bounds of the DNN that are parameterized by the new parameters ^^ . The systems and methods discussed herein are demonstrably more efficient and effective compared to prior DNN editing approaches. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0009] Various objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the detailed description taken in conjunction with the accompanying drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements.

[0010] FIG.1 is a diagram of an example of a deep neural network (DNN), according to some implementations;

[0011] FIG.2A is an illustration of graphs of example ReLU activation functions with linear relaxations, according to some implementations;

[0012] FIG.2B is an illustration of graphs of example affine functions with linear relaxations with variable parameters, according to some implementations;

[0013] FIG.3A shows illustrations of example DNNs and their input-output behavior, according to some implementations;

[0014] FIG.3B shows illustrations of decoupled DNNs and their input-output behavior, according to some implementations;

[0015] FIG.3C shows illustrations of edited decoupled DNNs and their input-output behavior, according to some implementations;

[0016] FIGs. 4 and 5 are graphs of cactus plots of runtime for various editing problems with different implementations of parameter editors, along with graphs of relative speed differences;

[0017] FIGs. 6A and 6B are a generalized and specific example implementation of a method for editing a deep neural network; and

[0018] FIGs. 7A and 7B are block diagrams depicting embodiments of computing devices useful in connection with the methods and systems described herein.

[0019] The details of various embodiments of the methods and systems are set forth in the accompanying drawings and the description below. DETAILED DESCRIPTION

[0020] Computational models such as neural networks and deep neural networks (DNNs) may be utilized to recognize patterns and solve complex problems. Such models may comprise interconnected nodes, or neurons, organized in layers such as an input layer, one or more hidden layers, and an output layer. Each neuron may process input data and pass the result to one or more neurons in the next layer through weighted connections. Adjusting these weights based on the error of the output compared to the desired result may allow for evolutionary learning and improvement of the model, using algorithms such as backpropagation.

[0021] A neural network may be considered a series of algorithms that seek to identify relationships for a given set of inputs. Various types of neural networks exist. For example, modular neural networks include a network of neural networks, each network may function independently to accomplish a sub-task that is part of tasks in a larger set. Breaking down tasks in the manner decreases the complexity of analyzing a large set of data. Similarly, gated neural networks are neural networks that incorporate memory such that the network is able to remember, and classify more accurately, long datasets.

[0022] FIG.1 illustrates a model of an example deep neural network, according to some implementations. A deep neural network may comprise a network of neurons 108, each of which may represent an activation function performed on an input to generate an output. Various activation functions may be utilized including linear activation functions (e.g. the output is 2 times the input), non-linear activation functions (e.g. the output is 1 if the input is greater than 0.5, but 0 if the input is equal to or less than 0.5), or a mix of linear and non-linear activation functions. Each input function may be associated with a weight or coefficient that is applied to an input value prior to processing via theactivation function. Each neuron 108 may exist in a theoretical layer such as an input layer 102 (to which initial input values are provided), an output layer 106 (from which an output value or vector is read), and one or more hidden layers 104. Each neuron 108 may be connected to one or more other neurons via connections 110. Weighting coefficients and, in some cases, parameters of activation functions may be determined by the system to conform an output to an expected or desired value via supervised or unsupervised learning.

[0023] Ensuring that a DNN is correct and avoids harmful behaviors is critical when deploying them, especially in safety-critical contexts (e.g. self-driving cars, factory automation, critical equipment command and control systems, etc.). In such scenarios, it is vital to guarantee that a DNN satisfies a given property, such as a safety specification, a required accuracy, a required threshold of false positives or false negatives, etc. For example, for safety-critical applications, these constraints may specify safety properties; e.g.; global safety properties for ACAS Xu networks. For scientific applications, these constraints may specify the physical invariants such as boundary conditions and conservation laws that the DNN approximating the physical system needs to globally satisfy. For image recognition and NLP applications, these constraints can represent local robustness properties.

[0024] Towards this end, DNN verifiers aim to determine whether a DNN satisfies the given property. There are efficient methods for DNN verification that can handle a wide-range of DNN architectures and properties, as seen in the annual DNN verification competition (Verification of Neural Networks Competition or VNN-COMP). However, the success in DNN verification reveals the real challenge: how to fix a buggy DNN. If a DNN failsverification or is identified as failing to meet the required property or properties, typically developers must restart and retrain the network, potentially from scratch, as it is impossible for a human to repair bugs in a DNN even given the violated constraints and counterexamples. This may require significant time, resources, and effort, and may waste prior resources spent training the system incorrectly.

[0025] Put another way, given a DNN N and specification φandinputdataX, the systems and methods described herein provide an edited DNN N’. The editing algorithm should have the following properties: (P1) efficacy: N’ should correctly classify all inputs in X; (P2) generalization: N’ should correctly classify inputs similar to those in X; (P3) locality: N’ should behave the same as N on inputs that are dissimilar to those in X; (P4) efficiency: the editing algorithm should be efficient.

[0026] Property P1 is vital in safety-critical applications such as autonomous vehicles and to ensure that DNN watermarks are not removed in a Machine Learning as a Service (MLaaS) scenario.

[0027] Retraining using the original training data augmented with X is one workaround available to current practitioners. However, this retrained network is not guaranteed to correctly classify all inputs in X, violating property P1. Retraining could produce a DNN that is very different from the original, violating property P3. Retraining is also computationally expensive, violating property P4. Moreover, the original training data might not be available, which is likely if one is using pre-trained DNNs in an MLaaS scenario.

[0028] Another workaround would be to maintain a lookup table for the inputs X, effectively overriding the DNN and ensuring that the correct result is returned. This approachsatisfies properties P1, P3, and P4. However, it does not satisfy property P2, because it is not robust to small changes to the input.

[0029] Instead, implementations of the system and methods discussed herein provide for editingedited model is guaranteed to satisfy the required constraint. The guarantee is provided for all inputs in an input polytope defined by a set of linear constraints. This may include polytopes in high-dimensional input spaces. Additionally, the editing problem is formulated as solving a linear program, and hence can be solved in polynomial time. Our empirical result demonstrates that gradient-descent-based approaches is unpredictable in terms of time and timeout in many experiments.

[0030]

[0031] training to guarantee that the DNN satisfies a property, or post-deployment to repair a given DNN.

[0032] The provable editing problem is challenging due to the presence of the universal quantifier; viz., finding new parameters such that for all inputs in the polytope P, the output of the edited network lies in the polytope Q. There previously have been no approaches for efficiently solving the provable editing problem. Regularization-based approaches encode the property into the loss during training, but are unable to guarantee the property-satisfaction. SMT-based approaches provide guarantees, but are not efficient because they directly solve an NP-hard problem. Prior Linear Programming (LP)-based approaches are efficient because they can only handle a restricted class of provable editing problems; e.g., they assume that the vertices of the input polytope or the linear regions of the DNN can be efficiently enumerated, or the DNN architecture can be modified.

[0033] Instead, implementations of the systems and methods discussed herein provide the first efficient technique for provable editing of DNNs that runs in polynomial time. Given a DNN and a property it violates (e.g. from the VNN-COMP'22 benchmarks), the system is able to provably edit the DNN to satisfy this property within a short time frame (e.g.1 minute, 45 seconds, 30 seconds, or less, etc.).

[0034] These systems and methods enable provable editing by constructing tight parametric bounds of the output for all inputs x in the input polytope P with respect to theparametersand then constraining these parametric bounds to lie within the output polytope Q. These parametric bounds are constructed compositionally per layer of theDNN, and are expressed as linear constraints, so that efficient LP solvers can be used to find an optimal solution.

[0035] As discussed above, neurons in a neural network may have an activation function, such as a rectified linear unit (ReLU) function in which the output is enabled and rises in linear proportion to the input after a threshold input level is reached. An example of this is shown in the black line in the left graph (a) of FIG.2A. The parametric bounds may be represented indirectly via underapproximations of the epigraph and hypograph of the DNN layer. Prior approaches overapproximate the ReLU’s upper bound with a linear function (e.g. red line in graph (a) of FIG.2A) using constant input boundsSimilarly, the ReLU’s lower bound may be represented with a similar linear function (e.g. blue line in graph (a) of FIG.2A). Such a linear relaxation (shown shaded) is loose, and is only sound within the given constant input bounds

[0036] Instead, the systems and methods discussed herein are able to exactly represent the upper

[0037] Implementations of this Parametric Linear Relaxation may be applied to DNN layers, with defines tight parametric bounds using linear constraints in terms of the layerapproaches one could achieve a sound, but loose linear relaxation, as shown in graph (a)of FIG. 2B. The Figure shows linear relaxations for boundingwith variable parameter 6^ . The red and blue lines denote the upper and lower bound over-approximations respectively, and the yellow shading denotes the relaxation. Graph (a) illustrates an imprecise relaxation fornot treated as a parameter but another input within constant bounds

[0038] In contrast, the systems and methods discussed herein exactly represents the output bounds without any relaxation (as shown in graphs (b) and (c) of FIG. 2B) building upon our key insight of capturing the epigraph and hypograph of the DNN layer in terms of the parameters. The bounds are parameterized by variableGraph (c) illustrates how our parametric bounds are defined for the function with respect to the variableparameterThe gray shaded portion denotes the true output region for all slope (input) The upper bound 5^. (shown in red line) is exactly represented by the epigraph(red dotted region) and is defined by a linear constraint and thelower bound^ is exactly represented by the hypograph (blue dotted region), and defined by a linear constraint

[0039] FIG.3A shows illustrations of example DNNs and their input-output behavior, according to some implementations. In graph (a), a ReLU DNN N1 is shown, with one input x, one output y, and three hidden units h1, h2, and h3 using a ReLU activation function. The input-output behavior of this network for the domain x ^ [−1, 2] is shown in graph (b). N1 is piecewise linear (PWL) (i.e. its input domain can be partitioned into finitely-many convex polytopes X1, X2, . .. , Xn such that, for each Xi, there exists some affine function fi s The linear regions are shown visually as coloredintervals on the x axis, which each map to the output according to some affine mapping which is specific to that region. In particular, we have three linear regions: LinRegions(N1, [−1, 2]) = {[−1, 0], [0, 1], [1, 2]}.

[0040] Each linear region corresponds to a particular activation pattern on the hidden neurons; i.e., which ones are in the zero region or the identity region. The first linear region, [−1, 0] (red), corresponds to the activation pattern where only h1 is activated. The second linear region, [0, 1] (blue), corresponds to the activation pattern where only h2 is activated. Finally, the third linear region, [1, 2] (green), corresponds to the activation pattern where both h2 and h3 are activated.

[0041]

[0042] A goal of manipulating DNNs is to enforce precise specifications. The first type of specification considered is a point editing specification. In this scenario, we are given a finite set of input points along with, for each such point, a subset of the output region which we would like that point to be mapped into by the network.

[0043]

[0044] These constraints are equivalent to the editing specification above.

[0045] The general affine constraint form used above is very expressive. For example, it can express constraints such as “the ith output component is larger than all others.” For a multi-label classification network, such a constraint is equivalent to ensuring that the point is classified with label i.

[0046] We could directly encode the DNN in an SMT solver and attempt to solve for weight assignments that cause the desired classification. However, this quickly becomes computationally infeasible. To understand this, consider what happens when a single weight in N1 is modified to construct the new DNN N2, shown in graph (c) of FIG. 2A. In particular, the weight on x ^ h3 is changed from a 1 to a 2. Comparing graph (d) with graph (b), we see that changing this weight has caused two distinct changes: the linear function associated with the green region has changed, and the linear regions themselves (shown on the x axis) have changed, with the green region growing to include parts of the space originally in the blue region. In particular, OPQRSTPUQV8@N: W)?:DX3 ^G ^^W)?: 1X: W1: 1ABX: W1AB: DX^, different from the three linear regions described above in connection with N1. The term Coupling refers to the fact that the weights in a ReLU DNN simultaneously control both of these aspects. This coupling is the primary reason why editing DNNs is computationally infeasible, because the impact of changing a weight in the network with respect to the output of the network on a fixed input is non-linear; it ‘jumps’ every time the linear region that the point falls into changes.

[0047] In contrast to a coupled DNN, a decoupled DNN (DDNN) allows us to decouple the two roles that the parameters play.^FIG. 3B shows illustrations of decoupled DNNs and their input-output behavior, according to some implementations. At (a) is illustrated a de decoupled DNN N3 equivalent to N1 from FIG.3A, illustration (a). Most notably, every decoupled DNN consists of two or more ‘sub-networks,’ or channels (which may bereferred to variously as sub-networks, regions, channels, sub-graphs, coupling regions, hypergraphs, or by similar terms). The activation channel, or the first sub-network shown in red at top left, is used to determine the positions of the linear regions. Meanwhile, the value channel or second sub-network shown at lower right determines the actual output map within each linear region. The activation channel influences the value channel via the blue edges, which indicate that the adjacent value node is activated if and only if the corresponding activation node is. For example, if the input to is negative, then ^willoutput zero regardless of the input to

[0048] To compute the output of a DDNN on a given input evaluate theactivation channel, and record which of the hidden nodes were active (received apositive input) or inactive (otherwise). Then, we set xv= x0and evaluate the value channel, except instead of activating a node if its input is non-negative, we activate the node if the corresponding activation channel node was activated. In this way, activation nodes can ‘mask’ their corresponding value nodes, as notated with the blue edges ingraph (a) of FIG.3B.

[0049] Now, consider what happens when we change a weight in only the value channel, as shown in graph (b) of FIG.3B. In that scenario, on any given point, the activation pattern for any given input does not change, and so the location of the linear regions on the xaxis of graph (d) are unchanged from graph (c). However, what we find is that the linear function within any given region does change. Note that in this case only the green line has changed, however in deeper networks changing a single weight may change all of the lines.

[0050] Editing Decoupled DNNs: for any given input, the output of a decoupled DNN depends linearly with the change of any layer of weights in the value channel. Using this fact, we can reduce point editing of a single layer in the DDNN to a linear programming (LP) problem.

[0051] Suppose we want to edit the first value layer of DDNN N3 to satisfy the following point editing specification:

[0052] 1_M1 _N? b a_ b ^ a])? `

[0053] We can solve for ^, and can optimize a linear objective, such as thenorm, to find the satisfying edit with the provably smallest ^. Finding the smallest ^ ensures locality of the edit and preserves the otherwise-correct existing behavior of the network. In this case, we find that the smallest edit with respect to theThe corresponding edited DDNN N5 is shown in graph (a) of FIG. 3Cand plotted in graph (c) of FIG.3C, where we can see that the edited network satisfies the constraints because N5(0.5) = −0.8 and N5(1.5) = −0.2. Notably, the linear regions of N5and N1 are the same.

[0054] Editing Non-ReLU, Non-Fully-Connected Layers: implementations of the above methods also hold for a generalization of DDNNs with arbitrary activation functions, such as tanh and sigmoid. Such generalized DDNNs are defined using the notion of Jacobians, a more general notion of linear approximations of vector functions. Jacobians can be computed automatically using standard automatic differentiation packages. Similarly, although we have formalized DNNs assuming fully-connected layers, our approach can edit convolutional and other similar types of layers as well. Thus, the point editing approach can be applied to arbitrary feed-forward networks.

[0055] In some instances, it may be possible that there is no layer-wise edit satisfying the given specification. Accordingly, in some implementations, the above approach may be extended to allow for efficient editing of multiple layers. For example, by modifying explicit bias terms for multiple layers, Provable Editing for DDNNs may remain an LP. Provable Editing can also be used to correct bias and ensure fairness in DNNs. For instance, gender and accent biases are present in voice assistants, gender biases exist in machine translation, and decisions made by DNNs have been shown to be unfair, discriminatory, or offensive.

[0056]

[0057] For piecewise-linear DDNNs, if we only change the value channel parameters of a DDNN, then we can reduce polytope editing to point editing. To see this, recall that the value channel parameters do not change the location of the linear regions, only the behavior within each one. Within each linear region, the behavior of the network is linear hence convex. Convexity guarantees that any given polytope is mapped into another polytope if and only if its vertices are mapped into that polytope.

[0058] Using the example above, consider that a line lies in the desired interval of [−0.8, −0.4] if and only if its endpoints do. In fact, the input region of interest in our example of [0.5, 1.5] overlaps with two of these lines. Hence, we must ensure that both of those lines have endpoints in [−0.8, −0.4]. Thus, the polytope editing specification is met if and only if the point editing specification is met, with

[0059] We call the points in K key points because the behavior of the edited network N^ on these points determines the behavior of the network on all of P1. (Duplication of points K2and K3 is a nuance needed to handle boundaries.) Therefore, we have reduced the problem ofediting on polytopes to editing on finitely-many key points, which are the vertices of the polytopes in the specification intersected with the polytopes defining the linear regions of the DNN N1(i.e., LinRegions(N1, P1)). Applying the algorithm for point editing discussed above to solve for a minimal edit to the first layer, we get the linear constraints:

[0060] Existing algorithms for LinRegions scale to low-dimensional polytopes for large DNNs, which is inadequate for many applications. To account for this drawback, in some implementations multiple low-dimensional slices may be used to extend the Provable Polytope Editing to higher-dimensional specifications, allowing for (local) robustness specifications. Consequently, one could provably edit DNNs to guarantee robustness.

[0061] The foregoing describes how to edit weights in the original DNN without decoupling to satisfy universally-quantified constraints with high-dimensional input polytopes, resulting in architecture-preserving provable editing of DNNs for higher-dimensional specifications.

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] As discussed above, this editing may be performed in real time or near real-time. FIG.4 shows at left a cactus plot of runtime for 423 single-property editing problems, with a 600 second time limit for each problem. As shown, the implementation of the systems and methods discussed herein, referred to as PREPARED, is significantly faster and more efficient. The implementation succeeded on all 423 problems, while the competing algorithms PFT[DL2] and Architecture-Preserving Provable Repair of Deep Neural Networks (APRNN) succeeded on only 185 and 117, respectively. At middle is shown the speedup of PREPARED relative to PFT[DL2], and at right is shown the speedup of PREPARED relative to PFT[APRNN].

[0080] Similarly, FIG. 5 shows at left a cactus plot of runtime for 66 all-properties editing problems, with a 3600 second time limit. PREPARED succeeded on 62 problems, while PFT[DL2] and PFT[APRNN] succeeded on 45 and 48 respectively. At middle and right are shown the speedup of PREPARED relative to PFT[DL2] and PFT[APRNN], respectively.

[0081] One implementation of the systems and methods described herein was programmed using PyTorch, an open source machine learning framework, and Gurobi, a mathematical optimization solver for linear programming (LP), quadratic programming (QP) and mixed integer programming (MIP) problems, ERAN, an abstraction-interpretation-based DNN analyzer that implements state-of-the-art bound computation approaches, as well as auto LiRPA, an automatic-linear-relaxation-based perturbation analysis library for DNNs that implements state-of-the-art bound computation approaches. The implementation supports a wide variety of DNN architectures, including fully-connected, convolutionaland residual DNNs with max and average pooling layers, as well as ReLU, Tanh and other activation layers.

[0082] FIGs. 6A and 6B are a generalized and specific example implementation of a method for editing a deep neural network. Referring first to FIG. 6A, at step 602, a system may encode variable bounds of a DNN output as a linear formula. At step 604, the system may freeze k layer variable parameters to a set of given parameters. At step 606, the system may encode variable bounds for the kthlayer as well as for any subsequent layers. At step 608, the system may encode variable bounds of the output. At step 610, the system may freeze variable weights %^ to constant weights W for the edited system.

[0083] An implementation of this is shown in more detail in FIG. 6B. The system selects a first layer of the DNN and determines a layer type. In some embodiments, the layer type may be a ReLU layer. In other embodiments, the layer type may be an affine layer. In still other embodiments not illustrated, the layer type may be another type of layer (e.g. softmax layer, sigmoid layer, convolution layer, etc.).

[0084] For a ReLU layer, the system determines a parametric linear relaxation as shown and discussed above. For an affine layer, the system determines whether the input bounds are constant or variable, and then determines a parametric linear relaxation as shown and discussed above. The system may repeat this process for each additional layer of the network.

[0085] Having discussed specific embodiments of the present solution, it may be helpful to describe aspects of the operating environment as well as associated system components (e.g., hardware elements) in connection with the methods and systems described herein.

[0086] The systems discussed herein may be deployed as and / or executed on any type and form of computing device, such as a computer, network device or appliance capable of communicating on any type and form of network and performing the operations described herein. FIGs.7A and 7B depict block diagrams of a computing device 700 useful for practicing an embodiment of the wireless communication devices 702 or the access point 706. As shown in FIGs.7A and 7B, each computing device 700 includes a central processing unit 721, and a main memory unit 722. As shown in FIG.7A, a computing device 700 may include a storage device 728, an installation device 716, a network interface 718, an I / O controller 723, display devices 724a-724n, a keyboard 726 and a pointing device 727, such as a mouse. The storage device 728 may include, without limitation, an operating system and / or software. As shown in FIG. 7B, each computing device 700 may also include additional optional elements, such as a memory port 703, a bridge 770, one or more input / output devices 730a-730n (generally referred to using reference numeral 730), and a cache memory 740 in communication with the central processing unit 721.

[0087] The central processing unit 721 is any logic circuitry that responds to and processes instructions fetched from the main memory unit 722. In many embodiments, the central processing unit 721 is provided by a microprocessor unit, such as: those manufactured by Intel Corporation of Mountain View, California; those manufactured by International Business Machines of White Plains, New York; or those manufactured by Advanced Micro Devices of Sunnyvale, California. The computing device 700 may be based on any of these processors, or any other processor capable of operating as described herein.Main memory unit 722 may be one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the microprocessor 721, such as any type or variant of Static random access memory (SRAM), Dynamic random access memory (DRAM), Ferroelectric RAM (FRAM), NAND Flash, NOR Flash and Solid State Drives (SSD). The main memory 722 may be based on any of the above described memory chips, or any other available memory chips capable of operating as described herein. In the embodiment shown in FIG.7A, the processor 721 communicates with main memory 722 via a system bus 750 (described in more detail below). FIG.7B depicts an embodiment of a computing device 700 in which the processor communicates directly with main memory 722 via a memory port 703. For example, in FIG. 7B the main memory 722 may be DRDRAM.

[0088] FIG.7B depicts an embodiment in which the main processor 721 communicates directly with cache memory 740 via a secondary bus, sometimes referred to as a backside bus. In other embodiments, the main processor 721 communicates with cache memory 740 using the system bus 750. Cache memory 740 typically has a faster response time than main memory 722 and is provided by, for example, SRAM, BSRAM, or EDRAM. In the embodiment shown in FIG. 7B, the processor 721 communicates with various I / O devices 730 via a local system bus 750. Various buses may be used to connect the central processing unit 721 to any of the I / O devices 730, for example, a VESA VL bus, an ISA bus, an EISA bus, a MicroChannel Architecture (MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or a NuBus. For embodiments in which the I / O device is a video display 724, the processor 721 may use an Advanced Graphics Port (AGP) to communicate with the display 724. FIG. 7B depicts an embodiment of a computer 700 inwhich the main processor 721 may communicate directly with I / O device 730b, for example via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technology. FIG. 7B also depicts an embodiment in which local busses and direct communication are mixed: the processor 721 communicates with I / O device 730a using a local interconnect bus while communicating with I / O device 730b directly.

[0089] A wide variety of I / O devices 730a-730n may be present in the computing device 700. Input devices include keyboards, mice, trackpads, trackballs, microphones, dials, touch pads, touch screen, and drawing tablets. Output devices include video displays, speakers, inkjet printers, laser printers, projectors and dye-sublimation printers. The I / O devices may be controlled by an I / O controller 723 as shown in FIG. 7A. The I / O controller may control one or more I / O devices such as a keyboard 726 and a pointing device 727, e.g., a mouse or optical pen. Furthermore, an I / O device may also provide storage and / or an installation medium 716 for the computing device 700. In still other embodiments, the computing device 700 may provide USB connections (not shown) to receive handheld USB storage devices such as the USB Flash Drive line of devices manufactured by Twintech Industry, Inc. of Los Alamitos, California.

[0090] Referring again to FIG. 7A, the computing device 700 may support any suitable installation device 716, such as a disk drive, a CD-ROM drive, a CD-R / RW drive, a DVD-ROM drive, a flash memory drive, tape drives of various formats, USB device, hard-drive, a network interface, or any other device suitable for installing software and programs. The computing device 700 may further include a storage device, such as one or more hard disk drives or redundant arrays of independent disks, for storing an operating system and other related software, and for storing application softwareprograms such as any program or software 720 for implementing (e.g., configured and / or designed for) the systems and methods described herein. Optionally, any of the installation devices 716 could also be used as the storage device. Additionally, the operating system and the software can be run from a bootable medium.

[0091] Furthermore, the computing device 700 may include a network interface 718 to interface to the network 704 through a variety of connections including, but not limited to, standard telephone lines, LAN or WAN links (e.g., 802.11, T1, T3, 56kb, X.25, SNA, DECNET), broadband connections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET), wireless connections, or some combination of any or all of the above. Connections can be established using a variety of communication protocols (e.g., TCP / IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad, CDMA, GSM, WiMax and direct asynchronous connections). In one embodiment, the computing device 700 communicates with other computing devices 700’ via any type and / or form of gateway ortunneling protocol such as Secure Socket Layer (SSL) or Transport Layer Security (TLS). The network interface 718 may include a built-in network adapter, network interface card, PCMCIA network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the computing device 700 to any type of network capable of communication and performing the operations described herein.

[0092] In some embodiments, the computing device 700 may include or be connected to one or more display devices 724a-724n. As such, any of the I / O devices 730a-730n and / or theI / O controller 723 may include any type and / or form of suitable hardware, software, or combination of hardware and software to support, enable or provide for the connection and use of the display device(s) 724a-724n by the computing device 700. For example, the computing device 700 may include any type and / or form of video adapter, video card, driver, and / or library to interface, communicate, connect or otherwise use the display device(s) 724a-724n. In one embodiment, a video adapter may include multiple connectors to interface to the display device(s) 724a-724n. In other embodiments, the computing device 700 may include multiple video adapters, with each video adapter connected to the display device(s) 724a-724n. In some embodiments, any portion of the operating system of the computing device 700 may be configured for using multiple displays 724a-724n. One ordinarily skilled in the art will recognize and appreciate the various ways and embodiments that a computing device 700 may be configured to have one or more display devices 724a-724n.

[0093] In further embodiments, an I / O device 730 may be a bridge between the system bus 750 and an external communication bus, such as a USB bus, an Apple Desktop Bus, an RS- 232 serial connection, a SCSI bus, a FireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a Gigabit Ethernet bus, an Asynchronous Transfer Mode bus, a FibreChannel bus, a Serial Attached small computer system interface bus, a USB connection, or a HDMI bus.

[0094] A computing device 700 of the sort depicted in FIGs. 7A and 7B may operate under the control of an operating system, which control scheduling of tasks and access to system resources. The computing device 700 can be running any operating system such as any of the versions of the MICROSOFT WINDOWS operating systems, the different releasesof the Unix and Linux operating systems, any version of the MAC OS for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating systems for mobile computing devices, or any other operating system capable of running on the computing device and performing the operations described herein. Typical operating systems include, but are not limited to: Android, produced by Google Inc.; WINDOWS 7 and 8, produced by Microsoft Corporation of Redmond, Washington; MAC OS, produced by Apple Computer of Cupertino, California; WebOS, produced by Research In Motion (RIM); OS / 2, produced by International Business Machines of Armonk, New York; and Linux, a freely-available operating system distributed by Caldera Corp. of Salt Lake City, Utah, or any type and / or form of a Unix operating system, among others.

[0095] The computer system 700 can be any workstation, telephone, desktop computer, laptop or notebook computer, server, handheld computer, mobile telephone or other portable telecommunications device, media playing device, a gaming system, mobile computing device, or any other type and / or form of computing, telecommunications or media device that is capable of communication. The computer system 700 has sufficient processor power and memory capacity to perform the operations described herein.

[0096] In some embodiments, the computing device 700 may have different processors, operating systems, and input devices consistent with the device. For example, in one embodiment, the computing device 700 is a smart phone, mobile device, tablet or personal digital assistant. In still other embodiments, the computing device 700 is an Android-based mobile device, an iPhone smart phone manufactured by Apple Computer of Cupertino, California, or a Blackberry or WebOS-based handheld device or smartphone, such as the devices manufactured by Research In Motion Limited. Moreover, the computing device 700 can be any workstation, desktop computer, laptop or notebook computer, server, handheld computer, mobile telephone, any other computer, or other form of computing or telecommunications device that is capable of communication and that has sufficient processor power and memory capacity to perform the operations described herein.

[0097] Although the disclosure may reference one or more “users”, such “users” may refer to user-associated devices or stations (STAs), for example, consistent with the terms “user” and “multi-user” typically used in the context of a multi-user multiple-input and multiple- output (MU-MIMO) environment.

[0098] Although examples of communications systems described above may include devices and APs operating according to an 802.11 standard, it should be understood that embodiments of the systems and methods described can operate according to other standards and use wireless communications devices other than devices configured as devices and APs. For example, multiple-unit communication interfaces associated with cellular networks, satellite communications, vehicle communication networks, and other non-802.11 wireless networks can utilize the systems and methods described herein to achieve improved overall capacity and / or link quality without departing from the scope of the systems and methods described herein.

[0099] It should be noted that certain passages of this disclosure may reference terms such as “first” and “second” in connection with devices, mode of operation, transmit chains, antennas, etc., for purposes of identifying or differentiating one from another or from others. These terms are not intended to merely relate entities (e.g., a first device and asecond device) temporally or according to a sequence, although in some cases, these entities may include such a relationship. Nor do these terms limit the number of possible entities (e.g., devices) that may operate within a system or environment.

[0100] It should be understood that the systems described above may provide multiple ones of any or each of those components and these components may be provided on either a standalone machine or, in some embodiments, on multiple machines in a distributed system. In addition, the systems and methods described above may be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture. The article of manufacture may be a floppy disk, a hard disk, a CD-ROM, a flash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general, the computer-readable programs may be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. The software programs or executable instructions may be stored on or in one or more articles of manufacture as object code.

[0101] Various methods are described herein, and each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as “first”, “second”, etc. may be used in various embodiments to modify an element, component, step, operation, etc., for example, a “first receiving” and a “second receiving”. Use of such terms does not imply an ordering to the modified operations unless specifically required. So, in this example, the first receiving need not be performed before the secondreceiving, and may occur, for example, before, during, or in an overlapping time period with the second receiving.

[0102] The implementations and aspects described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed may also be implemented in other forms (for example, an apparatus or program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, for example, computers, cell phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.

[0103] Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment. Additionally, this application may refer to “determining” various pieces of information. Determining the information may include one or more of, for example, estimating the information,calculating the information, predicting the information, or retrieving the information from memory.

[0104] Further, this application may refer to “accessing” various pieces of information. Accessing the information may include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0105] Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information may include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0106] It is to be appreciated that the use of any of the following “ / ”, “and / or”, and “at least one of’ , for example, in the cases of “A / B”, “A and / or B” and “at least one of A and B”, isintended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option(C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.

[0107] Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments the encoder signals a quantization matrix for de-quantization. In this way, in an embodiment the same parameter is used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways.

[0108] For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.

[0109] As will be evident to one of ordinary skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing amethod, or data produced by one of the described implementations. For example, a signal may be formatted to carry the bitstream of a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

Claims

What is Claimed:

1. A method for editing a neural network, comprising: receiving, by a computing device, a neural network model having neural network weights and a specification for the model; determining, by the computing device, one or more edits to the neural network weights to satisfy the received specification; and modifying, by the computing device, the neural network model weights based on the edited weights.

2. The method of claim 1, wherein the specification comprises a required accuracy.

3. The method of any one of claims 1-2, wherein the specification comprises a constraint or boundary condition.

4. The method of claim 3 wherein the constraint or boundary condition are universally quantified linear constraints, which can be represented by polytopes in a half-space representation.

5. The method of any one of claims 1-4, wherein the specification comprises a minimum local robustness.

6. The method of any one of claims 1-5, wherein determining the one or more edits to the neural network weights further comprises identifying a type of layer of the neural network model; and determining a parametric linear relaxation of the layer based on the identified type of layer.

7. The method of claim 6, further comprising identifying a type of input bounds of the layer of the neural network model; and wherein determining the parametric linear relaxation is further based on the identified type of input bounds.

8. The method of claim 7, wherein the type of layer is an affine layer and the type of input bounds is constant.

9. The method of claim 8, wherein the parametric linear relaxation ^ : ^ / ^K^is constrained by theformula10. The method of claim 7, wherein the type of layer is an affine layer and the type of input bounds is variable.

11. The method of claim 10, wherein:

12. The method of any one of claims 1-11, wherein the neural network includes a neuron having a rectified linear unit (ReLU) activation function, and the specification exactly represents an upper bound of an output of the ReLU activation function for any input upper bound variable and uses a linear relaxation for a lower bound of the output of the ReLU activation function, wherein the linear relaxation for the lower bound of the output of the ReLU activation function is determined13. The method of any one of claims 1-12, wherein: the neural network is a high-dimensional neural network, and determining the one or more edits to the neural networks weights comprises: slicing, by the computing device, the neural network into two or more low- dimensional slices; and determining, by the computing device, one or more edits to neural network weights of the two or more low-dimensional slices to satisfy the received specification.

14. A computing device comprising one or more processors configured to perform the method of any of claims 1-13.

15. A non-tangible computer readable medium comprising instructions that, when executed by one or more processors of a computing device, cause the computing device to perform the method of any of claims 1-13.

Images