System and method of monotone operator neural networks

Abstract

The present disclosure relates to computer systems with artificial intelligence capabilities, including neural networks. A system for training neural work includes an input interface for accessing input data for a neural network and a processor in communication with the input interface. The processor is programmed to receive the input at the neural network and output a trained neural network with forward propagation and back propagation, where the forward propagation includes identifying a fixed point associated with one or more parameters of the neural network with a root finding process, where the back propagation includes identifying a derivative of a loss associated with the parameters of the network.

Description

Technical Field

[0001] This disclosure relates to computer systems with artificial intelligence capabilities, including neural networks. Background Technology

[0002] The general goal of a DEQ (Deep Equilibrium) model can be to define some forward functions. f This allows us to first solve for the equilibrium point. The forward pass of the network is given. The backward pass can largely correspond to multiplication by the Jacobian. Its exact form requires a solution to a linear system. There are three main challenges in developing DEQ models. One of them is defining... f This makes the fixed point It exists, and ideally, it is unique. The second approach could be to define a root-finding process that finds a fixed point during forward propagation. The third approach could be to define backpropagation to solve for the Jacobian. product. Summary of the Invention

[0003] According to one embodiment, a computer-implemented method for predicting classification and training neural operations includes receiving input at a neural network, classifying the input to the neural network using forward propagation, wherein forward propagation includes identifying fixed points associated with one or more parameters of the neural network using a root-finding process, computing a vector by using a linear transformation of the fixed points, outputting a classification of the input in response to the maximum element of the vector, and training the neural network using backpropagation, wherein backpropagation includes identifying the derivative of the loss associated with the parameters of the network.

[0004] According to a second embodiment, a computer-implemented method for classifying inputs to a neural network includes receiving inputs at the neural network, identifying the classification of the inputs at the neural network using forward propagation, wherein the forward propagation includes identifying fixed points associated with the parameters of the neural network using a root-finding process, computing a vector by using a linear transformation of the fixed points, and outputting the classification of the inputs in response to the maximum element of the vector.

[0005] According to a third embodiment, a system for training neural operations includes an input interface for accessing input data of a neural network and a processor communicating with the input interface. The processor is programmed to receive input at the neural network and output a trained neural network using forward and backward propagation, wherein forward propagation includes identifying fixed points associated with one or more parameters of the neural network using a root-finding process, and wherein backward propagation includes identifying the derivatives of the loss associated with the network's parameters. Attached Figure Description

[0006] Figure 1 A system 100 for training neural networks is shown.

[0007] Figure 2 A computer-implemented method 200 for training a neural network is shown.

[0008] Figure 3 A flowchart 300 depicts the classification process of a monotonic network.

[0009] Figure 4 A flowchart 400 depicts the training process of a monotonic network.

[0010] Figure 5 A schematic diagram depicting the interaction between the computer-controlled machine 10 and the control system 12 is shown.

[0011] Figure 6 Depicting Figure 1 A schematic diagram of a control system configured to control a vehicle, which may be a partially autonomous vehicle or a partially autonomous robot.

[0012] Figure 7 Depicting Figure 1 A schematic diagram of a control system configured to control manufacturing machines, such as stamping tools, cutting tools, or gun drills, in a manufacturing system (such as a part of a production line).

[0013] Figure 8 Depicting Figure 1 A schematic diagram of a control system configured to control power tools, such as drills or drives, having at least a partially autonomous mode.

[0014] Figure 9 Depicting a person configured to control an automated personal assistant Figure 1 A schematic diagram of the control system.

[0015] Figure 10 Depicting Figure 1 A schematic diagram of a control system configured as a control and monitoring system, such as a control access system or a monitoring system.

[0016] Figure 11 Depicting Figure 1 A schematic diagram of a control system configured to control an imaging system, such as an MRI device, an X-ray imaging device, or an ultrasound device. Detailed Implementation

[0017] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various forms and alternative forms. The figures are not necessarily to scale; some features may be enlarged or minimized to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those skilled in the art to employ the embodiments in different ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any of the figures can be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of illustrated features provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of this disclosure may be desired for a particular application or implementation.

[0018] In previous DEQ models, defining a function with a fixed point could be heuristically accomplished using a network structure that empirically appears to produce a stable fixed point. This could include using existing transformer-like or convolutional architectures, attempting to provide a stable system with appropriate layer normalization. However, there is no formal guarantee that such layers have a fixed point. For the root-finding process, DEQ can use the nonlinear Broyden method, which also does not guarantee finding a root (even if a root or fixed point exists). Furthermore, the backpropagation can be solved linearly using a similar Broyden method, which guarantees multiplication by the appropriate inverse if one exists, but the Jacobian will not correspond to a meaningful derivative if no fixed point exists or if the forward propagation fails to converge.

[0019] Monotonic operator networks can be a specific form of the DEQ model that offers much better solutions to the three challenges identified above. Specifically, to define functions with fixed points, monotonic operator networks use functions... f The system has a specific parameterized form that guarantees the existence and uniqueness of fixed points. By applying tools from monotone operator theory, the system can derive two distinct iterative processes to find the equilibrium point. These processes are always guaranteed to converge to the equilibrium point. A similar set of algorithms can be devised to perform backpropagation.

[0020] Figure 1 A system 100 for training a neural network is shown. System 100 may include an input interface for accessing training data 192 of the neural network. For example, as... Figure 1As illustrated in the diagram, the input interface can be composed of a data storage interface 180, which can access training data 192 from a data storage device 190. For example, the data storage interface 180 can be a memory interface or a permanent storage interface 30, such as a hard disk or SSD interface, or it can be a personal area network (PAN), local area network (LAN), or wide area network (WAN) interface, such as a Bluetooth, Wi-Fi, or Ethernet or fiber optic interface. The data storage device 190 can be an internal data storage device of the system 100, such as a hard disk drive or SSD, but it can also be an external data storage device, such as a network-accessible data storage device.

[0021] In some embodiments, data storage device 190 may further include a data representation 194 of an untrained version of the neural network, which can be accessed by system 100 from data storage device 190. However, it should be understood that the training data 192 and data representation 194 of the untrained neural network may also each be accessed from a different data storage device, for example, via a different subsystem of data storage device interface 180. Each subsystem may be of the type of data storage interface 180 as described above. In other embodiments, the data representation 194 of the untrained neural network may be generated internally by system 100 based on the design parameters of the neural network, and therefore may not be explicitly stored on data storage device 190. System 100 may further include a processor subsystem 160, which may be configured to provide an iterative function as an alternative to the layer stack of the neural network to be trained during operation of system 100. Here, the corresponding layers of the replaced layer stack may have mutually shared weights and may receive the output of the previous layer as input, or, for the first layer of the layer stack, receive the initial activation and a portion of the input of the layer stack as input. Processor subsystem 160 may be further configured to iteratively train the neural network using training data 192. Here, the training iterations performed by processor subsystem 160 may include a forward propagation portion and a backward propagation portion. Processor subsystem 160 may be configured to perform the forward propagation portion by, among other operations, defining an executable forward propagation, determining an equilibrium point of the iterative function at which the iterative function converges to a fixed point, wherein determining the equilibrium point includes finding the root solution of the iterative function minus its input using a numerical root-finding algorithm, and by providing the equilibrium point as an alternative to the output of the layer stack in the neural network. System 100 may further include an output interface for outputting a data representation 196 of the trained neural network, which may also be referred to as trained model data 196. For example, as... Figure 1As illustrated in the diagram, the output interface can be comprised of a data storage interface 180, which in these embodiments is an input / output (“IO”) interface through which trained model data 196 can be stored in the data storage device 190. For example, a data representation 194 defining an “untrained” neural network can be at least partially replaced by a data representation 196 of a trained neural network during or after training, because the parameters of the neural network—such as weights, hyperparameters, and other types of parameters of the neural network—can be adapted to reflect training on the training data 192. This also... Figure 1 The figures are illustrated by reference numerals 194 and 196, which refer to the same data records on data storage device 190. In other embodiments, data representation 196 may be stored separately from data representation 194 defining the "untrained" neural network. In some embodiments, the output interface may be separate from data storage interface 180, but it is generally of the type described above for data storage interface 180.

[0022] Figure 2 A computer-implemented method 200 for training a neural network is shown. Method 200 can correspond to... Figure 1 The operation of system 100 is not required, because it can also correspond to the operation of another type of system, device or equipment, or because it can correspond to a computer program.

[0023] Method 200 is shown to include, in a step entitled "Providing a Data Representation of a Neural Network", providing a neural network 210, wherein providing the neural network includes providing an iterative function as an alternative to a layer stack of the neural network, wherein the corresponding layers of the replaced layer stack have mutually shared weights and receive the output of the previous layer as input, or, for the first layer of the layer stack, receiving an initial activation and a portion of the input of the layer stack as input. Method 200 is further shown to include accessing training data of the neural network 220 in a step entitled "Accessing Training Data". Method 200 is further shown to include iteratively training the neural network 230 using the training data in a step entitled "Iteratively Training the Neural Network Using Training Data", the training 230 including a forward propagation portion and a backward propagation portion. Performing the forward propagation portion via method 200 may include, in a step entitled "Determining an Equilibrium Point Using a Root-Solving Algorithm," determining an equilibrium point of 240 for an iterative function at which the iterative function converges to a fixed point, wherein determining the equilibrium point includes using a numerical root-solving algorithm to find the root solution of the iterative function minus its input, and in a step entitled "Providing an Equilibrium Point as an Alternative to the Output of the Layer Stack," providing an equilibrium point as an alternative to the output of the layer stack in the neural network. Method 200 may further include, after training, outputting a trained neural network 260 in a step entitled "Outputting the Trained Neural Network." The following example describes a neural network, including its training, wherein the layer stack is replaced by an iterative function, and wherein an equilibrium point is determined using a root-solving algorithm at which the iterative function converges to a fixed point, wherein determining the equilibrium point includes using a numerical root-solving algorithm to find the root solution of the iterative function minus its input, and in a step entitled "Providing an Equilibrium Point as an Alternative to the Output of the Layer Stack," providing an equilibrium point 250 as an alternative to the output of the layer stack in the neural network. Method 200 may further include, after training, and in a step entitled "Outputting the Trained Neural Network", outputting 260 of the trained neural network. This neural network may be further described in a patent application entitled "Deep Neural Network with a Balancing Solver," which is incorporated herein by reference in its entirety.

[0024] Figure 3 A flowchart 300 depicts the classification process for a monotonic network. In step 301, the system can receive input from an input interface. The input interface can be wireless communication (e.g., Wi-Fi, 4G, 5G, LTE, etc.) or wired communication (USB, serial port, CAN-BUS, etc.). The input interface can allow the network to communicate with various sensors such as cameras, radar, LiDAR, etc. The input can be any input data, including but not limited to image data. The input can, for example, include pixel values ​​of an image.

[0025] In step 303, the network can compute fixed points (e.g., equilibrium points) via forward propagation. In monotone operator networks, the network can utilize specific forms of equilibrium functions, particularly parameterization: in It can be some nonlinear function, and in certain embodiments it can be a neighbor operator of a convex function (in fact, this can mean...). It is monotonically increasing and has a Lipschitz constant less than or equal to one; for example, it is non-expanding. W and U It can be a linear operator such as matrix multiplication or multi-layer convolution, and b It is a constant. A key aspect of monotone operator networks lies in... W It has a specific form, namely in A and B It is also a linear operator such as matrix multiplication or convolution, and It is a positive constant, and is selected as the default value. In this example, the parameters defining the network are: This parameterization guarantees the existence of conditions that are satisfied. The only fixed point (e.g., the equilibrium point).

[0026] Using this parameterization, the system can calculate the equilibrium point using two different methods. The network can use a root-finding process to determine fixed points. One approach is the forward-backward (FB) splitting method. Another approach is the Peaceman-Rachford (PR) splitting method. These methods are implemented for monotone operator networks as follows: Algorithm: FB-forward solver (e.g., FB splitting method) initialization Repeat until convergence: return z Algorithm: PR-forward solver (e.g., Peacean-Rachford splitting method) initialization Repeat until convergence: return z For these two algorithms or methods This can be a free parameter similar to the step size in gradient-based optimization methods. Given a specific network, the free parameter can be tuned to improve convergence speed. For the PR-forward solver algorithm, the system can typically set... However, for the FB-forward algorithm, there may not be a single value that performs best in general, and this parameter can be tuned for a specific network.

[0027] PR-forward solving may require the system to be able to multiply by the inverse. In definition W of A and B When the operator is a dense matrix, this can be achieved by directly inverting the matrix. However, in A and B In the case of convolution, the system can use the Fast Fourier Transform to compute the necessary inverse.

[0028] In step 305, the neural network can compute a prediction vector. The network can perform certain mathematical operations to obtain the prediction vector. These operations can be performed in the hidden layers of the neural network (e.g., between the input and output layers). The prediction vector can be computed by taking a balance point and applying another linear operation. The prediction vector can be computed as follows: The prediction vector can be a function of the weights of the neural network.

[0029] In step 307, the system can output the predicted classification. The input predicted classification can be... The index of the maximum value of y. In the example, when the third entry of y is the largest, the system can predict that the classification is the class. The classification step can classify the input data. In one example, the input data can be an image, and based on the classification, the neural network can classify the image as an image of a vehicle or a pedestrian. The classification of the input data can also be used to train the neural network, as explained further below.

[0030] Figure 4 A flowchart 400 depicts the training process of a monotonic network. In step 401, the network can receive input including the corresponding category of the input data. The input can be defined as: enter : In the formula above, The input can be defined, and You can define actual classes, and i It can be an index of different examples in the training set.

[0031] In step 403, the network can determine whether the prediction needs to be repeated. The system can stop repeating when the parameter values ​​converge, or when the loss stops decreasing. If either scenario occurs, the system may stop repeating each step of the training process.

[0032] In step 405, the network can select a subset of the complete training data, called a minibatch B.

[0033] In step 407, the network can perform forward passes to compute. and The network can run forward computation to output the classification of each index in the mini-batch. Therefore, Figure 3 The steps can be performed for each index in a mini-batch.

[0034] In step 409, the system can calculate the loss. The loss can be calculated using cross-entropy loss. The loss can be calculated using the following equation: Cross-entropy loss measures the performance of a classification model whose output is a probability value between two values ​​(e.g., between 0 and 1). Cross-entropy loss can increase as the predicted probabilities deviate from the actual labels. For example, a perfect model would have zero loss.

[0035] In step 411, the system can calculate the derivative of the loss with respect to the network parameters. Backpropagation calculations may require the system to calculate the derivative of the loss with respect to the network parameters. That is, the system may need to calculate: For example, the system may need to left multiply. (Existing automatic differentiation tools can easily calculate the remainder term) Within the framework of monotonic operators, there exists a method similar to forward propagation to perform this multiplication. Specifically, given... The following two algorithms can produce the necessary product. The implementation of these methods for monotonic operator networks can be as follows: Algorithm: FB-forward solver (e.g., FB splitting method) initialization make ,in Marking The derivative of (where we define if) ,but ) repeat: For all i , (where if) Then the value is assigned to ) return (where multiplied by) (Executed by the standard automatic differentiation toolkit).

[0036] Another algorithm: PR-backward solving (e.g., the Peacean-Rachford splitting method). initialization make ,in Marking The derivative of (where we define if) ,but ) repeat: For all i , (where if) Then the value is assigned to ) return (where multiplied by) (Executed by the standard automatic differentiation toolkit).

[0037] As mentioned above, It can be a hyperparameter, which can be set to the PR variable. Furthermore, these algorithms need to be tuned to a specific architecture to minimize the number of iterations required for the FB variables. It is guaranteed that these algorithms converge to a unique solution in the backpropagation. Again, for one of them... W In the case of structured convolution operators, the Fast Fourier Transform can be used to perform the necessary inversion.

[0038] In step 413, the system can use their gradients to adjust their parameters. The gradient can be defined as: In step 415, the system can output the final network parameters. The final network parameters may include... And so on. If the network is fully connected, the parameters can be various weights of a matrix, or they can be the weights of a convolutional network.

[0039] Figure 5A schematic diagram depicting the interaction between a computer-controlled machine 10 and a control system 12 is shown. The computer-controlled machine 10 includes actuators 14 and sensors 16. Actuators 14 may include one or more actuators, and sensors 16 may include one or more sensors. Sensors 16 are configured to sense the condition of the computer-controlled machine 10. Sensors 16 may be configured to encode the sensed condition into a sensor signal 18 and transmit the sensor signal 18 to the control system 12. Non-limiting examples of sensors 16 include video, radar, LiDAR, ultrasonic, and motion sensors. In one embodiment, sensor 16 is an optical sensor configured to sense an optical image of the environment near the computer-controlled machine 10.

[0040] The control system 12 is configured to receive sensor signals 18 from the computer-controlled machine 10. As described below, the control system 12 may be further configured to calculate actuator control commands 20 based on the sensor signals and transmit the actuator control commands 20 to the actuators 14 of the computer-controlled machine 10.

[0041] like Figure 5 As shown, the control system 12 includes a receiving unit 22. The receiving unit 22 can be configured to receive sensor signals 18 from sensor 30 and transform the sensor signals 18 into input signals x. In an alternative embodiment, the sensor signals 18 are received directly as input signals x without the receiving unit 22. Each input signal x may be a portion of each sensor signal 18. The receiving unit 22 can be configured to process each sensor signal 18 to generate each input signal x. The input signals x may include data corresponding to the image recorded by sensor 16.

[0042] The control system 12 includes a classifier 24. The classifier 24 can be configured to use a machine learning (ML) algorithm, such as the neural network described above, to classify the input signal x into one or more labels. The classifier 24 is configured to be composed of parameters such as those described above (e.g., parameters). The parameters are parameterized. The parameters can be stored in or provided by non-volatile storage device 26. Classifier 24 is configured to determine an output signal y from an input signal x. Each output signal y includes information assigning one or more tags to each input signal x. Classifier 24 can transmit the output signal y to conversion unit 28. Conversion unit 28 is configured to convert the output signal y into an actuator control command 20. Control system 12 is configured to transmit the actuator control command 20 to actuator 14, actuator 14 being configured to actuate computer-controlled machine 10 in response to the actuator control command 20. In another embodiment, actuator 14 is configured to actuate computer-controlled machine 10 directly based on the output signal y.

[0043] When actuator 14 receives actuator control command 20, actuator 14 is configured to perform an action corresponding to the relevant actuator control command 20. Actuator 14 may include control logic configured to transform actuator control command 20 into a second actuator control command for controlling actuator 14. In one or more embodiments, instead of an actuator or in addition to an actuator, actuator control command 20 may also be used to control a display.

[0044] In another embodiment, instead of the computer-controlled machine 10 including sensor 16, or in addition to the computer-controlled machine 10 including sensor 16, the control system 12 also includes sensor 16. Instead of the computer-controlled machine 10 including sensor 16, or in addition to the computer-controlled machine 10 including actuator 10, the control system 12 may also include actuator 14.

[0045] like Figure 5 As shown, the control system 12 also includes a processor 30 and a memory 32. The processor 30 may include one or more processors. The memory 32 may include one or more memory devices. A classifier 24 (e.g., an ML algorithm) of one or more embodiments may be implemented by the control system 12, which includes a non-volatile storage device 26, a processor 30, and a memory 32.

[0046] Non-volatile storage device 26 may include one or more persistent data storage devices, such as hard disk drives, optical drives, tape drives, non-volatile solid-state devices, cloud storage, or any other device capable of persistently storing information. Processor 30 may include one or more devices selected from a high-performance computing (HPC) system, including high-performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field-programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other device that manipulates signals (analog or digital) based on computer-executable instructions residing in memory 32. Memory 32 may include a single memory device or multiple memory devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0047] Processor 30 may be configured to read into memory 32 and execute computer-executable instructions residing in non-volatile storage device 26 and embodying one or more ML algorithms and / or methods of one or more embodiments. Non-volatile storage device 26 may include one or more operating systems and applications. Non-volatile storage device 26 may store data compiled and / or interpreted from computer programs created using various programming languages ​​and / or techniques, which, without limitation, individually or in combination include Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL.

[0048] When executed by processor 30, the computer-executable instructions of non-volatile storage device 26 can cause control system 12 to implement one or more of the ML algorithms and / or methods disclosed herein. Non-volatile storage device 26 may also include ML data (including data parameters) that support the functionality, features, and processing of one or more embodiments described herein.

[0049] Program code embodying the algorithms and / or methods described herein can be distributed individually or collectively as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform one or more embodiments. Essentially non-transitory computer-readable storage media can include tangible media that are volatile and non-volatile, as well as removable and non-removable, implemented in any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can further include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, portable optical disc read-only memory (CD-ROM) or other optical storage devices, magnetic tape, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be read by a computer. Computer-readable program instructions can be downloaded from the computer-readable storage medium to a computer, another type of programmable data processing apparatus, or another device, or downloaded via a network to an external computer or external storage device.

[0050] Computer-readable program instructions stored in a computer-readable medium can be used to direct a computer, other type of programmable data processing apparatus, or other device to operate in a particular manner, causing the instructions stored in the computer-readable medium to produce an article of writing that includes instructions to implement the functions, actions, and / or operations specified in the flowchart or diagram. In some alternative embodiments, the functions, actions, and / or operations specified in the flowchart and diagram may be reordered, processed sequentially, and / or processed simultaneously in accordance with one or more embodiments. Furthermore, any of the flowchart and / or diagram may include more or fewer nodes or blocks than those illustrated in accordance with one or more embodiments.

[0051] Processes, methods, or algorithms can be implemented, either wholly or in part, using appropriate hardware components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or combinations of hardware, software, and firmware components.

[0052] Figure 6 A schematic diagram is depicted of a control system 12 configured to control a vehicle 50, which may be at least partially autonomous or at least partially autonomous. For example... Figure 5 As shown, vehicle 50 includes actuator 14 and sensor 16. Sensor 16 may include one or more video sensors, radar sensors, ultrasonic sensors, LiDAR sensors, and / or positioning sensors (e.g., GPS). One or more of these specific sensors may be integrated into vehicle 50. Alternatively, or in addition to the one or more specific sensors identified above, sensor 16 may include a software module configured to determine the state of actuator 14 upon execution. A non-limiting example of the software module includes a weather information software module configured to determine the current or future weather conditions near vehicle 50 or other locations.

[0053] The classifier 24 of the control system 12 of vehicle 50 can be configured to detect objects near vehicle 50 depending on the input signal x. In such an embodiment, the output signal y may include information characterizing objects near vehicle 50. An actuator control command 20 can be determined based on this information. The actuator control command 20 can be used to avoid collisions with the detected objects.

[0054] In embodiments where vehicle 50 is at least partially autonomous, actuator 14 may be embodied in the vehicle 50's brakes, propulsion system, engine, transmission system, or steering system. Actuator control command 20 can be determined to control actuator 14 so that vehicle 50 avoids collisions with detected objects. The detected objects can also be classified according to what classifier 24 deems them most likely to be, such as pedestrians or trees. Actuator control command 20 may be determined based on the classification.

[0055] In other embodiments where vehicle 50 is at least partially autonomous, vehicle 50 may be a mobile robot configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be at least partially autonomous lawnmower or at least partially autonomous cleaning robot. In such embodiments, actuator control commands 20 may be determined such that the propulsion unit, steering unit, and / or braking unit of the mobile robot can be controlled, enabling the mobile robot to avoid collisions with identified objects.

[0056] In another embodiment, vehicle 50 is at least partially autonomous in the form of a gardening robot. In such an embodiment, vehicle 50 may use an optical sensor, as sensor 16, to determine the state of plants in the environment near vehicle 50. Actuator 14 may be a nozzle configured to spray chemicals. Depending on the identified plant species and / or the identified plant state, actuator control command 20 may be determined to cause actuator 14 to spray the plants with an appropriate amount of appropriate chemicals.

[0057] Vehicle 50 may be a partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include washing machines, stoves, ovens, microwave ovens, or dishwashers. In such a vehicle 50, sensor 16 may be an optical sensor configured to detect the state of an object to be processed by the household appliance. For example, in the case of a washing machine, sensor 16 may detect the state of the clothes inside the washing machine. Actuator control command 20 may be determined based on the detected state of the clothes.

[0058] Figure 7 A schematic diagram of a control system 12 is depicted, which is configured to control a manufacturing machine 100, such as a stamping tool, cutting tool, or gun drill, of a manufacturing system 102 (such as part of a production line). The control system 12 may be configured to control an actuator 14, which is configured to control the manufacturing machine 100.

[0059] The sensor 16 of the manufacturing machine 100 may be an optical sensor configured to capture one or more properties of the manufactured product 104. The classifier 24 may be configured to determine the state of the manufactured product 104 based on one or more captured properties. The actuator 14 may be configured to control the manufacturing machine 100, depending on the determined state of the manufactured product 104, for subsequent manufacturing steps of the manufactured product 104. The actuator 14 may also be configured to control the function of the manufacturing machine 100 on the subsequent manufactured product 106, depending on the determined state of the manufactured product 104.

[0060] Figure 8 A schematic diagram of a control system 12 is depicted, which is configured to control a power tool 150, such as an electric drill or drive, having at least a partially autonomous mode. The control system 12 may be configured to control an actuator 14, which is configured to control the power tool 150.

[0061] The sensor 16 of the power tool 150 may be an optical sensor configured to capture one or more properties of the working surface 152 and / or the fastener 154 driven into the working surface 152. The classifier 24 may be configured to determine the state of the working surface 152 and / or the fastener 154 relative to the working surface 152 based on one or more of the captured properties. This state may be that the fastener 154 is flush with the working surface 152. Alternatively, this state may be the hardness of the working surface 154. The actuator 14 may be configured to control the power tool 150 such that the driving function of the power tool 150 is adjusted depending on the determined state of the fastener 154 relative to the working surface 152 or one or more captured properties of the working surface 154. For example, if the state of the fastener 154 is flush with the working surface 152, the actuator 14 may interrupt the driving function. As another non-limiting example, the actuator 14 may apply additional or lesser torque depending on the hardness of the working surface 152.

[0062] Figure 9 A schematic diagram is depicted of a control system 12 configured to control an automated personal assistant 200. The control system 12 can be configured to control an actuator 14, which in turn is configured to control the automated personal assistant 200. The automated personal assistant 200 can be configured to control household appliances such as a washing machine, stove, oven, microwave oven, or dishwasher.

[0063] Sensor 16 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive video images of the gestures 204 of the user 202. The audio sensor may be configured to receive voice commands from the user 202.

[0064] The control system 12 of the automated personal assistant 200 can be configured to determine an actuator control command 20 configured as the control system 12. The control system 12 can be configured to determine the actuator control command 20 based on a sensor signal 18 from a sensor 16. The automated personal assistant 200 is configured to transmit the sensor signal 18 to the control system 12. The classifier 24 of the control system 12 can be configured to execute a gesture recognition algorithm to identify a gesture 204 made by the user 202, determine the actuator control command 20, and transmit the actuator control command 20 to the actuator 14. The classifier 24 can be configured to retrieve information from non-volatile storage in response to the gesture 204 and output the retrieved information in a form suitable for reception by the user 202.

[0065] Figure 10 A schematic diagram of a control system 12 configured to control a monitoring system 250 is depicted. The monitoring system 250 can be configured to physically control access through a door 252. A sensor 16 can be configured to detect scenes related to determining whether access is permitted. The sensor 16 can be an optical sensor configured to generate and transmit image and / or video data. The control system 12 can use such data to detect a person's face.

[0066] The classifier 24 of the control system 12 of the surveillance system 250 can be configured to determine the identity of a person by interpreting image and / or video data by matching it with the identities of known persons stored in non-volatile storage device 26. The classifier 12 can be configured to generate and actuator control commands 20 in response to the interpretation of the image and / or video data. The control system 12 is configured to transmit the actuator control commands 20 to the actuator 12. In this embodiment, the actuator 12 can be configured to lock or unlock door 252 in response to the actuator control commands 20. In other embodiments, non-physical logical access control is also possible.

[0067] The monitoring system 250 can also be a supervision system. In such an embodiment, sensor 16 can be an optical sensor configured to detect the supervised scene, and control system 12 is configured to control display 254. Classifier 24 is configured to determine the classification of the scene, such as whether the scene detected by sensor 16 is suspicious. Control system 12 is configured to transmit actuator control command 20 to display 254 in response to classification. Display 254 can be configured to adjust the displayed content in response to actuator control command 20. For example, display 254 can highlight objects that classifier 24 considers suspicious.

[0068] Figure 11A schematic diagram of a control system 12 is depicted, configured to control an imaging system 300, such as an MRI apparatus, an X-ray imaging apparatus, or an ultrasound apparatus. Sensor 16 may be, for example, an imaging sensor. Classifier 24 may be configured to determine a classification of all or part of the sensed image. Classifier 24 may be configured to determine or select actuator control command 20 in response to a classification obtained by a trained neural network. For example, classifier 24 may interpret a region of the sensed image as a potential anomaly. In this case, actuator control command 20 may be determined or selected such that display 302 displays the image and highlights the potentially anomalous region.

[0069] The processes, methods, or algorithms disclosed herein may be deliverable to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, or algorithms may be stored in various forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media such as ROM devices and information changeably stored on writable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. The processes, methods, or algorithms may also be implemented in a software executable object. Alternatively, the processes, methods, or algorithms may be embodied, wholly or partially, using suitable hardware components, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or a combination of hardware, software, and firmware components.

[0070] While exemplary embodiments have been described above, they are not intended to describe all possible forms encompassed by the claims. The language used in this specification is descriptive and not restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form further embodiments of the invention that may not be explicitly described or illustrated. While various embodiments may have been described as providing an advantage in one or more desired characteristics or being preferred relative to other embodiments or prior art implementations, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, suitability, weight, manufacturability, ease of assembly, etc. Accordingly, to the extent that any embodiment is described as less desirable with respect to one or more characteristics than other embodiments or prior art implementations, these embodiments are not outside the scope of this disclosure and are desirable for a particular application.

Claims

1. A computer-implemented method for predicting classification and training neural networks, comprising: The neural network receives input, which includes image data. The inputs of a neural network are classified using forward propagation, wherein forward propagation includes using a root-finding process to identify fixed points associated with one or more parameters of the neural network, which has an equilibrium function. ,in It is a nonlinear function. W and U It is a linear operator, and b It is a constant; W has In the form of, A and B It is a linear operator, where m is a positive constant; Vectors are calculated using linear transformations of fixed points; The classification responds to the maximum element of the vector and outputs the input. and Backpropagation is used to train a neural network, where backpropagation includes the derivative of the loss that identifies the parameters of the neural network.

2. The computer-implemented method of claim 1, wherein performing backpropagation includes utilizing a forward-backward splitting method.

3. The computer-implemented method of claim 1, wherein performing backpropagation includes utilizing the Peaceman-Rachford splitting method.

4. The computer implementation method according to claim 1, wherein the forward propagation includes identifying the fixed point using a forward-backward splitting method.

5. The computer implementation method of claim 1, wherein the forward propagation includes utilizing the forward propagation until convergence.

6. The computer-implemented method of claim 1, wherein the forward propagation includes identifying the fixed point using the Peaceman-Rachford splitting method.

7. The computer-implemented method of claim 1, wherein the backpropagation comprises multiplying by the Jacobian of the partial derivative of the fixed point of the nonlinear function.

8. The computer-implemented method of claim 1, wherein the backpropagation includes utilizing hyperparameters.

9. A computer implementation method for classifying inputs to a neural network, comprising: The neural network receives input, which includes image data. The forward propagation is used to identify the classification of inputs at a neural network, wherein the forward propagation includes using a root-finding process to identify fixed points associated with the parameters of the neural network, which has a balance function. ,in It is a nonlinear function. W and U It is a linear operator, and b It is a constant; W has In the form of, A and B It is a linear operator, where m is a positive constant; The prediction vector is calculated by using a linear transformation of the fixed point; and The output classifies the input in response to the maximum value of the predicted vector.

10. The computer-implemented method of claim 9, wherein performing forward propagation includes utilizing a forward-backward splitting method.

11. The computer-implemented method of claim 9, wherein performing forward propagation includes utilizing the Peaceman-Rachford splitting method.

12. The computer-implemented method of claim 9, wherein the input at the neural network comprises image data received from one or more cameras.

13. The computer-implemented method of claim 9, wherein a nonlinear function is used to identify the fixed point.

14. A system for training a neural network, comprising: An input interface for accessing input data of a neural network, wherein the input data includes image data; and A processor that communicates with an input interface, the processor being programmed to: Receive input data at the neural network; and A trained neural network is output using forward and backward propagation, wherein forward propagation includes identifying fixed points associated with one or more parameters of the neural network using a root-finding process, and backward propagation includes identifying the derivatives of the loss associated with the network's parameters, wherein the neural network has a balance function. ,in It is a nonlinear function. W and U It is a linear operator, and b It is a constant; W has In the form of, A and B It is a linear operator, where m is a positive constant.

15. The system of claim 14, wherein performing backpropagation includes utilizing a forward-backward splitting method.

16. The system of claim 14, wherein performing backpropagation includes utilizing the Peaceman-Rachford splitting method.

17. The system of claim 14, wherein the processor is further programmed to adjust the parameters of the neural network using gradients associated with the parameters.

18. The system of claim 14, wherein the processor is further programmed to perform forward propagation for each mini-batch in response to an index.

19. The system of claim 14, wherein the neural network is a convolutional neural network.

Images