Memory recall for neural networks

Abstract

Memory recall for a neural network includes: receiving one or more inputs for the neural network; determining whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes a plurality of entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; providing the corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing the output by processing the one or more inputs by the neural network to generate an output in response to the entry not being stored in the memory lookup table.

Description

Background Technology

[0001] This disclosure relates to methods, apparatus, and products for memory recall in neural networks. Summary of the Invention

[0002] According to embodiments of this disclosure, various methods, apparatuses, and products for memory recall of neural networks are described herein. In some aspects, memory recall of a neural network includes: receiving one or more inputs for the neural network; determining whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes a plurality of entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; and, in response to the entry being stored in the memory lookup table, providing the corresponding output for the entry; and, in response to the entry not being stored in the memory lookup table, providing the output by processing the one or more inputs by the neural network to generate an output. This allows recalling outputs with high confidence generated by the neural network, preventing low-confidence outputs due to drift, and reducing the overall computational overhead used by the neural network.

[0003] In some aspects, determining whether an entry corresponding to the one or more inputs is stored in the memory lookup table includes: generating a signature for the one or more inputs by processing the one or more inputs by the neural network up to a memory check layer of the neural network; and comparing the signature for the one or more inputs with the memory lookup table. This allows the memory lookup table to be referenced using a sequence of activated neurons and their corresponding confidence levels to identify previously generated high-confidence outputs.

[0004] In some aspects, the method may include: in response to meeting an output threshold for the one or more inputs, marking the one or more inputs for potential storage in the memory lookup table. This allows inputs with high-confidence outputs to be marked for future analysis and evaluation regarding whether they should be included in the memory lookup table.

[0005] In some aspects, an apparatus may include a processing device; and a memory operatively coupled to the processing device, wherein the memory stores computer program instructions that, when executed, cause the processing device to: receive one or more inputs to a neural network; receive one or more inputs for the neural network; determine whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes a plurality of entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; and, in response to the entry being stored in the memory lookup table, provide the corresponding output for the entry; and, in response to the entry not being stored in the memory lookup table, provide the output by processing the one or more inputs by the neural network to generate an output. This allows for the invocation of high-confidence outputs generated by the neural network, prevents low-confidence outputs due to drift, and reduces the overall computational overhead used by the neural network.

[0006] In some aspects, determining whether an entry corresponding to the one or more inputs is stored in the memory lookup table includes: generating a signature for the one or more inputs by processing the one or more inputs by the neural network up to a memory check layer of the neural network; and comparing the signature for the one or more inputs with the memory lookup table. This allows the memory lookup table to be referenced using a sequence of activated neurons and their corresponding confidence levels to identify previously generated high-confidence outputs.

[0007] In some respects, computer program instructions, when executed, cause the processing device to: mark the one or more inputs for potential storage in the memory lookup table in response to meeting an output threshold for the one or more inputs. This allows inputs with high-confidence outputs to be marked for future analysis and evaluation regarding whether they should be included in the memory lookup table.

[0008] In some aspects, a computer program product includes a computer-readable storage medium comprising computer program instructions that, when executed, receive one or more inputs to a neural network; receive the one or more inputs for the neural network; determine whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes a plurality of entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; and, in response to the entry being stored in the memory lookup table, provide the corresponding output for the entry; and, in response to the entry not being stored in the memory lookup table, provide the output by processing the one or more inputs by the neural network to generate an output. This allows for the invocation of high-confidence outputs generated by the neural network, prevents low-confidence outputs due to drift, and reduces the overall computational overhead used by the neural network.

[0009] In some aspects, determining whether an entry corresponding to the one or more inputs is stored in the memory lookup table includes: generating a signature for the one or more inputs by processing the one or more inputs by the neural network up to a memory check layer of the neural network; and comparing the signature for the one or more inputs with the memory lookup table. This allows the memory lookup table to be referenced using a sequence of activated neurons and their corresponding confidence levels to identify previously generated high-confidence outputs.

[0010] In some respects, when executed, the computer program instructions: in response to meeting an output threshold for the one or more inputs, mark the one or more inputs for potential storage in the memory lookup table. This allows inputs with high-confidence outputs to be marked for future analysis and evaluation regarding whether they should be included in the memory lookup table. Attached Figure Description

[0011] Figure 1 A block diagram of an example computing environment for memory recall for a neural network according to some embodiments of the present disclosure is shown.

[0012] Figure 2 An example flowchart of memory recall for a neural network according to some embodiments of the present disclosure is shown.

[0013] Figure 3 Another example flowchart of memory recall for a neural network according to some embodiments of the present disclosure is shown.

[0014] Figure 4A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0015] Figure 5 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0016] Figure 6 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0017] Figure 7 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0018] Figure 8 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0019] Figure 9 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown.

[0020] Figure 10 A flowchart of another example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Detailed Implementation

[0021] Artificial intelligence models, such as neural networks, are trained to recognize patterns and become effective at making predictions and decisions. Such models can be trained continuously over time (e.g., repeatedly) using additional training data, such as inputs previously provided to the model and outputs previously generated by the model. If the input data to the model, and therefore the training data used for the model, changes significantly over time, the model may drift. For example, a visual recognition module, such as those used in autonomous vehicles, can become efficient at recognizing items in a first environment in which it operates (e.g., in a city) through reinforcement learning. If the vehicle is later used in a second, different environment, such as a rural area, reinforcement learning will modify the model to adapt to its new environment and better recognize items in the rural area. If the vehicle returns to the urban environment, the visual recognition system may struggle to recognize items that were previously effectively recognized. As another example, large language models (LLMs) can be trained based on an influx of information about a particular viral epidemic, thus becoming more effective at providing answers relevant to that topic. The LLM can then adapt over time to new questions after the viral influx relevant to that topic disappears. In the future, the LLM may not produce results relevant to the previous viral topic with the same confidence. Therefore, if a model drifts over time due to continuous retraining, it may be beneficial to provide a mechanism for neural networks or other models to recall information from previous outputs with high confidence.

[0022] Now for reference Figure 1 This document illustrates example computing environments according to various aspects of the present disclosure. Computing environment 100 includes examples of execution environments such as neural network memory module 107, which are involved in performing at least some of the computer code described herein. In addition to neural network memory module 107, computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end-user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, computer 101 includes a processor set 110 (including processing circuitry 120 and a cache 121), communication infrastructure 111, volatile memory 112, persistent storage device 113 (including an operating system 122 and block 107, as described above), a peripheral device set 114 (including a user interface (UI) device set 123, storage device 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. Remote server 104 includes a remote database 130. Public cloud 105 includes gateway 140, cloud coordination module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0023] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases such as remote database 130. As is well known in the field of computer technology, and depending on the technology, the performance of a computer-implemented method can be distributed across multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the detailed discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 can reside in the cloud, even... Figure 1 It is not shown in the cloud, and on the other hand, computer 101 does not need to be in the cloud unless it can be indicated with certainty to any extent.

[0024] Processor assembly 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple cooperating integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be readily accessible by the threads or cores running on processor assembly 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache in the processor assembly may be located “off-chip.” In some computing environments, processor assembly 110 may be designed to work with qubits and perform quantum computing.

[0025] Computer-readable program instructions are typically loaded onto computer 101 to cause the processor set 110 of computer 101 to perform a series of operational steps to implement a computer-implemented method, such that the instructions thus executed instantiate the method specified in the flowcharts and / or descriptive descriptions of the computer-implemented method included in this document. These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the computer-implemented method. In computing environment 100, at least some of the instructions for performing the method of the present invention may be stored in block 107 of permanent storage device 113.

[0026] Communication structure 111 is a signal transmission path that allows the various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths that form buses, bridges, physical input / output ports, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0027] Volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, but this is not necessary unless explicitly stated otherwise. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally relative to computer 101.

[0028] Persistent storage device 113 is any form of non-volatile memory known now or developed in the future for use in a computer. The non-volatility of this memory means that the stored data is retained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage device 113. Persistent storage device 113 may be read-only memory (ROM), but typically at least a portion of persistent memory allows data to be written, deleted, and rewritten. Some common forms of persistent storage include hard disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or operating systems employing an open-source portable operating system interface type with a kernel. The code included in block 107 generally includes at least some of the computer code involved in performing the computer implementation of the methods described herein.

[0029] Peripheral device set 114 includes a set of peripheral devices for computer 101. Data communication connections between peripheral devices and other components of computer 101 can be implemented in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections made by cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., secure digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as displays, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage device 124 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. Storage device 124 can be permanent and / or volatile. In some embodiments, storage device 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 requires substantial storage (e.g., where computer 101 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed for storing very large amounts of data, such as a Storage Area Network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 comprises sensors that can be used in IoT applications. For example, one sensor could be a thermometer, while another could be a motion detector.

[0030] Network module 115 is a collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers via WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi transceiver, software for packetizing and / or depacketizing data transmitted over the communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing computer-implemented methods can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in network module 115.

[0031] WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any technology known now or developed in the future for transmitting computer data. In some embodiments, WAN 102 may be replaced by and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0032] End User Equipment (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and can take any of the forms discussed above in conjunction with computer 101. EUD 103 typically receives useful and available data from the operation of computer 101. For example, assuming computer 101 is designed to provide recommendations to an end user, these recommendations are typically transmitted from network module 115 of computer 101 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present recommendations to the end user. In some embodiments, EUD 103 can be client equipment such as a thin client, heavy client, mainframe, desktop computer, etc.

[0033] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 can be controlled and used by the same entity operating computer 101. Remote server 104 represents a machine that collects and stores useful and available data used by other computers, such as computer 101. For example, if computer 101 is designed and programmed to provide recommendations based on historical data, that historical data can be provided to computer 101 from a remote database 130 of remote server 104.

[0034] Public cloud 105 is any computer system that can be used by multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities (particularly data storage (cloud storage) and computing power) without the need for direct, active management by users. Cloud computing typically leverages resource sharing to achieve scalability consistency and economy. Direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud coordination module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers constituting the host physical machine set 142, which is the entirety of physical computers in and / or available to the public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCEs are instantiated. Cloud coordination module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages the active instantiation of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that allow public cloud 105 to communicate via WAN 102.

[0035] Now, we will provide some further explanation of Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of the programs running within them, these isolated user-space instances typically appear as actual computers. Computer programs running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this is a characteristic known as containerization.

[0036] Private cloud 106 is similar to public cloud 105, except that computing resources are available only to a single enterprise. While private cloud 106 is depicted as communicating with WAN 102, in other embodiments, private cloud may be completely disconnected from the Internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardization or proprietary technology that enables coordination, management, and / or data / application portability across the multiple component clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0037] Figure 2 A flowchart 200 is provided illustrating a method for memory recall for a neural network according to some embodiments of the present disclosure. Beginning at block 205, an initially trained neural network is deployed. Although the methods described herein are presented in the context of neural networks, those skilled in the art will understand, as will be appreciated, that the methods described herein can be applied to any type of trained model. For example, as will be appreciated, the methods described herein can be applied to neural networks (e.g., feedforward neural networks, natural language processing models, LLMs, etc.), regression models, deep learning models, support vector machines (SVMs), random forests, other generative artificial intelligence models, and / or other machine learning models.

[0038] At block 210, input for the neural network is received. Input is specific data to be provided to the neural network for processing to generate a particular output. The input to the neural network can be received from various sources. For example, the input to the neural network can be generated by a process or service executing within the same device or computing environment as the neural network. As another example, input can be received via a network from a remotely located computing device. The input can include any type of data that can be understood, such as text data, image data, video data, etc.

[0039] In block 215, it is determined whether the input has been labeled for potential storage in a memory lookup table, which will be described in further detail below. For example, in some embodiments, a table or data structure (e.g., a labeling table) may store a particular input or its identifier, such as a hash value of the particular input, such that including them in the labeling table indicates that the input has been labeled. If not, the process proceeds to block 220, where the input is run through a pre-checking layer of the neural network. A pre-checking layer is one or more layers of the neural network preceding the memory-checking layer, in which the signature of the input is compared against the memory lookup table. In other words, the pre-checking layers are those layers of the neural network that will always be run or applied to a given input, regardless of whether the output of that input has a corresponding output stored in the memory lookup table.

[0040] In some embodiments, the memory-checking layer may be the first layer of a neural network, meaning that the memory lookup table can be accessed or queried without any processing by the neural network. In some embodiments, the memory-checking layer may be any other layer within the neural network. In some embodiments, the neural network may include multiple memory-checking layers. In such embodiments, the current signature of the input may be used to query the memory lookup table at each encountered memory-checking layer. Those skilled in the art will understand that a neural network with fewer memory-checking layers can be more efficient when the input must be fully processed by the neural network to generate an output (e.g., in the case where no entries are stored in the memory lookup table). However, a neural network with more checking layers can generate more recall from the memory lookup table and produce faster output generation for those input sets. In some embodiments, the optimal or preferred layer for the last (or only) memory-checking layer may be calculated based on the number of previous checks (e.g., queries to the memory lookup table), where performing checks after that layer offers no computational benefit (e.g., based on the relative computational burden of performing memory-check lookups compared to processing neural network layers).

[0041] At block 225, it is determined whether the memory check layer has been reached. If not, the process returns to block 220 to continue to the next layer of the neural network. If yes, the process proceeds to block 230, where the current signature of the input is used to query the memory lookup table. The signature of a given input includes the activated neurons and output accuracy predictions (e.g., output confidence scores or percentages) at each layer up to the current memory check layer. A memory lookup table is a database, table, or other data structure that associates specific neural network inputs with their respective outputs. Specifically, each entry in the memory lookup table can associate some inputs previously provided to the neural network with their respective outputs, including previous versions of the neural network. As described herein, a version of the neural network is an instance of a neural network trained on some corpus of training data. Therefore, each retraining of the neural network, whether continuous or repeated, produces a different version of the neural network.

[0042] Each entry in a memory lookup table includes a value or identifier of a specific input to a neural network and an output generated by a version of the neural network based on that specific input. In some implementations, the value of the identifier of a specific input may include a signature of that input generated when the input is processed by the neural network. Therefore, in some embodiments, the signature of the input may be used to index or reference the memory lookup table. For example, in some embodiments, the signature of the input may be hashed or otherwise processed to generate an index value for accessing the memory lookup table. In some embodiments, the memory lookup table may be traversed to compare signatures in stored entries with the signature of the current input. In some embodiments, the signature of the input may be considered a match of the signatures of entries in the memory lookup table, wherein each point in the sequence of the input signatures falls within a range or standard deviation (e.g., a “tolerance range”) of the corresponding point in the signatures of the entries in the memory lookup table. In some embodiments, the signatures in the memory lookup table may correspond to a different level or layer than the signature of the input. For example, depending on when the entry in the memory lookup table is created, the signature in the memory lookup table entry may end at a layer later or earlier than the signature of the input. In such embodiments, the memory lookup table and / or the signature of the input can be truncated so that they terminate at the same layer, after which they can be compared as described above. In some embodiments, instead of using signatures, a memory neural network can be used. A memory neural network is a portion of a previous version of a neural network up to a specific verification layer. For example, a memory neural network can be used when the performance on a particular input set is above a high threshold, such as 98 percent accuracy.

[0043] In block 235, it is determined whether a match for the input signature is found in the memory lookup table. If a match is found in block 240, the corresponding entry in the memory lookup table is accessed. The output stored in that entry is then provided as a response to the received input. In some embodiments, the process proceeds to block 245, where the signature of the matching entry in the memory lookup table is updated if necessary. For example, if the features of the input matching the entry drift toward one end of the tolerance range or the other, the model may experience some drift, but will still be accurate when producing the output. Therefore, the signature of the entry in the memory lookup table can be adjusted such that the newer input signature will not fall outside the tolerance range of the entry in the memory lookup table. After performing any signature updates, the process returns to block 210, where another input to the neural network is received.

[0044] Returning to block 235, if no match is found in the memory lookup table, the process proceeds to block 250, where the remaining layers of the neural network after the memory-checking layer are used to process the input to generate some output. In some embodiments, instead of performing the determination of whether the input is labeled by the pre-checking layer before processing the input at block 215, the determination is performed before block 250, and if no match is found. In such embodiments, the label table may instead store signatures of labeled inputs, allowing the label table to be queried using the currently generated signature.

[0045] In block 255, the neural network can be continuously retrained using the most recent inputs and outputs. For example, in some implementations, the neural network can be retrained after each time it generates an output using some inputs. In some embodiments, the neural network can be retrained every N iterations (e.g., after the neural network has generated N outputs), at predetermined intervals using the inputs and the outputs generated during those intervals, or in response to other understandable conditions. The neural network can be retrained using supervised learning, unsupervised learning, semi-supervised learning, or any other understandable training method.

[0046] The process then proceeds to block 260, where it is determined whether an output threshold has been reached for a given input (e.g., the input processed during the most recent iteration of the neural network). The output threshold is satisfied if the confidence level of the output for the given input exceeds a certain threshold (e.g., 98 percent confidence or greater). In some embodiments, the output threshold may be satisfied if the confidence level of the output for the given input exceeds an iteration count threshold (e.g., N outputs of the given input have confidence levels higher than the output threshold) or a time threshold (e.g., the output for the given input has a confidence level higher than the output threshold for at least one month).

[0047] If the output threshold is met, then in block 265, the given input is labeled for potential storage (e.g., inclusion) in a memory lookup table. For example, an entry for the input can be added to the labeling table as described above, where the entry includes the input, a hash value based on the input, a signature for the input, or other information as may be understood depending on the specific implementation of the labeling table. After labeling the input in block 265, or if the output threshold is not met in block 260, the process returns to block 210, where the neural network can receive another input. Returning to block 215, for the input to the neural network that has previously been labeled for potential storage in the memory lookup table, the process proceeds to block 270, where execution takes place. Figure 3 The process described in flowchart 300.

[0048] therefore, Figure 3 A flowchart 300 is shown for determining whether a labeled input should have an entry added to a memory lookup table. In block 305, the input is processed through all layers of the neural network until a predetermined layer threshold is reached. In some embodiments, the layer threshold may be a layer where querying the memory lookup table instead of fully processing the input using the neural network may no longer be computationally beneficial. In other words, the layer threshold may be a layer where the computational burden of querying the memory lookup table exceeds the computational burden of processing the input using the remaining layers. In some embodiments, the layer threshold may include layers where a validation layer has already been added. In some embodiments, the layer threshold may be a layer after some other validation layers to allow for moving validation layers or adding new validation layers to the neural network. In some embodiments, the layer threshold may be after the last layer of the neural network, such that the input has been fully processed.

[0049] At block 310, the output accuracy prediction for each layer of the processed input (e.g., each layer that reaches a layer threshold) is calculated. At block 315, it is determined whether a layer output threshold has been reached. Here, a layer output threshold is reached if the output accuracy prediction for each layer exceeds a certain threshold. In some embodiments, this threshold may be different from (e.g., greater than or less than) the initial threshold used to label the input described above. In some embodiments, this threshold may be configured based on specific design or engineering considerations.

[0050] If a layer output threshold is reached, the process proceeds to block 320, where the output sequence (e.g., the activated neurons of each layer for processing the labeled input and their corresponding output accuracy predictions) is stored in a temporary table. As an entry exists in this temporary table, the count in the associated entry is incremented. In some embodiments, the count may be incremented only if the output of a given sequence matches the output from the input fully run through the neural network and has a similar sequence up to a certain layer of the neural network. In some embodiments, instead of using a temporary table, new data generated in the flowchart is added to an entry in the label table previously mentioned in the description of flowchart 200.

[0051] In block 325, it is determined whether the sequence threshold for marking the input has been reached. The sequence threshold is a threshold that is compared to the count in the temporary table. Therefore, when the count of entries in the temporary table exceeds a certain threshold, the sequence threshold for marking the input is satisfied. If not, the process moves to block 330, thus returning to... Figure 2 Block 250. If the sequence threshold is met, then in block 335, an entry for the labeled input is created in the memory lookup table using the output sequence of the labeled input as described above. The new entry in the memory lookup table stores the output for the labeled input, which can be retrieved from the memory lookup table when the input is subsequently encountered by the neural network. In some embodiments, since the input has been processed by the neural network up to the memory check layer, the output can be generated by stopping the processing of the neural network or skipping layers up to the output layer. In some embodiments, the output can be stored in a label table when the input is initially labeled. The output in the label table entry can be loaded and stored in a new entry in the memory lookup table. In some embodiments, the output generated by running the input through the entire neural network can be used in a new memory lookup table entry, where the count increases only in response to the output from the sequence matching the output from running through the entire neural network. Then, the label indicating that the input might be stored in the memory lookup table is removed (e.g., by removing the entry in the label table).

[0052] Then, at block 340, a memory check layer for the neural network is determined. This may include determining whether a memory check layer should be added or whether an existing memory check layer should be moved. In some embodiments, the current version of the neural network may be saved until a layer output threshold is reached (e.g., a layer of the neural network that processes previously labeled inputs). In some embodiments, if the layer output threshold is achieved at a layer after the current memory check layer, an additional layer may be added or an existing layer may be shifted. In some embodiments, if the layer output threshold is achieved at a layer before the current memory check layer, an additional layer may be added, or signatures may be tracked up to the current memory check layer, even if a particular signature is not required. In some embodiments, the memory check layer may be shifted based on analysis of entries within the memory lookup table. For example, the memory check layer may be shifted or established at the highest layer in the memory lookup table entry signatures, the average layer at the end of the memory lookup table entry signatures, etc. The process then moves to block 345, returning to... Figure 2 Block 240, thereby accessing the output value from the newly created entry in the memory lookup table.

[0053] The method described above allows for the maintenance of a memory lookup table for a specific neural network. Entries can be created in the memory lookup table where the neural network provides outputs with high confidence for some inputs. If a future version of the neural network encounters a similar input as indicated by the input's signature, the output of that input can be loaded from the memory lookup table instead of fully processing the input to generate a particular output. This improves computational efficiency by eliminating the need to fully process some inputs. Furthermore, this eliminates the possibility of the neural network providing lower-confidence outputs for inputs for which it previously generated high-confidence outputs due to drift.

[0054] To further explain, Figure 4 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 4 The method can be executed, for example, by the neural network memory module 107 described above. Figure 4 The method includes receiving 402 one or more inputs for the neural network. As described above, the neural network can include various neural networks as can be understood, and the one or more inputs can include any type of data that can be analyzed by the neural network or otherwise used to generate some output. In particular, the neural network can include a neural network that is continuously or otherwise repeatedly retrained, thereby generating new versions of the neural network over time. Figure 4 The method also includes determining whether the entry corresponding to one or more inputs for 404 is stored in a memory lookup table for neural networks.

[0055] A memory lookup table is a database, table, or other data structure that associates inputs previously provided to a neural network (e.g., a version of the neural network) with corresponding outputs previously generated by the neural network. For example, in some embodiments, each entry in the memory lookup table may include an identifier of the input, and is therefore indexed using the input identifier. Such an identifier may include a signature generated by processing the input at least partially using the neural network, which will be described in further detail below. Therefore, in some embodiments, determining whether 404, an entry corresponding to one or more inputs, is stored in the memory lookup table for the neural network may include: determining whether an entry in the memory lookup table has an input identifier (e.g., a signature) that matches the input identifier of the received 402 one or more inputs.

[0056] If entries are stored, then Figure 4 The method includes providing a corresponding output for the entry in response to the entry being stored in a memory lookup table. For example, it can be understood that the output value in the matching entry can be accessed and provided to the source or another entity of one or more inputs received in 402. Therefore, the output of one or more inputs received in 402 can be provided without fully processing these inputs using a neural network.

[0057] If no entry is found in the memory lookup table, then Figure 4 The method involves providing a 408 output by processing one or more inputs to generate an output in response to the entry not being stored in the memory lookup table. Therefore, if no entry is found, the input must be fully processed to generate some output. As will be described in further detail below, after processing the received 402 input to generate the output, the received output can be evaluated and labeled for potential inclusion in the memory lookup table, thus saving future computational burden associated with the neural network processing that input.

[0058] In some embodiments, a memory lookup table or some other metadata or data structure can associate a specific input identifier (e.g., an input signature, as described below) with a version of the neural network used to generate a high-confidence output for the corresponding input. For example, a previous version of the neural network or a portion thereof can be stored. In some embodiments, the identifier (e.g., a signature) of a given input can be compared with the identifiers of these stored neural networks. In response to a match, instead of loading a previously generated output, a stored version of the neural network can be loaded, and one or more received inputs can be provided to the loaded neural network to generate some output.

[0059] To further explain, Figure 5A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 5 Methods and Figure 4 The similarities are, Figure 5 The method also includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 the output in response to the entry not being stored in the memory lookup table by processing the one or more inputs by the neural network to generate an output.

[0060] Figure 5 Methods and Figure 4 The difference lies in the fact that determining whether a 404 entry corresponding to one or more inputs is stored in a memory lookup table used by the neural network involves generating a signature for the 404 for one or more inputs by having the neural network process one or more inputs up to a memory-checking layer of the neural network. The memory-checking layer is a designated layer of the neural network that, once reached, uses the generated signature to query the memory lookup table. If no match is found in the memory lookup table, the neural network continues processing the input from the memory-checking layer. The signature for one or more inputs is data describing the neurons activated at each layer of the neural network and the corresponding output accuracy predictions (e.g., confidence scores) for each layer of the neural network used to process the inputs so far (e.g., up to the memory-checking layer).

[0061] Figure 5 Methods and Figure 4The difference lies in that determining whether the entry corresponding to one or more inputs is stored in the memory lookup table for the neural network includes comparing the signatures of the one or more inputs with the memory lookup table. For example, the signatures of one or more inputs can be compared with the signature values ​​of entries in the memory lookup table. Comparing the signatures of one or more inputs with the signature values ​​of entries in the memory lookup table can be performed by iterating through the memory lookup table or by searching the memory lookup table according to a search algorithm that depends on the organization of the memory lookup table. It can be considered that the signatures of one or more inputs (e.g., input signatures) match the signatures in the entries of the memory lookup table (e.g., entry signatures), wherein each value of the input signature matches or falls within a range or standard deviation (e.g., a “tolerance range”) of the corresponding value of the entry signature. In some embodiments, the neural network may include multiple memory checking layers. Therefore, in such an embodiment, generating the signature (502) and comparing the signature with the memory lookup table (504) can be performed at each memory checking layer.

[0062] To further explain, Figure 6 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 6 Methods and Figure 5 The similarities are, Figure 6 The method also includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, including: generating 404 a signature for the one or more inputs by processing the one or more inputs by the neural network until a memory check layer of the neural network is reached; and comparing 504 the signature for the one or more inputs with the memory lookup table; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 the output by processing the one or more inputs by the neural network to generate an output in response to the entry not being stored in the memory lookup table.

[0063] Figure 6 Methods and Figure 5 The difference is that, Figure 6The method also includes updating the signatures of one or more inputs stored in the entry at 602. In some embodiments, one or more values ​​of the input signature (e.g., for a single matching input or multiple matching inputs varying over time) may be shifted if they approach the tolerance range limit of the entry signature. For example, a neural network may be drifting, causing input signatures to approach the tolerance range limit of the entry signature, but the neural network can still provide high-confidence outputs for these inputs. Therefore, one or more values ​​of the entry signature may be shifted to prevent future input signatures from falling outside the tolerance range and to prevent the output from being loaded from the memory lookup table.

[0064] To further explain, Figure 7 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 7 Methods and Figure 4 The similarities are, Figure 7 The method also includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 the output in response to the entry not being stored in the memory lookup table by processing one or more inputs by the neural network to generate an output.

[0065] Figure 7 Methods and Figure 4 The difference is that, Figure 7 The method further includes: 702 marking the one or more inputs for potential storage in a memory lookup table in response to satisfying an output threshold for the one or more inputs. The output threshold is satisfied when the confidence level of the output of the one or more inputs exceeds a certain threshold (e.g., 98% confidence or greater). In some embodiments, the output threshold may be satisfied after exceeding a threshold iteration number (e.g., N outputs from N instances of the one or more received inputs having a confidence level higher than the output threshold) or a threshold time amount (e.g., outputs from the one or more received instances having a confidence level higher than the output threshold for at least one month).

[0066] To further explain, Figure 8 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 8 Methods and Figure 4 The similarities are, Figure 8The method also includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 the output in response to the entry not being stored in the memory lookup table by processing one or more inputs by the neural network to generate an output.

[0067] Figure 8 Methods and Figure 4 The difference is that, Figure 8 The method further includes determining whether one or more additional inputs to 802 are tagged to potentially be stored in a memory lookup table. The one or more additional inputs may include other inputs provided to the neural network for processing. Therefore, in some implementations, determining whether one or more additional inputs to 802 are tagged to potentially be stored in a memory lookup table can be performed prior to any processing of the neural network, such as processing the other one or more inputs up to the memory check layer to generate a signature. In some embodiments, tables, data structures, etc., may store identifiers or other metadata indicating the tagged input values. Thus, such tables or data structures can be accessed to determine whether one or more additional inputs to 802 are tagged.

[0068] Figure 8 The method also includes, in response to the labeling of one or more other inputs, processing 804 of the other one or more inputs by the neural network up to a predetermined layer threshold. In some embodiments, the layer threshold may be a layer where it would no longer be computationally beneficial to query a memory lookup table instead of fully processing the other one or more inputs using the neural network. In other words, the layer threshold may be a layer where the computational burden of querying the memory lookup table exceeds the computational burden of processing the other one or more inputs using the remaining layers. In some embodiments, the layer threshold may include a layer in which a check layer has already been added. In some embodiments, the layer threshold may be a layer after some other check layers to allow for moving check layers or adding new check layers to the neural network. In some embodiments, the layer threshold may be after the last layer of the neural network, such that the other one or more inputs are fully processed.

[0069] Figure 8The method further includes creating 806 new entries in a lookup table for one or more other inputs in response to the output accuracy prediction of at least one layer of the neural network reaching a predetermined layer threshold that satisfies the layer output threshold. Here, the layer output threshold is satisfied if the output accuracy prediction of at least one layer exceeds a certain threshold. In some embodiments, this threshold may be different from (e.g., greater than or less than) the initial threshold used to label one or more other inputs described above. In some embodiments, this threshold may be configured based on specific design or engineering considerations.

[0070] In some embodiments, 806 new entries may be created in a memory lookup table in response to a sequence threshold reached for one or more other inputs. For example, whenever a specific labeled input is received and its layer output threshold is met, an entry may be created in a temporary table, or the count of an existing entry may be updated. Thus, when the count of entries in the temporary table for one or more other inputs meets the sequence threshold, 806 entries for one or more other inputs may be created as new entries in the memory lookup table. The new entry in the memory lookup table stores the output for the labeled input, which can be retrieved from the memory lookup table when the neural network subsequently encounters the input. In some embodiments, since the input has been processed by the neural network up to the memory check layer, the output may be generated by stopping the processing of the neural network or skipping layers up to the output layer. In some embodiments, the output may be stored in a label table when the input is initially labeled. The output in the label table entry may be loaded and stored in a new entry in the memory lookup table. In some embodiments, the output generated by running the input through the entire neural network may be used in a new memory lookup table entry, where the count increases only in response to the output from the sequence matching the output from running through the entire neural network. Tags indicating that one or more inputs may be stored in the memory lookup table may be removed.

[0071] To further explain, Figure 9 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 9 Methods and Figure 8 The similarities are, Figure 9The method also includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 an output by processing one or more inputs by the neural network to generate an output in response to the entry not being stored in the memory lookup table; determining 802 whether another one or more inputs are marked to potentially be stored in the memory lookup table; processing 804 another one or more inputs by the neural network until a predetermined layer threshold is reached in response to the other one or more inputs being marked; and creating 806 new entries in the lookup table for the other one or more inputs in response to the output accuracy prediction of at least one layer of the neural network reaching a predetermined layer threshold that satisfies the layer output threshold.

[0072] Figure 9 The method also includes determining a memory check layer in the 902 neural network. This may include determining whether a memory check layer should be added or whether an existing memory check layer should be moved. In some embodiments, the current version of the neural network may be saved until a layer output threshold is reached (e.g., a layer of the neural network that processes one or more other previously labeled inputs). In some embodiments, if the layer output threshold is achieved at a layer after the current memory check layer, an additional layer may be added or an existing layer may be shifted. In some embodiments, if the layer output threshold is achieved at a layer before the current memory check layer, an additional layer may be added, or signatures may be tracked up to the current memory check layer, even if a specific signature is not required. In some embodiments, the memory check layer may be shifted based on analysis of entries within the memory lookup table. For example, the memory check layer may be shifted or established at the highest layer in the memory lookup table entry signatures, the average layer at the end of the memory lookup table entry signatures, etc.

[0073] To further explain, Figure 10 A flowchart illustrating an example method for memory recall for a neural network according to some embodiments of this disclosure is shown. Figure 10 Methods and Figure 4 The similarities are, Figure 10 The method further includes: receiving 402 one or more inputs for a neural network; determining 404 whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network; providing 406 a corresponding output for the entry in response to the entry being stored in the memory lookup table; and providing 408 the output in response to the entry not being stored in the memory lookup table by processing one or more inputs by the neural network to generate an output.

[0074] Figure 10 Methods and Figure 4 The difference is that, Figure 10 The method also includes periodically retraining the 1002 neural network. For example, in some implementations, the neural network can be retrained after each time it generates an output using some input. In some embodiments, the neural network can be retrained every N iterations (e.g., after the neural network has generated N outputs), at predetermined intervals using the inputs and generated outputs during that interval, or in response to other understandable conditions. The neural network can be retrained using supervised learning, unsupervised learning, semi-supervised learning, or any other understandable training method.

[0075] Various aspects of this disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). Regarding any flowchart, depending on the technology involved, operations may be performed in a different order than that shown in a given flowchart. For example, again according to the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0076] Computer Program Product Embodiment (“CPP Embodiment” or “CPP”) is a term used in this disclosure to describe any collection of one or more storage media (also referred to as “media”) collectively included in a collection of one or more storage devices, the collection of one or more storage devices collectively including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device capable of holding and storing instructions used by a computer processor. Without limitation, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punch cards or pits / platforms formed in the main surface of the disk), or any suitable combination of the foregoing. Computer-readable storage media, as used in this disclosure, should not be construed as storing transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As those skilled in the art will understand, data is typically moved at certain incidental points in time during the normal operation of the storage device, such as during access, defragmentation, or garbage collection; however, this does not make the storage device transient, because the data is not transient when it is stored.

[0077] Various embodiments of this disclosure have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method comprising: Receives one or more inputs for the neural network; Determine whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes multiple entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; In response to the entry being stored in the memory lookup table, provide the corresponding output for the entry; and In response to the entry not being stored in the memory lookup table, the output is provided by the neural network processing the one or more inputs to generate an output.

2. The method according to claim 1, wherein, Determining whether an entry corresponding to one or more inputs is stored in the memory lookup table includes: A signature for the one or more inputs is generated by processing the one or more inputs by the neural network up to the memory check layer of the neural network; and The signature used for the one or more inputs is compared with the memory lookup table.

3. The method according to claim 2, further comprising: Update the signature stored in the entry for the one or more inputs.

4. The method according to claim 1, further comprising: In response to satisfying an output threshold for the one or more inputs, the one or more inputs are marked to be potentially stored in the memory lookup table.

5. The method according to claim 2, wherein, The comparison of the signature used for the one or more inputs with the memory lookup table is based on a tolerance range.

6. The method according to claim 1, further comprising: Determine whether one or more other inputs are marked as potentially stored in the memory lookup table; In response to the labeling of the other one or more inputs, the neural network processes the other one or more inputs until a predetermined layer threshold is reached; as well as In response to the output accuracy prediction for at least one layer of the neural network reaching the predetermined layer threshold that satisfies the layer output threshold, a new entry for the other one or more inputs is created in the memory lookup table.

7. The method according to claim 6, wherein, Creating the new entry in the memory lookup table is performed in response to a count associated with the one or more inputs satisfying a count threshold.

8. The method according to claim 6, further comprising: Identify the memory check layer in the neural network.

9. The method according to claim 1, further comprising: The neural network is retrained periodically.

10. An apparatus comprising: Processing equipment; as well as A memory operatively coupled to the processing device, wherein the memory stores computer program instructions that, when executed, cause the processing device to: Receives one or more inputs for the neural network; Determine whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes multiple entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; In response to the entry being stored in the memory lookup table, provide the corresponding output for the entry; and In response to the entry not being stored in the memory lookup table, the output is provided by the neural network processing the one or more inputs to generate an output.

11. The apparatus according to claim 10, wherein, Determining whether an entry corresponding to one or more inputs is stored in the memory lookup table includes: A signature for the one or more inputs is generated by processing the one or more inputs by the neural network up to the memory check layer of the neural network; and The signature used for the one or more inputs is compared with the memory lookup table.

12. The apparatus according to claim 11, wherein, When executed, the computer program instructions also cause the processing device to update the signature stored in the entry for the one or more inputs.

13. The apparatus according to claim 12, wherein, When the computer program instructions are executed, the processing device also causes the processing device to mark the one or more inputs for potential storage in the memory lookup table in response to satisfying an output threshold for the one or more inputs.

14. The apparatus according to claim 11, wherein, The comparison of the signature used for the one or more inputs with the memory lookup table is based on a tolerance range.

15. The apparatus according to claim 10, wherein, When the computer program instructions are executed, they also cause the processing device to: Determine whether one or more other inputs are marked to potentially be stored in the memory lookup table; In response to the labeling of the other one or more inputs, the neural network processes the other one or more inputs until a predetermined layer threshold is reached; as well as In response to the output accuracy prediction for at least one layer of the neural network reaching the predetermined layer threshold that satisfies the layer output threshold, a new entry for the other one or more inputs is created in the memory lookup table.

16. The apparatus according to claim 15, wherein, Creating the new entry in the memory lookup table is performed in response to a count associated with the one or more inputs satisfying a count threshold.

17. The apparatus according to claim 15, wherein, When the computer program instructions are executed, the processing device also determines the memory check layer in the neural network.

18. The apparatus according to claim 10, wherein, When the computer program instructions are executed, the processing device also causes the neural network to periodically retrain.

19. A computer program product comprising a computer-readable storage medium, wherein, The computer-readable storage medium includes computer program instructions, which, when executed: Receives one or more inputs for the neural network; Determine whether an entry corresponding to the one or more inputs is stored in a memory lookup table for the neural network, wherein the memory lookup table includes multiple entries, each of which associates a corresponding neural network input with a corresponding output generated by a version of the neural network; In response to the entry being stored in the memory lookup table, provide the corresponding output for the entry; and In response to the entry not being stored in the memory lookup table, the output is provided by the neural network processing the one or more inputs to generate an output.

20. The computer program product according to claim 19, wherein, Determining whether an entry corresponding to one or more inputs is stored in the memory lookup table includes: A signature for the one or more inputs is generated by processing the one or more inputs by the neural network up to the memory check layer of the neural network; and The signature used for the one or more inputs is compared with the memory lookup table.

21. The computer program product according to claim 20, wherein, When the computer program instructions are executed, they update the signature stored in the entry for the one or more inputs.

22. The computer program product according to claim 21, wherein, When the computer program instructions are executed, in response to satisfying an output threshold for the one or more inputs, the one or more inputs are marked to be potentially stored in the memory lookup table.

23. The computer program product according to claim 20, wherein, The comparison of the signature used for the one or more inputs with the memory lookup table is based on a tolerance range.

24. The computer program product according to claim 19, wherein, When the computer program instructions are executed: Determine whether one or more other inputs are marked to potentially be stored in the memory lookup table; In response to the labeling of the other one or more inputs, the neural network processes the other one or more inputs until a predetermined layer threshold is reached; as well as In response to the output accuracy prediction for at least one layer of the neural network reaching the predetermined layer threshold that satisfies the layer output threshold, a new entry for the other one or more inputs is created in the memory lookup table.

25. The computer program product according to claim 24, wherein, Creating the new entry in the memory lookup table is performed in response to a count associated with the one or more inputs satisfying a count threshold.

Images