Dynamic data-dependent neural network processing system and method

Abstract

Dynamic data-dependent neural network processing systems and methods improve computational efficiency in neural network processing by uniquely processing data based on the data itself and / or configuration parameters used to process the data. In embodiments, this is achieved by receiving, at a controller, input data to be processed by a first device using a first set of parameters in a first layer in a sequence of processing layers of a neural network. The input data is analyzed to determine whether it is modified, whether processing the (modified) data in a second layer will save at least one computational resource, or whether a different set of parameters is applied. Based on the determination, the sequence of processing layers is modified and the (modified) data is processed according to the modified sequence to reduce data movement and conversion, thereby saving computational resources.

Description

[0001] Cross-reference to related applications

[0002] This invention claims priority to U.S. Nonprovisional Patent Application No. 17 / 472,074, filed September 10, 2021, by Mark Alan Lovell and Robert Michael Muxell, entitled “DYNAMIC DATA-DEPENDENT NEURAL NETWORK PROCESSING SYSTEMS AND METHODS”. The foregoing patent document is incorporated herein by reference in its entirety. background

[0003] A. Technical Field

[0004] This disclosure generally relates to data processing in machine learning applications. More specifically, this disclosure relates to systems and methods for improving computational efficiency in machine learning applications.

[0005] B. Background Technology

[0006] Machine learning, a subfield of artificial intelligence, enables computers to learn from examples without explicit programming in the conventional sense. Many machine learning applications utilize convolutional neural networks (CNNs), supervised networks capable of solving complex classification or regression problems, such as those for image or video processing applications. CNNs use large amounts of multidimensional training data (such as images or sensor data) as input to learn salient features from that data. The trained network can be fine-tuned to learn additional features. During the inference phase, i.e., after training or learning is complete, CNNs use unsupervised operations to detect or insert previously unseen features or events into new input data to classify objects or compute outputs (such as regression). For example, a CNN model can be used to automatically determine whether an image can be classified as containing a person or an animal. CNNs apply multiple hierarchical network layers and sublayers to the input image when making their determinations or predictions. Among other parameters, network layers are defined by kernel size. Convolutional layers can use several kernels that apply a set of weights to the pixels of a convolutional window of the image. For example, a two-dimensional convolution operation involves generating an output feature map for the current layer using data from a two-dimensional window of the previous layer. As the amount of data subjected to convolution operations increases and the complexity of the operations continues to grow, the additional steps of storing and retrieving intermediate results from memory to complete arithmetic operations only reveal some of the shortcomings of existing designs.

[0007] Therefore, there is a need for systems and methods that allow hardware, including conventional hardware accelerators, to perform a large number of complex processing steps on large amounts of data with low power and high efficiency, ideally without increasing hardware costs. Attached Figure Description

[0008] Reference will be made to embodiments of the invention, examples of which may be illustrated in the accompanying drawings. These drawings are intended to be illustrative and not restrictive. While the invention has been described in general within the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these specific embodiments. Items in the drawings are not drawn to scale.

[0009] Figure 1 illustrates a typical embedded machine learning accelerator system.

[0010] Figure 2 A dynamic data-dependent neural network processing system according to various embodiments of this disclosure is demonstrated.

[0011] Figure 3 This is a flowchart of dynamic data-dependent neural network processing according to various embodiments of this disclosure.

[0012] Figure 4 This is a block diagram illustrating exemplary condition execution control circuitry according to various embodiments of this disclosure.

[0013] Figure 5 A simplified block diagram of a computing device / information processing system according to embodiments of the present disclosure is depicted. Detailed Implementation

[0014] In the following description, specific details are set forth for purposes of explanation in order to provide an understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these details. Furthermore, those skilled in the art will recognize that the embodiments of the invention described below can be implemented in various ways, such as processes, apparatuses, systems, devices, or methods, on tangible computer-readable media.

[0015] The components or modules shown in the figures illustrate exemplary embodiments of the invention and are intended to avoid obscuring the invention. It should also be understood that throughout this discussion, a component can be described as a separate functional unit that may include subunits; however, those skilled in the art will recognize that various components or portions thereof may be divided into separate components or may be integrated together, including within a single system or component. It should be noted that the functions or operations discussed herein can be implemented as components. Components can be implemented in software, hardware, or a combination thereof.

[0016] Furthermore, the connections between components or systems shown in the accompanying drawings are not intended to be limited to direct connections. Instead, data between these components can be modified, reformatted, or otherwise altered via intermediate components. Moreover, additional connections or fewer connections may be used. It should also be noted that the terms “coupled,” “connected,” or “communicationally coupled” should be understood to include direct connections, indirect connections via one or more intermediate devices, and wireless connections.

[0017] In this specification, references to "one embodiment," "preferred embodiment," "an embodiment," or "multiple embodiments" mean that a specific feature, structure, characteristic, or function described in connection with an embodiment is included in at least one embodiment of the invention and may be included in more than one embodiment. Furthermore, the above phrases appearing in various places throughout this specification do not necessarily refer to the same embodiment or the same embodiment.

[0018] The use of certain terms in various places throughout this specification is illustrative and should not be construed as restrictive. A service, function, or resource is not limited to a single service, function, or resource; the use of these terms may refer to related groups of services, functions, or resources, which may be distributed or aggregated.

[0019] In this document, the terms “optimize,” “optimization,” etc., refer to improvements in results or processes, and do not require that the specified results or processes have reached an “optimal” or peak state. The terms “include,” “including,” “comprise,” and “comprising” should be understood as open-ended terms, and any listed items following them are examples and not intended to limit us to those listed items. The terms “memory,” “memory device,” and “register” are used interchangeably. Similarly, the terms kernel, filter, weight, parameter, and weight parameter are used interchangeably. The term “layer” refers to a neural network layer. “Neural network” includes any neural network known in the art. “Hardware accelerator” refers to any circuit or optical loop that can be used to perform mathematical operations and related functions (including auxiliary control functions). “Circuit” includes “subcircuit” and can refer to both custom circuits (such as special hardware) and general-purpose circuits. The terms “computational resource,” “computing resource,” “computing efficiency,” and “data processing efficiency” refer to the computing speed, network capacity, power efficiency, and similar parameters (including metrics that measure performance and computing resources, such as latency and throughput) in computing systems and other circuits.

[0020] Figure 1 illustrates a typical embedded machine learning accelerator system that processes data in multiple stages. System 100 includes volatile memory 102, non-volatile memory 104, a clock 106, clock I / O peripherals, a microcontroller 110, a power supply 112, and a machine learning accelerator 114. The microcontroller 110 can be a conventional DSP or a general-purpose computing device, and the machine learning accelerator 114 can be implemented as a CNN accelerator including hundreds of registers (not shown). As depicted in Figure 1, the machine learning accelerator 114 interfaces with the other parts of the embedded machine learning accelerator system 100.

[0021] In operation, the microcontroller 110 performs arithmetic operations in software. The machine learning accelerator 114 typically uses weight data to perform matrix multiplication and related convolution calculations on input data to which the weight data is applied. The weight data can be unloaded from the accelerator 114, for example, to load new or different weight data before the accelerator 114 performs a new set of operations using a new set of weight data. More commonly, the weight data remains unchanged, and each new computation involves loading new input data into the accelerator 114 to perform the computation.

[0022] For at least some of the many possible neural network computations, the machine learning accelerator 114 lacks hardware acceleration. These missing operators are typically simulated in software using software functions embedded in the microcontroller 110. However, such methods are very costly in terms of both power and time; and for many computationally intensive applications, such as real-time applications, general-purpose computing hardware cannot perform the necessary operations in a timely manner because the computational rate is limited by the computational resources and capabilities of existing hardware designs.

[0023] Furthermore, using the arithmetic functions of the microcontroller 110 to generate intermediate results comes at the cost of computation time, as it adds steps to transferring data, allocating storage, and retrieving intermediate results from memory locations to complete the operation. For example, many conventional multipliers are scalar machines that use a CPU or GPU as their computational unit and use registers and caches to process data stored in non-volatile memory, relying on a series of software and hardware matrix operation steps (such as address generation, transpose, bitwise addition, and shift) to convert multiplication into addition and output the result to a specific internal register. In practice, these repetitive read / write operations performed on input data with a large number of weight parameters and multiple dimensions and / or a large number of channels often result in unwanted data movement in the data path and thus increase power consumption.

[0024] Existing neural network systems load data into memory and process input data (such as image or audio data) indiscriminately, regardless of content or format—that is, without checking any characteristics of the input data. However, in many cases, not all loaded data is processed. Furthermore, the data processed is often handled in a predetermined order based on a network model trained in a specific manner (e.g., in a fixed order of predetermined layers in the neural network). This rigidity in handling large amounts of data in this traditional way leads to many unnecessary preparation and computational steps that consume valuable computational resources, including computation time and power. The computational complexity involved in convolutions and other operations performed by CNNs, and the associated excessive power consumption, makes more efficient hardware acceleration and energy saving particularly desirable.

[0025] Therefore, flexible systems and methods are needed that can intelligently determine how to load and process input data in neural networks to optimize the use of available computing resources and significantly reduce power consumption without negatively impacting the overall operation or performance of the computing system.

[0026] Figure 2Dynamic data-dependent neural network processing systems according to various embodiments of this disclosure are illustrated. In embodiments, system 200 may include branch circuitry or conditional execution control circuitry coupled to receive input data. As used herein, the term "input data" includes raw data as well as some or all of the data output by a neural network layer or a combination of two or more network layers. The data-dependent neural network processing system 200 may further include registers that may include configuration data, such as weight data or any other type of configuration information.

[0027] It should be understood that, in the embodiments, each computational unit in the data-dependent neural network processing system 200 may be directly or indirectly routed to each other via feedback loops (not shown), and weight memory and data memory (not shown) may be coupled to each processing unit. Each layer / circuit may act as at least partially independent computational unit, which may be assigned dedicated tasks, such as performing operations that can be executed in parallel on some or all of the entire CNN network.

[0028] In embodiments, the conditional execution control circuitry may include preprocessing circuitry and comparator circuitry (e.g., logic circuitry), and may be implemented at one or more output stages of the processor. The preprocessing circuitry may be implemented as a CPU, a DSP, or logic circuitry that may be implemented, for example, in a CNN accelerator. In operation, the conditional execution control circuitry may output modified data and / or a modified sequence of processing layers in which the input data is to be processed.

[0029] In embodiments, the conditional execution control circuitry may further monitor and / or receive input data, such as image or audio data to be processed in a neural network layer, for example, based on a network model that may include any number of neural network layers. As described in more detail below, for the purposes of this disclosure, each individual layer may be processed by one or more circuits or processors. In embodiments, the network model may include an initial sequence of processing layers, for which the model may have been independently trained. It should be understood that more than one network model may be used, for example, in combination with each other as a hybrid or overall model.

[0030] In an embodiment, the preprocessing circuitry may analyze at least some of the input data to determine how some or all of the input data should be loaded into a memory device (not shown) for further processing, and how the input data should be modified if computational efficiency is to be improved. Similarly, the preprocessing circuitry may analyze the input data to determine where or how to preprocess or process the input data and / or modified input data (hereinafter also referred to as "input data") in any neural network layer (e.g., in a sequence different from the initial sequence of the processing layer), for example, to improve computational efficiency. In an embodiment, to further improve computational efficiency, the conditional execution control circuitry may, for example, obtain a set of network or configuration parameters from a memory register, and using any number of such parameters, for example along with information about the input data, determine whether to modify the parameters and apply the modified set of parameters to the input data to obtain modified data.

[0031] In an embodiment, based on analysis, the conditional execution control circuitry can determine that processing the first layer by one processing device (e.g., a processor within a hardware accelerator) will consume more power than processing the same network layer by another processing device (e.g., dedicated logic circuitry that may have been optimized to perform specific types of operations more efficiently than a processor in a hardware accelerator).

[0032] Based on this determination, the conditional execution control circuit can transfer the execution of the process to another processing device and / or another network layer, for example, to achieve arbitrary, non-sequential processing of the network layer. Conversely, if the conditional execution control circuit determines that processing a particular layer may require high computational power, for example, to extract different or more detailed features, it can activate additional high-power computing devices. In embodiments, analyzing and processing the input data itself or in combination with other data, such as weight data, to determine whether and how the data is subsequently processed in various network layers (e.g., according to a modified sequence or in a non-sequential manner) can actually cause the neural network to behave differently depending on the input data.

[0033] Advantageously, changes to the processing layer sequence related to the input data can allow for early exit, for example, by prematurely terminating the sequence, thereby saving computational resources. Additionally, in embodiments, for example, when controlling multiple networks simultaneously, computations can be skipped; for instance, in image processing applications, after determining that a specific contour (e.g., eyes) has not been detected, there is no need to compute other contours (e.g., faces), which further saves computational resources.

[0034] In embodiments, a multi-stage process can be used. In the first stage, for example, a probability score generated from or at the output of a network layer is used to determine whether an object has been detected in the image, in order to decide whether to continue processing to a layer where the task is to detect finer contours. In audio applications, conditional execution control circuitry can evaluate the output data from the network layer to determine, for example, whether the audio signal contains sufficient spectral energy in the first stage, in order to decide whether to continue processing other network layers aimed at detecting speech in the second stage. Additionally, in embodiments, before deciding which path a branch circuit should take, the input data can first be evaluated to detect one or more environmental features, such as noise.

[0035] In embodiments, data-dependent decisions can be made at hardware boundaries, such as processor inputs, or at logical boundaries. Additionally, decisions can be made at channel boundaries or layer boundaries. It should be understood that at least some of the input data to a network layer in a neural network can be the output of the preceding network layer.

[0036] In this embodiment, decisions can be made by evaluating input data in-situ (i.e., before it is transmitted to a processing device or written to memory). For example, a decision can be made between the time when the output is transmitted or written and the time when the next batch of input data is received. In other words, unlike software applications, decisions can be made without data being passed to and from memory, thus significantly saving time and computational resources. Furthermore, in this embodiment, decisions can advantageously be made within a single processing cycle.

[0037] In embodiments, analyzing input data may include using statistical tools and comparators to detect trends or anomalies in the input data when compared to previous data, such as trends or anomalies that may have been learned during the training phase of a neural network. Exemplary statistical calculations may include evaluating probability scores, ReLU functions, or any other statistical functions. For example, at least a portion of the input data may be compared to a mean, maximum, range of values, address range, address region, or other metrics that may represent the outcome (such as intermediate or cumulative results, statistical sampling, mini-batch, or time-varying results). In embodiments, this may be implemented using comparator circuitry, a state machine, or logic that performs multiple comparisons based on the input data, such logic that compares current data with previous data to determine trends or anomalies, for example, to automatically determine when and how to modify (or terminate) the processing sequence and where to write the output data.

[0038] In image processing, once a network has identified an object to a certain extent, one of the layers of a different network can be used to process the input data to perform a task to help identify the relevant object. Conversely, in response to the failure to detect, for example, multiple contours in the image data associated with the scene, the process of identifying a specific object can be stopped together and / or a new set of input data can be loaded, for example, from a different source. In embodiments, different kernels can be used to perform weight adjustment, or different networks can be used, or the same network can be used with different kernels to detect different features. Advantageously, such embodiments can also be performed without writing the input data to a memory device.

[0039] In embodiments, to reduce the amount of unnecessary computation and thus conserve computational resources, the modified processing sequence may include stopping processing at least one layer in the processing layer sequence based on stopping conditions (such as conditional expressions including values, ranges of values ​​at specific addresses, etc.), for example, in response to determining that the expected accuracy gain is unlikely to be met. In embodiments, processing of one or more network layers may be delayed instead of using stopping conditions to terminate the network layers. Other exemplary decisions include restarting or rerouting the associated process. In embodiments, addressing may involve setting bits to instruct the hardware accelerator to resume operation at a specific address in memory (e.g., an address corresponding to a different branch of the network layer).

[0040] It should be understood that, in the embodiments, the results from any number of layers / circuits can be combined in any suitable manner. For example, for a given application, one or more circuits may be used to generate a different type of data than another number of circuits. Furthermore, any number of layers / circuits may be disabled individually or in combination, depending, for example, on the multiple inputs received by system 200 for processing at any given time for a particular application.

[0041] In embodiments, the conditional execution control circuitry can, for example, select any number of layers / circuits based on user-programmable register values ​​(which may be associated with one or more performance metrics) that should process specific neural network layers within the CNN to produce the desired results. For instance, the conditional execution control circuitry can use a preprocessor to consider parameters such as capacity (e.g., throughput and availability of each circuit), network layer information, information about the data being processed, and configuration information (e.g., input or output data size).

[0042] The preprocessor can further use at least some parameters to estimate the duration of a specific process in various possible combinations, such as to assess power requirements or whether one circuit or circuit should be used more frequently for a specific task than another. For example, the preprocessor can determine that operating three circuits once is computationally less costly than operating a specific processor three times to achieve the same result. In response, the preprocessor can dynamically schedule layers / circuits and determine their priorities accordingly, for example, to reduce overall system power consumption. In embodiments, for example, where lower computing power is acceptable, the preprocessor can select one circuit to perform a dedicated task and stop other circuits to save power.

[0043] In embodiments, the selected circuitry may have different capabilities than unselected circuitry. For example, relatively small circuitry or dedicated low-power logic carrying less computational overhead can be selected layer by layer to perform specific operations, such as arithmetic calculations, more efficiently, faster, and with less power. Furthermore, the circuitry can be optimized to process specific types of input data; the optimized circuitry may include predetermined configuration parameters that reduce the configuration computation time.

[0044] In this embodiment, the preprocessing circuitry can determine under what circumstances which circuits / layers should be selected, including when to switch to a different configuration or previously used circuitry (e.g., a general-purpose processor). Overall, dynamic circuit selection advantageously reduces power consumption and improves computational efficiency performance.

[0045] In embodiments, dynamic circuit selection may use comparator circuitry that may include any logic elements known in the art or as described below. Figure 4 The conditional execution control circuitry discussed is used to implement this. In an embodiment, the preprocessing circuitry can dynamically select any number of configurations or network parameters (e.g., parameters associated with the outputs from network layers of a neural network) to match characteristics (e.g., performance metrics for a given circuit / layer).

[0046] It should be understood that Figure 2 The data-dependent neural network processing 200 is not limited to the structural details shown herein or described in the accompanying text. Those skilled in the art will understand that suitable systems may include different or additional elements and / or connections, including storage devices.

[0047] Figure 3This is a flowchart of dynamic data-dependent neural network processing according to various embodiments of this disclosure. In one or more embodiments, process 300 may begin at step 302, whereby, at a controller, input data to be processed by a first device, for example, in a first layer of a processing layer sequence in the neural network, using a first set of parameters. In embodiments, at least some of the input data may include data already output by previous network layers (i.e., network layers preceding the first layer in the neural network).

[0048] At step 304, the input data can be analyzed to determine at least one of the following: (1) whether modified input data is obtained; (2) whether processing the input data or the modified input data in the second layer will save at least one computational resource; or (3) whether a second set of parameters different from the first set of parameters is applied to the input data or the modified input data. In an embodiment, for example, after delaying processing in the first layer, the second layer may be processed by a second device different from the first device.

[0049] At step 306, in response to a determination that can be made at a hardware or logical boundary (e.g., a channel or layer boundary), the processing layer sequence can be modified to obtain a modified sequence. In an embodiment, the modified sequence may include a stopping condition that terminates processing at least one processing layer in the sequence, for example, to conserve computational resources.

[0050] In an embodiment, this determination may be made by the controller in a single processing cycle using a state machine or logic circuit implemented in the CNN accelerator, prior to, for example, analyzing the input data without writing it to a memory device, and further analyzing a first set of parameters and / or a second set of parameters, for example, by comparing the input data with an average, region, address range, or value range. In an embodiment, the control circuitry may determine trends and / or anomalies associated with the input data and that may have already been learned during the training phase.

[0051] Finally, at step 308, the modified input data can be processed according to the modified sequence so that the neural network behaves differently depending on the input data, thereby saving computational resources. Those skilled in the art will recognize that: (1) specific steps herein may be performed optionally; (2) the steps may not be limited to the specific order set forth herein; (3) specific steps may be performed in different orders; and (4) specific steps may be performed simultaneously.

[0052] Figure 4This is a block diagram illustrating exemplary conditional execution control circuitry according to various embodiments of this disclosure. In embodiments, one or more functions can be implemented by conditional execution control circuitry that can be implemented in a processor (such as a neural network hardware accelerator) and configured via an advanced peripheral bus (APB). In embodiments, the conditional execution control circuitry can be used to monitor CNN output channel write addresses and / or data, and, for example, provide status and / or control for future CNN execution layer by layer based on comparison results and selected functions.

[0053] In an embodiment, execution control may include: (1) continuing sequential layer execution, (2) forcing a branch to a specified layer, and (3) stopping execution, for example, to prevent certain types of input data from being used or processed, thereby reducing unnecessary and computationally costly data movement (such as data movement requiring read, write, buffer, and data storage operations) to significantly reduce power consumption. This execution control may be enabled, for example at a layer boundary, after a selected condition is detected. As discussed in more detail below, the conditional execution control circuitry may include address matching circuitry, data matching circuitry, and counting circuitry.

[0054] In this embodiment, the address matching circuit and the data matching circuit can be cross-coupled before each counting circuit to create a finer matching function. Additionally, the outputs of each matching circuit following the counter can be used independently or in combination to control branching and stopping. In this embodiment, the matching conditions and the counting / accumulation function can be controlled using registers accessible via an APB interface, which can also be used to read counting / accumulation results and control signals.

[0055] like Figure 4 The described conditional execution control circuit may include an address matching circuit, a data matching circuit, a counter / accumulator / maximum value detection logic, a count enable matching detection logic, a count matching logic, and a final matching logic. The counter / accumulator / maximum value detection logic includes a data matching counter, a data accumulator, and a maximum value register, and the final matching logic includes a branch circuit and a stop circuit.

[0056] In this embodiment, common designs can be used to match CNN output channel write addresses, data, or resulting counter / accumulator results. In this embodiment, three APB accessible register bits determine the selected matching function, and two APB accessible registers determine the matching value. Each matching circuit can be enabled when the selected layer is active and the global matching enable signal is active. For data matching functions, one or more byte values ​​can be compared. For example, if one byte is selected, the output of one sub-circuit can be used. If two bytes are selected, the outputs of two sub-circuits, such as zero and one, can be used. If three bytes are used, the outputs of three sub-circuits can be compared. Conversely, if no bytes are enabled to compare outputs, these bits can be forced to zero values.

[0057] In this embodiment, the count enable matching detection logic determines the conditions for enabling the counter / accumulator. Figure 4 In this configuration, two APB-accessible register bits determine the selected matching function. In an embodiment, the counter / accumulator / maximum value detection logic can extend the functionality of the matching circuit by adding a matching function to data-dependent content. The counting or accumulation result can be read via the APB, and the selected result can be evaluated using dedicated matching logic. Zero, one, or more functions can be enabled simultaneously, from which one function is selected for comparison.

[0058] In this embodiment, the data match counter can allow the number of cycles for accumulating a match condition. When a match condition is detected, the counter can increment once per clock cycle and can be cleared via the APB interface (e.g., by writing zero to a register). The binary counter can be, for example, 32 bits wide. Based on the function selection logic identified in the address, data, and counter match circuitry, the resulting accumulator value can be selected and compared with a user-programmable register value.

[0059] In this embodiment, the data accumulator can sum the data values ​​present during the matching period to generate an accumulated count value for all matching periods. The counter can be cleared via the APB interface by writing zero to a register, and the existing CNN data value can be added to the accumulator each clock cycle when a match condition is detected. Similar to the data match counter, this binary counter can also be, for example, 32 bits wide, and the resulting accumulator value can be selected and compared with a user-programmable register value based on function selection logic identifying the address, data, and count match circuitry.

[0060] In this embodiment, the maximum value register can load the data value present during the matching cycle when the input data value is greater than the value currently stored in the register. Additionally, the address accompanying this data value can be loaded into the corresponding address register. The counter can be cleared via the APB interface by writing zero to the register, and the existing CNN data value can be loaded into the accumulator each clock cycle when a matching condition is detected and the input value is greater than the current register value. The data register can be, for example, 32 bits wide, and the accompanying address register can be, for example, 21 bits wide. Based on the function selection logic identified by the address, data, and counter matching circuitry, the resulting registered data value can be selected and compared with a user-programmable register value. One of the values ​​described above can be selected and passed to a user-definable comparison function.

[0061] In an embodiment, the final matching logic can determine whether a matching condition has been met, for example, to enable a stop or branch at the end of the layer where a match was detected. The function can select any combination of address, data, or counter / accumulator matching outputs. In an embodiment, if any matching circuitry is disabled, it is effectively excluded from the matching logic. If a matching type—address, data, or counter / accumulator—is enabled, the matching signal from the associated matching logic can be logically ANDed with the results of other active matching types. If all enabled matching results are true, the overall match is true and can be fed forward to the branch or stop logic.

[0062] If the stop or branch function is enabled, and all conditions are met, the associated action can occur at the end of the current network layer. If both the stop and branch options are enabled, and all enabled matching conditions are met, the stop can take precedence. In embodiments including two or more independent matching circuits, each matching circuit can be configured independently and operate in parallel independently. Within a single network, any number of branch conditions can be configured and triggered. In embodiments where one or more branch or matching circuits are configured, priority coding or configurable priorities can be used to select the branch or stop condition to be executed.

[0063] Figure 5 A simplified block diagram of an information processing system (or computing system) according to embodiments of this disclosure is depicted. It will be understood that the illustrated functions of system 500 can operate to support various embodiments of the computing system; however, it should be understood that the computing system can be configured differently and include different components, including those having, etc. Figure 5 The text describes fewer or more components.

[0064] like Figure 5As shown, the computing system 500 includes one or more CPUs 501, which provide computing resources and control the computer. The CPUs 501 may be implemented using microprocessors or the like, and may also include one or more graphics processing units 519 and / or floating-point coprocessors for mathematical calculations. The system 500 may also include system memory 502, which may take the form of random access memory (RAM), read-only memory (ROM), or both.

[0065] Multiple controllers and peripherals can also be provided, such as Figure 5 As shown. Input controller 503 represents an interface to various input devices 504, such as a keyboard, mouse, touchscreen, and / or stylus. Computing system 500 may also include a storage controller 507 for interfacing with one or more storage devices 508, each of which includes a storage medium such as magnetic tape or disk, or an optical medium that can be used to record instruction programs for operating systems, utilities, and applications, which may include embodiments of programs implementing various aspects of this disclosure. Multiple storage devices 506 may also be used to store processed data or data to be processed according to this disclosure. System 500 may also include a display controller 509 for providing an interface to a display device 511, which may be a cathode ray tube (CRT), thin-film transistor (TFT) display, organic light-emitting diode, electroluminescent panel, plasma panel, or other type of display. Computing system 500 may also include one or more peripheral device controllers or interfaces 505 for one or more peripheral devices 508. Examples of peripheral devices may include one or more printers, scanners, input devices, output devices, sensors, etc. The communication controller 514 can interface with one or more communication devices 515, enabling the system 500 to connect to remote devices via any of various networks, including the Internet, cloud resources (e.g., Ethernet cloud, Ethernet Fibre Channel (FCoE) / Data Center Bridge (DCB) cloud, local area network (LAN), wide area network (WAN), storage area network (SAN)), or via any suitable electromagnetic carrier signal (including infrared signals). Processed data and / or data to be processed according to this disclosure can be communicated via the communication device 515.

[0066] In the system shown, all major system components can be connected to bus 516, which can represent more than one physical bus. However, the individual system components may or may not be physically close to each other. For example, input and / or output data can be remotely transferred from one physical location to another. Additionally, programs implementing various aspects of this disclosure can be accessed from a remote location (e.g., a server) via a network. Such data and / or programs can be transmitted via any of a variety of machine-readable media, including but not limited to: magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured to store or store and execute program code, such as ASICs, programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices.

[0067] The aspects of this disclosure can be implemented using instructions coded on one or more non-transitory computer-readable media for use with one or more processors or processing units to enable the execution of steps. It should be noted that the one or more non-transitory computer-readable media should include both volatile and non-volatile memory. It should be noted that alternative implementations are possible, including hardware implementations or software / hardware implementations. The functionality of the hardware implementation can be implemented using ASICs (multiple) ASICs, programmable arrays, digital signal processing circuit systems, etc. Therefore, the term "means" in any claim is intended to cover both software and hardware implementations. Similarly, the term "one or more computer-readable media" as used herein includes software and / or hardware, or a combination thereof, having a program of instructions embodied thereon. In consideration of these alternative implementations, it will be understood that the accompanying drawings and description provide functional information that a person skilled in the art would need to write program code (i.e., software) and / or manufacture circuitry (i.e., hardware) to perform the desired processing.

[0068] It should be noted that embodiments of this disclosure may further relate to computer products having a non-transitory tangible computer-readable medium having computer code thereon for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may belong to a class well-known or available to those skilled in the art. Examples of tangible computer-readable media include, but are not limited to: magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured for storing or for storing and executing program code, such as ASICs, PLDs, flash memory devices, and ROM and RAM devices. Examples of computer code include machine code generated by a compiler and files containing higher-order code executed by a computer using an interpreter. Embodiments of this disclosure may be implemented, in whole or in part, as machine-executable instructions that may reside in a program module executed by a processing device. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In a distributed computing environment, program modules may reside physically in a local, remote, or both environment.

[0069] Those skilled in the art will recognize that no computing system or programming language is essential to the practice of this disclosure. They will also recognize that the various elements described above may be physically and / or functionally divided into multiple sub-modules or combined together.

[0070] Those skilled in the art will understand that the foregoing examples and embodiments are exemplary and not limited to the scope of this disclosure. All arrangements, enhancements, equivalents, combinations, and modifications thereof that will be apparent to those skilled in the art upon reading this specification and studying the accompanying drawings are intended to be included within the true spirit and scope of this disclosure. It should also be noted that the elements of any claim can be arranged in different ways, including having multiple dependencies, configurations, and combinations.

Claims

1. A method for processing dynamic data-dependent neural networks, comprising: At the controller, input data to be processed in the first layer of the processing layer sequence of the neural network using a first set of parameters is received, the first layer being processed by a first device; In the controller, the input data is analyzed to make a determination about at least one of the following: Whether to acquire modified input data that will improve the computational efficiency of at least one computing resource; Whether processing the input data or the modified input data at the second layer will improve the computational efficiency of the at least one computing resource; or Whether applying a second set of parameters, different from the first set of parameters, to at least one of the input data or the modified input data would improve the computational efficiency of the at least one computing resource; In response to this determination, the sequence of processing layers in the neural network is modified to obtain a modified sequence using each processing layer in the processing layer; as well as Process the input data or at least one of the modified input data according to the modified sequence so that the neural network behaves differently depending on the input data, thereby improving the computational efficiency of the at least one computing resource.

2. The method of claim 1, wherein, This determination is made at least at one of the hardware or logical boundaries, including either a channel boundary or a layer boundary.

3. The method of claim 1, wherein, The analysis of the input data is performed in the controller without writing the input data to the memory device.

4. A non-transitory machine-readable storage medium comprising instructions that, when executed by a circuit of a computing device, cause the circuit to perform a dynamic data-dependent neural network processing method, comprising the following operations: Receive input data to be processed in a layer of a neural network processing layer sequence using a first set of parameters, the layer being processed by a first device; Analyze the input data to make a determination about at least one of the following: Whether to acquire modified input data that will improve the computational efficiency of at least one computing resource; Whether processing the input data or at least one of the modified input data in another layer of the neural network's processing layer sequence will improve the computational efficiency of the at least one computing resource; as well as Whether applying a second set of parameters, different from the first set of parameters, to at least one of the input data or the modified input data would improve the computational efficiency of the at least one computing resource; In response to this determination, the sequence of processing layers in the neural network is modified to obtain a modified sequence using each processing layer in the processing layer; as well as Process the input data or at least one of the modified input data according to the modified sequence so that the neural network behaves differently depending on the input data, thereby improving the computational efficiency of the at least one computing resource.

5. The non-transitory machine-readable storage medium of claim 4, wherein, Perform analysis of the input data without writing it to a memory device.

6. The non-transitory machine-readable storage medium of claim 4, wherein, This determination is made at least at one of the channel boundaries or layer boundaries.

7. A dynamic data-dependent neural network processing system, comprising: A source memory for storing input data to be processed in the first layer of a sequence of processing layers in a neural network using a first set of parameters, the first layer being processed by a first device; Conditional execution control circuitry, coupled to the source memory, includes: Preprocessing circuitry; and Comparator circuit; The preprocessing circuit analyzes the input data to make a determination regarding at least one of the following: Whether to acquire modified input data that will improve the computational efficiency of at least one computing resource; Whether processing the input data at a second layer or at least the modified input data will improve the computational efficiency of the at least one computing resource; or Whether applying a second set of parameters, different from the first set of parameters, to at least one of the input data or the modified input data would improve the computational efficiency of the at least one computing resource; In response to the determination, the preprocessing circuit modifies the sequence of processing layers of the neural network to obtain a modified sequence using each processing layer, and processes at least one of the input data and the modified input data according to the modified sequence; and A register that stores at least the first set of parameters and the second set of parameters.

8. The system of claim 7, wherein, The comparator circuit compares at least a portion of the input data with at least one of an average value, a region, an address range, or a value range.

9. The system of claim 7, wherein, The preprocessing circuit analyzes the input data without writing it to a memory device.

Images