Device and method for performing machine learning

Abstract

Embodiments of the present disclosure provide a method and system for performing machine learning. The method may include: receiving training data; and training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating weights associated with connections of the machine learning model, wherein each connection has a corresponding weight, removing one or more connections from the machine learning model whose weights do not meet a threshold condition; and updating the machine learning model after removing the connection.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims the benefit of priority to U.S. Provisional Application No. 62 / 663,955, filed on April 27, 2018, the entire contents of which are incorporated herein by reference. Background Art

[0003] As machine learning programs have evolved, the dimensions of machine learning models have increased significantly to improve model accuracy. However, deep machine learning models consume a lot of storage space, memory bandwidth, energy consumption, and computing resources during model training or inference. These issues make it difficult to deploy deep machine learning models on mobile and embedded devices.

[0004] Embodiments of the present disclosure solve the above-mentioned problems by providing methods and systems for performing machine learning. Summary of the invention

[0005] An embodiment of the present disclosure provides a computer-implemented method. The method may include: receiving training data; training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating weights associated with connections of the machine learning model, wherein each of the connections has a corresponding weight; removing one or more connections from the machine learning model whose weights do not meet a threshold condition, and updating the machine learning model after removing the connections.

[0006] Embodiments of the present disclosure also provide a computer-implemented method for performing machine learning. The method may include: receiving input data to be provided to a machine learning model, the machine learning model including multiple layers, each layer having one or more nodes, the one or more nodes having one or more connections with nodes from another layer of the machine learning model; processing the input data through a filter; and providing the processed input data to the machine learning model.

[0007] Embodiments of the present disclosure also provide a non-transitory computer-readable medium storing an instruction set, which can be executed by at least one processor of a computer system to cause the computer system to perform a method for simplifying a machine learning model. The method may include: receiving training data; and training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating weights associated with connections of the machine learning model, wherein each connection has a corresponding weight; removing one or more connections from the machine learning model whose weights do not meet a threshold condition; and updating the machine learning model after removing the connection.

[0008] An embodiment of the present disclosure also provides a non-temporary computer-readable medium storing an instruction set, which can be executed by at least one processor of a computer system to enable the computer system to execute a method for performing machine learning, which method may include: receiving input data to be provided to a machine learning model, the machine learning model including multiple layers, each layer having one or more nodes, and the one or more nodes having one or more connections with nodes from another layer of the machine learning model; processing the input data through a filter; and providing the processed input data to the machine learning model.

[0009] The embodiment of the present disclosure also provides a computer system. The computer system may include: a memory storing an instruction set; and at least one processor configured to execute the instruction set to enable the system to perform: receiving training data; training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating weights associated with connections of the machine learning model, wherein each connection has a corresponding weight; removing one or more connections whose weights do not meet a threshold condition from the machine learning model; and updating the machine learning model after removing the connection.

[0010] The embodiment of the present disclosure also provides a computer system for performing machine learning. The computer system may include: a memory storing an instruction set; at least one processor configured to execute the instruction set to enable the system to perform: receiving input data to be provided to a machine learning model, the machine learning model including multiple layers, each layer having one or more nodes, the one or more nodes having one or more connections with nodes from another layer of the machine learning model; processing the input data through a filter; and providing the processed input data to the machine learning model.

[0011] Additional features and advantages of the disclosed embodiments will be partially set forth in the following description, and in part will become apparent from the description, or may be learned through the implementation of the embodiments. The features and advantages of the disclosed embodiments may be realized and obtained through the elements and combinations set forth in the claims.

[0012] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings.The various features illustrated in the drawings are not drawn to scale.

[0014] Figure 1 A neural network processing architecture according to an embodiment of the present disclosure is shown.

[0015] Figure 2A -F shows a schematic diagram of a machine learning model according to an embodiment of the present disclosure.

[0016] Figure 3 A schematic diagram of simplified input data according to an embodiment of the present disclosure is shown.

[0017] Figure 4 A computer-implemented method according to an embodiment of the present disclosure is shown.

[0018] Figure 5 A computer-implemented method for performing machine learning according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] Certain aspects of the disclosure are described in further detail below.In the event of a conflict with a term or definition incorporated by reference, the terms and definitions provided herein shall control.

[0020] The technology described in the present disclosure has one or more of the following technical effects. In some embodiments, the technology described in the present disclosure provides a computer-implemented method, including: receiving training data; training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating the weights associated with the connections of the machine learning model, wherein each connection has a corresponding weight; deleting one or more connections whose connection weights do not meet a threshold condition from the machine learning model; and updating the machine learning model after deleting the connection. In some embodiments, the technology described in the present disclosure simplifies the machine learning model by removing one or more connections of the machine learning model. This can also reduce the storage space, computing resources, and power consumption required for the machine learning model. In some embodiments, the technology described in the present disclosure also generates a filter for evaluating input data to be provided to the machine learning model. The filter can process the input data of the machine learning model at runtime. The filter can reduce the input data by removing a portion of the input data. This allows the storage space, bandwidth, computing resources, and power consumption required for the machine learning model to be reduced. In some embodiments, the filter can also merge guidance information into the input data so that the input data can bypass at least one connection of the machine learning model based on the guidance information without deteriorating the reasoning result. This allows for a reduction in the computational resources and power consumption required by machine learning models.

[0021] As used herein, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, composition, article, or apparatus, element that comprises the following includes more than just those elements but may include other elements not expressly listed or inherent to such process, method, composition, article, or apparatus. The term "exemplary" is used in the sense of "example" rather than "ideal."

[0022] Figure 1 FIG. 1 shows an exemplary neural network processing architecture 100 according to an embodiment of the present disclosure. Figure 1 The architecture 100 may include a chip communication system 102, a host memory 104, a memory controller 106, a direct memory access (DMA) unit 108, a joint test action group (JTAG) / test access completion (TAP) controller 110, a peripheral interface 112, a bus 114, a global memory 116, etc. It should be understood that the chip communication system 102 may perform algorithmic operations (e.g., machine learning operations) based on the transferred data.

[0023] The chip communication system 102 may include a global manager 1022 and a plurality of cores 1024. The global manager 1022 may include at least one task manager to coordinate with one or more cores 1024. Each task manager may be associated with an array of cores 1024 that provide synaptic / neuron circuits for a neural network. For example, Figure 1 The top layer of the processing element of the core may provide circuits representing the input layer of the neural network, while the second layer of the core may provide circuits representing the hidden layer of the neural network. Therefore, the chip communication system 102 may also be referred to as a neural network processing unit (NPU). Figure 1 As shown, the global manager 1022 may include two task managers to coordinate with the two core arrays.

[0024] Core 1024 may include one or more processing elements, each processing element including a single instruction, multiple data (S1MD) architecture, the architecture including one or more multiple processing units configured to perform one or more operations (e.g., multiplication, addition, multiply-accumulate, etc.) based on instructions received from global manager 1022. In order to perform operations on the transmitted data packets, core 1024 may include one or more processing elements for processing information in the data packets. Each processing element may include any number of processing units. In some embodiments, core 1024 may be considered as a tile or the like.

[0025] The host memory 104 may be an off-chip memory, such as a memory of a host CPU. For example, the host memory 104 may be a DDR memory (e.g., DDR SDRAM), etc. Compared to an on-chip memory integrated in one or more processors that acts as a high-level cache, the host memory 104 may be configured to store a large amount of data at a lower access speed.

[0026] The memory controller 106 may manage the reading and writing of data to a specific memory block within the global memory 116 having on-chip memory blocks (e.g., 4 blocks of 8GB second-generation high bandwidth memory (HBM2)) as main memory. For example, the memory controller 106 may manage the reading and writing of data from outside the chip communication system 102 (e.g., from the DMA unit 108 or a DMA unit corresponding to another NPU) or from inside the chip communication system 102 (e.g., from a local memory in the 2D grid control core 1024 controlled by the task manager of the global manager 1022). In addition, although in Figure 1 One memory controller is shown in FIG. 1 , but it should be understood that more than one memory controller may be provided in architecture 100. For example, there may be one memory controller for each memory block (eg, HBM2) within global memory 116.

[0027] The memory controller 106 may generate addresses for the memory and initiate memory read or write cycles. The memory controller 106 may include several hardware registers that may be written and read by one or more processors. These registers may include a memory address register, a byte count register, one or more control registers, and other types of registers. These registers may specify a source, a destination, a transfer direction (read from an input / output (I / O) device or write to an I / O device), the size of a transfer unit, the number of bytes to be transferred in a burst, or some combination of other typical functions of a memory controller.

[0028] DMA unit 108 may assist in transferring data between host memory 104 and global memory 116. Additionally, DMA unit 108 may assist in transferring data between multiple NPUs (e.g., NPU 100). DMA unit 108 may allow off-chip devices to access on-chip and off-chip memory without causing CPU interruptions. Thus, DMA unit 108 may also generate memory addresses and initiate memory read or write cycles. DMA unit 108 may also include several hardware registers that may be written to and read from one or more processors, including a memory address register, a byte count register, one or more control registers, and other types of registers. These registers may specify some combination of source, destination, transfer direction (read from or write to an input / output (I / O) device), transfer unit size, or the number of transfers in a burst. It should be understood that architecture 100 may include a second DMA unit that may be used to transfer data between other neural network processing architectures to allow multiple neural network processing architectures to communicate directly without involving a host CPU.

[0029] The JTAG / TAP controller 110 may designate a dedicated debug port that implements a serial communication interface (e.g., a JTAG interface) to enable low-overhead access to the NPU without directly accessing the system address and data bus from the outside. The JTAG / TA controller 110 may also have an on-chip test access interface (e.g., a TAP interface) that implements a protocol for accessing a set of test registers that present chip logic levels and device capabilities of various parts.

[0030] Peripheral interface 112 (eg, PCIe interface), if present, acts as (typically) an inter-chip bus providing communications between architecture 100 and other devices.

[0031] Buses 114 include intra-chip buses and inter-chip buses. The intra-chip buses connect all the internal components to each other, which is required by the system architecture. Although not all components are connected to every other component, all components do have some connection to other components with which they need to communicate. Inter-chip buses connect the NPU to other devices, such as off-chip memory or peripherals. Typically, if a peripheral interface 112 (e.g., an inter-chip bus) is present, bus 114 is only related to the intra-chip bus, although in some implementations, bus 114 may still be related to specialized inter-bus communications.

[0032] The chip communication system 102 may be configured to perform operations based on a neural network.

[0033] The architecture 100 may also include a host unit 120. The host unit 120 may be one or more processing units (e.g., an X86 central processing unit). In some embodiments, a host system having the host unit 120 and the host memory 104 may include a compiler (not shown). A compiler is a program or computer software that converts computer code written in a programming language into NPU instructions to create an executable program. In a machine application, the compiler may perform various operations, such as preprocessing, lexical analysis, parsing, semantic analysis, conversion of input programs to intermediate representations, code optimization, and code generation or a combination of the foregoing operations. In addition to compiling programs, the host system may also analyze input data to be processed by the chip communication system 102, so that features of the input data may be extracted.

[0034] In some embodiments, the compiler that generates the NPU instructions may be on the host system, which pushes the commands to the chip communication system 102. Based on these commands, each task manager may assign any number of tasks to one or more cores (e.g., core 1024). Certain commands may instruct the DMA unit 108 to load instructions (generated by the compiler) and data (e.g., input data) from the host memory 104 into the global memory 116. The loaded instructions may then be assigned to each core corresponding to the task to which the instructions are assigned, and one or more cores may process the instructions.

[0035] Figure 2A -F shows a schematic diagram of a machine learning model according to an embodiment of the present disclosure. As an example of a machine learning model, a neural network may include multiple layers, and each layer may include multiple nodes (also called artificial neurons). Connections between multiple nodes may be established by training the neural network with training data. Figures 2A-2FIn the figure, the bold arrows can represent many connections between layers. The training of a neural network can also be called machine learning, and a trained neural network can also be called a machine learning model. Each connection can be assigned a weight comprising a number of bits (e.g., 32 bits). The weight of a connection can increase or decrease the signal strength at the connection. For example, Figure 2A As shown, the weight of the connection 2011 between the two nodes 221 and 231 may be zero. Therefore, the signal passing through the connection 2011 is reduced to zero. In other words, the signal cannot pass through the connection 2011, and the two nodes 221 and 231 are not connected.

[0036] Figure 2A A schematic diagram of an exemplary neural network 201 is shown. Neural network 201 may include four layers (e.g., layers 210, 220, 230, and 240), each layer including multiple nodes (e.g., nodes 211, 212, 213, etc.). In some embodiments of the present disclosure, neural network 201 may be an initial neural network for training. The initial neural network is a neural network with default parameters (e.g., connection weights). The parameters of the neural network may be related to the connections between nodes and the weights of the connections. In some embodiments, neural network 201 may be a trained neural network obtained by training the initial neural network based on training data, so that multiple connections may be established between multiple nodes of the layer.

[0037] According to an embodiment of the present disclosure, multiple connections in the neural network 201 may be simplified. In some embodiments, the simplification may be performed on the neural network 201 during training. The simplification during training may also be referred to as static time simplification.

[0038] In some embodiments, simplification may remove a connection between two nodes during training. Figure 2B FIG. 2 shows a schematic diagram of a neural network 202 after pruning, which involves the removal of some connections (also called synapses) according to an embodiment of the present disclosure. Figure 2B In the Neural Network 201, Figure 2A 2011. As described above, each connection of the trained neural network is assigned a connection weight. Then, it can be determined whether the connection weight meets the threshold. For example, when the connection weight is greater than or equal to the threshold, the connection weight meets the threshold and the corresponding connection is maintained. Otherwise, when the connection weight is less than the threshold, the connection weight does not meet the threshold. If the connection weight of the connection does not meet the threshold, the corresponding connection can be deleted. In some embodiments, the connection can be deleted by setting the connection weight of the connection to zero. In some embodiments, the connection can also be removed by deleting the connection from the neural network.

[0039] In some embodiments, in the trained neural network, the connection weight of at least one connection to be removed can be set to zero, and the trained neural network can be updated for accuracy evaluation. If the accuracy of the updated neural network is satisfactory, the at least one connection can be finally deleted from the neural network. On the other hand, if the accuracy of the updated neural network is not satisfactory, the threshold for deleting the connection can be adjusted. It can be understood that the threshold can be adjusted according to different neural networks and different factors (e.g., accuracy, energy consumption, etc.) of the application neural network.

[0040] Based on the remaining connections, the neural network 201 can be updated to a final neural network (e.g., Figure 2B Neural network 202 shown). It should be understood that the weights of the remaining connections may also be updated. Since the original weights may be multiple bits (e.g., 32 bits), each updated weight may still include that number of bits.

[0041] Figure 2C 2 shows a schematic diagram of a neural network 203 after removing layers according to an embodiment of the present disclosure. In some embodiments, layers of the neural network can be deleted during the training process to further simplify the neural network. For example, Figure 2C , layer 230 in 2A or 2B has been removed from the neural network.

[0042] In some embodiments, the layer weights of each layer of the neural network can be determined based on the connections of the neural network. The layer weight of a layer can be related to the connection weights of the nodes of the layer. For example, the layer weight can be the sum of the connection weights of the nodes of the layer. Then, it can be determined whether the layer weight of the layer meets a threshold. For example, when the layer weight is greater than or equal to the threshold, the layer weight meets the threshold. Otherwise, when the layer weight is less than the threshold, the layer weight does not meet the threshold. If the layer weight of the connection does not meet the threshold, the layer (e.g., layer 230) can be removed. In some embodiments, the layer can be "removed" by setting the connection weights of all connections related to the layer to zero. In some embodiments, the layer can also be removed by deleting the layer in the neural network.

[0043] Based on the remaining layers, the neural network 201 can be updated to a final neural network (e.g., Figure 2C It should be understood that the connections and connection weights in the remaining layers may be updated.

[0044] In addition to modifying the neural network based on the training data, the training data can be modified before it is used to train the neural network. In some embodiments, a portion of the training data can be removed. For example, the training data can include multiple dimensions (e.g., 10 dimensions). Before the training data is input for training the neural network, at least one dimension of the training data can be removed.

[0045] In some embodiments, time information can be related to neural networks such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs). The removed portion of the training data can be related to the time domain. It is understood that these neural networks can process data sequences. Therefore, the stimulation of the neural network can come not only from new input data at time T, but also from historical information at time T-1. Therefore, before the training data is input to the neural network, the first training data associated with time T1 can be removed, and the second training data associated with time T2 can be provided to the neural network for training.

[0046] It should be understood that the connections between nodes and layers are related to the training data. As described above, some connections or layers can be removed because the weights of the connections and layers are less than a given threshold. Therefore, when the neural network is used at runtime, the input data can bypass (or skip) the connection or layer. Moreover, a portion of the training data may have little effect on the results of the neural network and can therefore be ignored by the neural network. Therefore, filters can be generated based on the training of the neural network. In some embodiments, filters can be generated by a learning algorithm. For example, filters can be trained based on at least one of the removed connections, removed layers, remaining connections and layers, and the output of the neural network. In some embodiments, filters can be gates between layers and / or gated neural networks. For example, filters can include a limited number of convolutional layers, average pooling layers, and fully connected layers to output a dimensional vector. The filter can rely only on the output of the previous layer and apply a small number of convolution and pooling operations.

[0047] In some embodiments, the filter can be manually programmed to merge filter rules. In some embodiments, the filter can also include rules such as determined by a software engineer. In other words, the filter can be generated or designed by a software engineer.

[0048] In machine applications, filters can be deployed in Figure 1on the host system. Thus, before the input data is transmitted to the chip communication system 102, the filter can guide the input data to bypass at least one connection of the neural network at runtime. For example, the filter can merge guidance information into the input data, and the guidance information can include a route for the input data to pass through at least one connection of the machine learning model. It should be understood that when all connections of a layer are bypassed, the layer is bypassed. For example, the filter can merge guidance information into the input data so that certain connections or layers can be bypassed according to the guidance information. The filter can also reduce the input data by deleting a portion of the input data. For example, when the input data passes through the filter, the portion (e.g., at least one dimension of the input data) can be removed from the input data. Therefore, in addition to reducing the processing burden of the neural network, the amount of data communication between the host unit 120 and the chip communication system 102 can also be reduced. Therefore, the power consumption and bandwidth usage of the architecture 100 can be reduced.

[0049] Figure 2D-2F An example of neural network 204-206 at runtime according to an embodiment of the present disclosure is shown. In some embodiments, simplification can also be performed on neural network 201 during inference. Simplification during inference can also be referred to as simplification at runtime.

[0050] As described above, the input data may include guidance information generated by the filter. In some embodiments of the present disclosure, the guidance information may bypass at least one layer of the input data. The guidance information may be generated by the filter based on at least one of the input data and the neural network. Therefore, depending on the input data and the neural network, the guidance information generated by the filter may be different.

[0051] Figure 2D Schematic diagram of the runtime neural network 204 is shown. Figure 2D As shown, some input data can bypass layer 230. In some embodiments, neural network 204 can generate accurate results for some input data without layer 230 at runtime. For example, more complex input data can be routed through more layers, while less complex input data can be routed through fewer layers. Therefore, based on the input data, layer 230 can be bypassed at runtime. Therefore, the filter can generate guidance information that guides the input data to bypass layer 230.

[0052] It should be understood that Figure 2D , the connections associated with layer 230 are still provided in neural network 204 and are shown as dashed arrows. Therefore, some other input data can pass through layer 230 via these connections.

[0053] The connection weights associated with layer 230 may be set to zero or any other value. Therefore, in an embodiment of the present disclosure, by bypassing at least one layer (e.g., setting the connection weights of the layer to zero), the computing resources required to process the input data may be reduced.

[0054] In addition to bypassing at least one layer, embodiments of the present disclosure may also bypass at least one connection between nodes at runtime. Figure 2E Schematic diagram of the runtime neural network 205 is shown. Figure 2E As shown, between layers 210 and 220 , neural network 205 includes a first connection between nodes 213 and 222 , a second connection between nodes 212 and 222 , and a third connection between nodes 211 and 223 .

[0055] In some embodiments, neural network 205 may be configured without at least one connection (e.g., Figure 2E Thus, for example, the first and second connections may be bypassed at runtime without degrading the results of the neural network 205. Thus, the filter may generate guidance information instructing the input data to bypass the first and second connections.

[0056] In this way, the computational burden of running a neural network can be reduced without modifying the neural network (e.g., permanently removing at least one connection from the neural network). In addition to reducing the computational burden, this also allows for greater flexibility in running the neural network. As described above, filters can be generated by software engineers or based on training. Therefore, different filters can be deployed to machine learning models in different application scenarios.

[0057] It is understood that the at least one connection bypassed at runtime may be different according to different input data. Figure 2F As shown in , based on the guidance information, the second and third connections of the neural network 205 can be bypassed by the second input data that is different from the first input data. As described above, the guidance information can include a route for the input data to pass through at least one connection of the machine learning model. In some embodiments, the routes for different input data can be different, such as Figure 2E and Figure 2F as shown in .

[0058] Therefore, in the embodiments of the present disclosure, by bypassing at least one connection, the computing resources required to process the input data can also be reduced.

[0059] In addition to simplifying the training data at static time, the input data can also be simplified at runtime. Figure 3 A simplified input data schematic diagram according to an embodiment of the present disclosure is shown.

[0060] Reduction of the input data may be performed by removing a portion of the input data before it is sent to the neural network.

[0061] In some embodiments, the input data at runtime may involve multiple dimensions. At least one dimension may be removed from the input data. Figure 3 As shown, input data 301 may include multiple dimensions (e.g., at least one dimension 3011). Before input data 301 is sent to neural network 300 for processing, at least one dimension 3011 may be removed from input data 301 (e.g., by a filter). In some embodiments, the filter may determine the at least one dimension 3011 to be removed based on input data 301 and neural network 300.

[0062] In some embodiments, the removed portion of the input data may be related to the time domain. For example, when the input data is related to time (e.g., a video), the input data may include a sequence of data related to a moment. For example, the data of a video may include a series of frames distributed over a period of time, and the data of each frame corresponds to a moment. In some embodiments, a portion of the input data (e.g., a frame of a video or multiple frames within a given time period) may be bypassed by the neural network with very limited impact on the final result. For example, in the data of a video, the filter may determine to remove frames within the first five seconds from the input data without significantly affecting the final result.

[0063] It should be understood that the portion to be removed from the input data may be different depending on the nature of the input data.

[0064] By removing a portion of the input data, the bandwidth occupied by the machine learning model can be reduced, and the computing resources and power consumption required by the machine learning model can also be reduced.

[0065] Figure 4 A computer-implemented method 400 according to an embodiment of the present disclosure is shown. The method 400 may be implemented by a computer system, for example Figure 1 The computer system may include: a memory storing an instruction set and at least one processor configured to execute the instruction set so that the computer system performs the method 400. The at least one processor may include, for example Figure 1 The computer system may also include a communication interface (e.g., Figure 1 Please refer to the peripheral interface 112). Figure 4 , method 400 may include the following steps.

[0066] In step 402, the computer system may receive training data (eg, via a communication interface). The training data may be stored in a database of the computer system or in another computer system. The training data may be reduced by deleting a portion of the training data.

[0067] In some embodiments, when the training is supervised training, the training data may include a large amount of labeled data. In some embodiments, the training data may include an input vector and a corresponding output vector pair. Therefore, the training data may include multiple dimensions. Each dimension may be associated with a feature of the training data. In some embodiments, the removed portion may be associated with at least one dimension.

[0068] In some embodiments, the training data is related to the time domain. For example, the training data may include first data related to a first moment and second data related to a second moment. And when removing a portion of the training data, the first data associated with the first moment may be removed.

[0069] In step 404, the computer system may train a machine learning model based on the training data. Prior to training the machine learning model, parameters of the machine learning model have not yet been determined. The parameters may, for example, include connectivity of the machine learning model. As an example of a machine learning model, a neural network may include multiple layers, and each layer may include multiple nodes. And multiple nodes may be connected to generate connections between the nodes. Each connection may have a corresponding connection weight, so the connection is weighted. Through training, the connectivity of the machine learning model may be determined based on the training data. For example, connections may be established between nodes, and weights may be determined and assigned to the connections.

[0070] Similarly, the computer system can also determine the layer weights associated with each layer of the machine learning model. In the machine learning model, each layer can have a corresponding layer weight. As described above, the layer weight of each layer of the machine learning model can be determined based on the connection weights.

[0071] In step 406, the computer system may evaluate the weight associated with the connection of the machine learning model. For example, the computer system may determine whether the connection weight of the connection satisfies a threshold condition. In some embodiments, a connection can be removed from the machine learning model without significantly affecting the final result of the machine learning model. Such a connection may be determined as a removable connection. Accordingly, if the removal of a connection results in an inaccurate result of the machine learning model, the connection may be determined as a non-removable connection. The threshold condition may be associated with a connection weight threshold for determining removable connections and non-removable connections. For example, when the connection weight of a connection is greater than or equal to a connection weight threshold, the connection does not satisfy the threshold condition and is evaluated as a non-removable connection. Also for example, when the connection weight of a connection is less than a connection weight threshold, the connection satisfies the threshold condition and is evaluated as a removable connection.

[0072] Similarly, in some embodiments, an entire layer can be removed from the machine learning model without significantly affecting the final results of the machine learning model. Such a layer can be determined as a removable layer. Accordingly, if the removal of the layer can lead to inaccurate results of the machine learning model, the layer can be determined as a non-removable layer. Therefore, the layer threshold condition can be associated with a layer weight threshold for identifying removable layers and non-removable layers, and the computer system can further determine whether the layer weight of the layer satisfies the layer threshold condition. For example, when the layer weight of the layer is greater than or equal to the layer weight threshold, the layer does not satisfy the layer threshold condition and is determined to be a non-removable layer. For another example, when the layer weight of the layer is less than the layer weight threshold, the connection satisfies the layer threshold condition and is determined to be a removable layer.

[0073] In step 408, the computer system may delete one or more connections whose weights do not meet the threshold condition from the machine learning model. In other words, the determined removable connections may be deleted from the machine learning model.

[0074] Similarly, removable layers can also be removed from the machine learning model.

[0075] In step 410, after the connection is removed, the computer system can update the machine learning model. It will be appreciated that after the movable connection is ultimately removed from the machine learning model, the computer system can establish a new connection between the nodes where the original connection has been removed.

[0076] By removing at least one connection or layer from the machine learning model, the complexity of the machine learning model can be reduced. Therefore, the storage space used for the machine learning model and the power consumption used to run the machine learning model can also be reduced.

[0077] In addition, the computer system can also generate a filter for evaluating input data to be provided to the machine learning model. The filter can be used to perform machine learning, which will be referred to below. Figure 5 Further description.

[0078] Figure 5 A computer-implemented method 500 for performing machine learning according to an embodiment of the present disclosure is shown. The method 500 can be implemented by a computer system, such as Figure 1 The computer system may include: a storage storing a set of instructions; and at least one processor configured to execute the set of instructions so that the computer system performs the method 500. The at least one processor may include, for example, a host unit 120 and Figure 1 The computer system may also include a communication interface (eg, Figure 1 Peripheral interface 112). Figure 5 , method 500 may include the following steps.

[0079] In step 502, the computer system may receive input data to be provided to the machine learning model. The machine learning model may include multiple layers, each layer having one or more nodes, the one or more nodes having one or more connections with nodes from another layer of the machine learning model. The machine learning model may be used in a machine learning application. The machine learning application may be executed in the neural network processing architecture 100. The machine learning application may be divided into multiple tasks, and one of the tasks may be executed on the host unit 120, while another may be determined as a machine learning task and executed by a machine learning model of the on-chip communication system. The input data may be related to the machine learning task and may be provided to the machine learning model for processing.

[0080] In step 504, the computer system may process the input data through a filter. After receiving the input data, the filter may be called to process the input data. The filter may be part of a machine learning application, may be executed by the computer system, or may be a function provided by the computer system. Figure 1-3 As discussed, filters can be generated based on the training of a machine learning model, or they can be designed by software engineers.

[0081] Based on the input data, features of the input data can be extracted and compared with the machine learning model. In some embodiments, guidance information for the input data can be generated by a filter. The guidance information can be incorporated into the input data so that the processed input data can be configured to bypass at least one connection based on the guidance information. For example, to bypass a connection, the connection weight of a given input data can be set to zero. It is understood that for another input data, the weight of the connection may not change. In some embodiments, a connection may include a default weight and a bypass weight. The default weight is determined based on training, and the bypass weight is zero. Based on the guidance information, the computer system can determine whether the connection weight is a default weight or a bypass weight. It is understood that the bypass weight can be another value.

[0082] In some embodiments, the guidance information may instruct the input data to bypass the connection associated with the layer, so that the layer can be bypassed by the input data. It should be understood that when a layer is bypassed, the at least one connection may include all connections related to the layer.

[0083] In some embodiments, by using a filter, the computer system can remove a portion of the input data and update the input data accordingly. For example, based on the characteristics of the input data, the computer system can determine that a portion of the input data can be deleted without deteriorating the results of running the machine learning model, and a portion of the input data can be deleted. The input data can be updated and provided to the machine learning model. In some embodiments, the input data can include multiple dimensions, and the removed portion is associated with at least one dimension. In some embodiments, the input data can be related to the time domain. For example, the input data can include first data associated with a first moment and second data associated with a second moment. And when removing a portion of the input data, the first data associated with the first moment can be removed.

[0084] In step 506, the computer system may provide the processed input data to the machine learning model. The machine learning model may generate machine learning results. For example, as an inference engine, the chip communication system 102 (involving one or more accelerators) may use the machine learning model to generate results and send the results back to the host unit 120.

[0085] It should be understood that the filter can be applied to an unsimplified machine learning model. Although the unsimplified machine learning model includes complete connections, the input data can bypass one or more connections at runtime by using a filter. Since the input data bypasses one or more connections at runtime, the computing resources and power consumption used to run the machine learning model can be reduced even though the machine learning model is not simplified. Removing a portion of the input data can not only reduce computing resources and power consumption, but also further reduce the communication load between the host unit 120 and the chip communication system 102.

[0086] The embodiments of the present disclosure also provide a computer program product. The computer program product may include a non-transitory computer-readable storage medium on which computer-readable program instructions are stored, and the instructions may be used to enable a processor to execute the above method.

[0087] The computer-readable storage medium may be a tangible device that can store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device (such as a punch card or a raised structure in a groove on which instructions are recorded), and any suitable combination of the above.

[0088] The computer-readable program instructions for executing the above method can be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​and conventional process programming languages). The computer-readable program instructions can be executed completely on a computer system as an independent software package, or partially on a first computer and partially on a second computer away from the first computer. In the latter case, the second remote computer can be connected to the first computer through any type of network (including a local area network (LAN) or a wide area network (WAN)).

[0089] Computer-readable program instructions may be provided to a processor of a general or special-purpose computer, or other programmable data processing apparatus, to form a machine, so that when the instructions are executed via the processor of the computer or other programmable data processing apparatus, a program for implementing the above-described method is created.

[0090] The flow charts and schematic diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the various embodiments of the specification. In this regard, the boxes in the flow chart or schematic diagram may represent software programs, code segments or code portions, which include one or more executable instructions for implementing a specific function. It should also be noted that in some alternative embodiments, the functions indicated in the boxes may not occur in the order indicated in the figure. For example, depending on the functions involved, the two boxes shown in succession can actually be executed substantially simultaneously, or sometimes these boxes can be executed in reverse order. It should also be noted that each box in the schematic diagram or flow chart, as well as the combination of the boxes in the schematic diagram and the flow chart, can be implemented by a combination of a hardware-based special-purpose system or special-purpose hardware and computer instructions that perform a specified function or action.

[0091] It should be understood that certain features of the specification described in the context of separate embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of the specification described in the context of a single embodiment for brevity may also be provided individually or in any suitable subcombination or as appropriate in any other described embodiment of the invention. Certain features described in the context of various embodiments should not be considered essential features of those embodiments unless the embodiment is inoperable without those elements.

Claims

1. A computer-implemented method comprising: receiving training data, wherein the training data comprises video data; Training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; Evaluate weights associated with connections of the machine learning model, wherein each of the connections has a corresponding weight, the connections include a default weight and a bypass weight, the default weight is determined according to training, and whether the connection weight is the default weight or the bypass weight is selected based on guidance information, wherein the guidance information is generated by a filter based on at least one of input data and a neural network; removing from the machine learning model one or more connections whose weights do not satisfy a threshold condition; and After removing the connection, the machine learning model is updated.

2. The method according to claim 1, wherein: Receiving training data also includes: The training data is reduced by removing a portion of the training data.

3. The method according to claim 2, wherein: The training data includes multiple dimensions, and the removed portion is associated with at least one dimension.

4. The method according to claim 1, further comprising: evaluating weights associated with each layer of the machine learning model, wherein each layer has a corresponding layer weight; Removing one or more layers from the machine learning model whose weights do not satisfy a layer threshold condition; and After removing one or more layers, the machine learning model is updated.

5. The method according to claim 2, wherein: The training data includes first data associated with a first moment and second data associated with a second moment, And reducing the training data by removing a portion of the training data further comprises: The first data associated with the first time instant is removed from the training data.

6. The method according to claim 1, further comprising: Generate filters for evaluating input data to be provided to the machine learning model.

7. A computer-implemented method for performing machine learning, comprising: Receiving input data to be provided to a machine learning model, the machine learning model comprising multiple layers, each layer having one or more nodes, the one or more nodes having one or more connections to nodes from another layer of the machine learning model; Processing the input data through a filter, the filter directing the input data to bypass at least one connection of the neural network at runtime, the connection comprising a default weight and a bypass weight, the default weight being determined according to training, and whether the connection weight is the default weight or the bypass weight being selected based on guidance information, wherein the guidance information is generated by the filter based on at least one of the input data and the neural network; and providing the processed input data to the machine learning model; Wherein, the input data includes video data.

8. The method according to claim 7, wherein: Processing the input data further includes: Remove a portion of the input data; and The input data is updated.

9. The method according to claim 8, wherein: The input data includes a plurality of dimensions, and the removed portion is associated with at least one dimension.

10. The method according to claim 9, wherein: The input data includes first data associated with a first moment and second data associated with a second moment, And removing the portion of the input data further comprises: The first data associated with the first time instant is removed from the input data.

11. The method according to claim 7, wherein: The processed input data further includes guidance information, and the processed input data is further configured to bypass at least one connection based on the guidance information.

12. The method according to claim 11, wherein: The at least one connection includes all connections associated with a layer.

13. The method according to claim 7, wherein: The filter is generated based on the training of the machine learning model.

14. A non-transitory computer readable medium storing a set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method for simplifying a machine learning model, the method comprising: receiving training data, wherein the training data comprises video data; Training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; Evaluate weights associated with connections of the machine learning model, wherein each connection has a corresponding weight, the connection comprises a default weight and a bypass weight, the default weight is determined according to training, and whether the connection weight is the default weight or the bypass weight is selected based on guidance information, wherein the guidance information is generated by a filter based on at least one of input data and a neural network; removing from the machine learning model one or more connections whose weights do not satisfy a threshold condition; and After removing the connection, the machine learning model is updated.

15. The non-transitory computer readable medium of claim 14, wherein: Receiving training data also includes: The training data is reduced by removing a portion of the training data.

16. The non-transitory computer readable medium of claim 14, wherein: The training data includes multiple dimensions, and the removed portion is associated with at least one dimension.

17. The non-transitory computer readable medium of claim 14, wherein: The set of instructions is further executable by at least one processor of the computer system to cause the computer system to execute: evaluating weights associated with each layer of the machine learning model, wherein each layer has a corresponding layer weight; Removing one or more layers from the machine learning model whose weights do not satisfy a layer threshold condition; and After removing one or more layers, the machine learning model is updated.

18. The non-transitory computer readable medium of claim 15, wherein: The training data includes first data associated with a first moment and second data associated with a second moment, and reducing the training data by removing a portion of the training data further includes: The first data associated with the first moment is removed from the training data.

19. The non-transitory computer readable medium of claim 14, wherein: The set of instructions is further executable by at least one processor of the computer system to cause the computer system to execute: Generate filters for evaluating input data to be provided to the machine learning model.

20. A non-transitory computer readable medium storing a set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method for performing machine learning, the method comprising: Receiving input data to be provided to a machine learning model, the machine learning model comprising multiple layers, each layer having one or more nodes having one or more connections to nodes from another layer of the machine learning model; Processing the input data through a filter, the filter directing the input data to bypass at least one connection of the neural network at runtime, the connection comprising a default weight and a bypass weight, the default weight being determined according to training, and whether the connection weight is the default weight or the bypass weight being selected based on guidance information, wherein the guidance information is generated by the filter based on at least one of the input data and the neural network; and providing the processed input data to the machine learning model; Wherein, the input data includes video data.

21. A computer system comprising: A memory storing an instruction set; and at least one processor configured to execute the set of instructions to cause the system to perform: Receiving training data, wherein the training data includes video data: Training a machine learning model based on the training data, wherein the machine learning model includes multiple layers, each layer having one or more nodes, and the one or more nodes have one or more connections with nodes from another layer of the machine learning model; evaluating weights associated with connections of the machine learning model, wherein each connection has a corresponding weight, the connection comprising a default weight and a bypass weight, the default weight being determined according to training, and whether the connection weight is the default weight or the bypass weight being selected based on guidance information, wherein the guidance information is generated by a filter based on at least one of input data and a neural network; removing from the machine learning model one or more connections whose weights do not satisfy a threshold condition; and After removing the connection, the machine learning model is updated.

22. A system for performing machine learning, comprising: A memory storing an instruction set; and at least one processor configured to execute the set of instructions to cause the system to perform: Receiving input data to be provided to a machine learning model, the machine learning model comprising multiple layers, each layer having one or more nodes having one or more connections to nodes from another layer of the machine learning model; Processing the input data through a filter, the filter directing the input data to bypass at least one connection of the neural network at runtime, the connection comprising a default weight and a bypass weight, the default weight being determined according to training, and whether the connection weight is the default weight or the bypass weight being selected based on guidance information, wherein the guidance information is generated by the filter based on at least one of the input data and the neural network; and providing the processed input data to the machine learning model; Wherein, the input data includes video data.

Images