Systems and methods for efficient computation of deep neural network layers with

Abstract

Systems and methods for efficient computations of deep neural network layers with repetitive weights. A method for neural network computing. The method includes receiving a codebook having a plurality of entries, multiplying a respective input of a layer of the neural network by each of the plurality of entries of the codebook, and determining an intermediate value for the respective input of the layer of the neural network based on a sum of each result of multiplying the respective layer of the neural network by each of the plurality of entries of the codebook. The method also includes storing intermediate values associated with respective inputs of a layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values corresponding to other inputs of the layer of the neural network. The method also includes determining each output of a layer of the neural network using the lookup table.

Description

Technical Field

[0001] The present disclosure relates to deep neural networks, and in particular to systems and methods for efficient computation of deep neural network layers with repeated weights. Background Art

[0002] Deep neural networks have proven to be very powerful tools for a variety of tasks ranging from image and speech recognition to natural language processing. However, the large number of parameters of such networks can pose challenges. For example, a typical deep neural network consists of millions of learnable parameters, and in some cases, billions of learnable parameters, making training and deploying such networks computationally demanding and resource intensive. Summary of the invention

[0003] An aspect of the disclosed embodiment includes a method for neural network computation. The method includes receiving a codebook having a plurality of entries, multiplying a corresponding input of a layer of the neural network by each of the plurality of entries of the codebook, and determining an intermediate value for the corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook. The method also includes storing the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network. The method also includes determining each output of the layer of the neural network using the lookup table.

[0004] Another aspect of the disclosed embodiment includes a system for neural network computing. The system includes a processor and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: receive a codebook having a plurality of entries; multiply a corresponding input of a layer of the neural network by each of the plurality of entries of the codebook; determine an intermediate value for a corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook; store the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network; and determine each output of the layer of the neural network using the lookup table.

[0005] Another aspect of the disclosed embodiment includes an apparatus comprising a processor and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: retrieve a codebook having a plurality of entries and a dimension corresponding to the number of inputs of a layer of a neural network, wherein the neural network comprises a vector quantized deep neural network; multiply a corresponding input of the layer of the neural network by each of the plurality of entries of the codebook; determine an intermediate value for the corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook; store the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network; and determine each output of the layer of the neural network using the lookup table. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A system for training a neural network in accordance with the principles of the present disclosure is generally illustrated.

[0007] Figure 2 A computer-implemented method for training and utilizing a neural network in accordance with the principles of the present disclosure is generally illustrated.

[0008] Figure 3 Aspects of neural network computing techniques in accordance with principles of the present disclosure are generally illustrated.

[0009] Figure 4 is a flow chart generally illustrating a neural network computation method according to the principles of the present disclosure.

[0010] Figure 5 Depicted is a schematic diagram of the interaction between a computer-controlled machine and a control system in accordance with the principles of the present disclosure.

[0011] Figure 6 Depicted is a schematic diagram of a device according to the principles of the present disclosure. Figure 5 Schematic diagram of a control system configured to control a vehicle, which can be a partially autonomous vehicle, a fully autonomous vehicle, a partially autonomous robot, or a fully autonomous robot.

[0012] Figure 7 Depicted Figure 5 Schematic diagram of a control system configured to control a manufacturing machine, such as a punch cutter, a cutter, or a gun drill, of a manufacturing system (such as part of a production line).

[0013] Figure 8 Depicted Figure 5 Schematic diagram of a control system configured to control a power tool, such as a drill or driver, having an at least partially autonomous mode.

[0014] Fig. 9 Depicting a device configured to control an automated personal assistant Figure 5 Schematic diagram of the control system.

[0015] Fig.10 Depicted Figure 5 Schematic diagram of a control system configured to control a surveillance system, such as a controlled access system or a surveillance system.

[0016] Fig.11 Depicted Figure 5 Schematic diagram of a control system configured to control an imaging system, such as an MRI device, an x-ray imaging device, or an ultrasound device.

[0017] Fig. 12A and 12B Illustrated is an example vectorization of a quantized linear layer according to an embodiment. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure are described herein. However, it is to be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The drawings are not necessarily to scale; some features may be exaggerated or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as restrictive, but merely as a representative basis for teaching those skilled in the art to adopt the embodiments in various ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the figures in the accompanying drawings may be combined with the features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of the illustrated features provides a representative embodiment of a typical application. However, for a particular application or implementation, various combinations and modifications of features consistent with the teachings of the present disclosure may be desired.

[0019] As mentioned, deep neural networks have proven to be very powerful tools for a variety of tasks ranging from image and speech recognition to natural language processing. However, the large number of parameters of such networks can pose challenges. For example, a typical deep neural network consists of millions of learnable parameters, and in some cases, billions of learnable parameters, making training and deploying such networks computationally demanding and resource intensive.

[0020] Therefore, it may be desirable to be configured to provide efficient computation of deep neural network layers and systems and methods, such as the systems and methods described herein. In some embodiments, the systems and methods described herein may be configured to accelerate computations in deep neural network layers (e.g., at least when weights exhibit repeating patterns). The systems and methods described herein may be configured to provide efficient computations for layers of any suitable neural network, such as a vector quantized deep neural network and / or any other suitable neural network.

[0021] In some embodiments, the systems and methods described herein can be configured to account for high parameter counts. For example, the size of a deep neural network can be a double-edged sword. In some embodiments, a large number of parameters can enable complex modeling and better representation of complex patterns in data, but can also result in large memory usage and computational overhead.

[0022] The systems and methods described herein can be configured to account for parameter redundancy. In some embodiments, a deep neural network may exhibit parameter redundancy, meaning that not all parameters are optimally utilized during inference. In some embodiments, this redundancy may highlight inefficiencies in the architecture of the network, where a portion of the parameters may not contribute significantly to the predictive performance of the model.

[0023] The systems and methods described herein can be configured to consider pruning techniques to address parameter redundancy. In some embodiments, the systems and methods described herein can combine various pruning techniques. The systems and methods described herein can utilize these techniques to remove certain parameters or set certain parameters to zero, which can reduce the parameter count of the model. In some embodiments, pruning can result in a more compact model, better resource utilization, reduced memory usage, and increased reasoning speed. In addition, pruning can potentially produce a more interpretable and easier to understand model.

[0024] The systems and methods described herein propose novel approaches for solving the problem of high parameter counts in deep neural networks. The systems and methods described herein can be configured to utilize the concept of vector quantization to enable the utilization of a reduced number of parameters in deep neural networks. By employing vector quantization, the systems and methods described herein can be configured to capture essential information of the original parameters while reducing their overall size.

[0025] The systems and methods described herein can be configured to strike a balance between model complexity and efficiency. Although there may be a small computational overhead during the training phase, by utilizing vector quantization to reduce the number of parameters while maintaining model performance, the systems and methods described herein can be configured to achieve more resource-efficient deep neural networks at inference time. The systems and methods described herein can be configured to improve computational resource utilization, reduce inference latency, and facilitate the development of more interpretable models, thereby contributing to the wider adoption of deep learning techniques in real-world applications.

[0026] The systems and methods described herein introduce techniques that differentiate them from existing approaches, which can improve effectiveness and versatility in addressing challenges of parameter count, computational efficiency, and model interpretability.

[0027] The systems and methods described herein can be configured to utilize a global codebook for multiple layers. Unlike many other existing methods that use a separate codebook for each layer, the systems and methods described herein utilize a single global codebook for all layers of the same type in a deep neural network. The systems and methods described herein can demonstrate that implementing the same codebook on all layers may not degrade performance, and the model can reuse the same codebook to perform the same task. The systems and methods described herein can result in a reduction in the total number of parameters required for vector quantization. By sharing a common codebook across multiple layers, the systems and methods described herein reduce parameter redundancy and effectively compress the model, which can produce a more compact and resource-efficient model.

[0028] The systems and methods described herein can be configured to be compatible with prior art. An advantage of the systems and methods described herein is that they do not conflict with many other prior art techniques. The systems and methods described herein can be a complementary approach that can be integrated with various pruning, weight sharing, and other parameter reduction or quantization methods. In some embodiments, compatibility ensures that the systems and methods described herein can be used as a base framework on which other advanced techniques can be applied to further enhance the efficiency and performance of deep neural networks.

[0029] The systems and methods described herein can be configured to improve pruning efficiency. In some embodiments, the small codebook size used in the systems and methods described herein may result in parameter redundancy within the network. This redundancy due to the shared codebook can provide an inherent mechanism to promote the efficiency of existing pruning techniques. During the pruning process, redundant parameters can be naturally identified and eliminated, resulting in a more streamlined model without sacrificing performance. Therefore, the systems and methods described herein can not only directly reduce parameter counts through vector quantization, but also indirectly improve the efficiency of subsequent pruning processes.

[0030] The systems and methods described herein can be configured to take advantage of these improvements and can provide solutions to the challenges posed by large parameter counts in deep learning models. In some embodiments, the use of a global codebook, compatibility with other techniques, and the potential for enhanced pruning efficiency can collectively make the systems and methods described herein a compelling choice for researchers and practitioners seeking to develop resource-efficient, interpretable, and high-performance deep neural networks.

[0031] The systems and methods described herein can be configured to enhance edge computing and Internet of Things devices (IoT). Edge computing and IoT devices typically have limited computing resources and memory. The systems and methods described herein may include the ability to significantly reduce parameter counts and are particularly suitable for deploying deep neural networks on resource-constrained devices. In some embodiments, by compressing the model without compromising its performance, the systems and methods described herein enable efficient and low-latency AI processing at the edge, thereby allowing real-time reasoning and intelligent decision making in IoT applications.

[0032] The systems and methods described herein can be configured to enhance mobile and wearable devices. Mobile phones, wearable devices, and other portable devices may be becoming increasingly popular platforms for AI-based applications. However, the computational limitations of these devices make it challenging to run large and complex deep neural networks. The systems and methods described herein can facilitate the deployment of complex AI models on mobile and wearable devices by reducing memory usage and computational requirements, and can enhance the user experience of AI-driven applications.

[0033] The systems and methods described herein can be configured to enhance explainable AI systems. In applications where explainability is critical, such as medical diagnosis, finance, and legal fields, the systems and methods described herein provide the advantage of having a large amount of redundancy in network weights.

[0034] The systems and methods described herein can be configured to address the urgent need for efficient, interpretable, and real-world applicable AI models. In some embodiments, the systems and methods described herein can reduce parameter counts by adopting global codebooks for multiple layers, thereby improving resource efficiency and model interpretability. The compatibility of the systems and methods described herein with other technologies can allow seamless integration and further optimization. The systems and methods described herein can be consistent with the growing demand for edge computing, IoT devices, and explainable AI, while taking into account advances in deep learning hardware.

[0035] The systems and methods described herein are based on the principles of both data compression and deep learning. Unlike conventional methods that use a separate codebook for each layer, the systems and methods described herein use a single global codebook for multiple layers, thereby reducing parameter redundancy and model complexity. In addition, the systems and methods described herein can be compatible with prior art, which can allow for smooth integration, thereby enabling further optimization and improvement. The systems and methods described herein can be configured to provide novel solutions to enhance model efficiency, interpretability, and applicability in various deep learning fields.

[0036] The systems and methods described herein may be configured to utilize vector quantization, which is a data compression technique that can use a smaller number of representative vectors (called codewords) to represent a set of data points. The systems and methods described herein may be configured to create a codebook, which is a collection of codewords representing clusters of similar data points. The systems and methods described herein may be configured to divide the data space into non-overlapping regions, each of which is associated with a specific codeword from the codebook. In some embodiments, during encoding, each data vector (e.g., model parameter) may be mapped to the nearest codeword, which can effectively quantize the data. In some embodiments, during decoding, the original data may be reconstructed using the corresponding codeword from the codebook.

[0037] The systems and methods described herein can be configured to introduce vector quantization into deep neural networks by partitioning the parameters of each layer into non-overlapping regions (customized for the layer type). For 2D convolutional layers, the systems and methods described herein partition the weight matrix in the channel input and channel output dimensions. For example, the weight matrix has a shape of (Cout, Cin, K, K). The systems and methods described herein can be configured to partition the weight matrix into Cout*Cin / L regions of size K*K*L. This partitioning strategy may be effective for modern deep neural networks because they often utilize 1x1, 3x3, and 2D convolutional layers.

[0038] The systems and methods described herein can be configured to determine the size of each codebook. For example, E can be used as a hyperparameter to define the number of codewords in each codebook. The hyperparameter can be adjusted to optimize the performance of the model. In addition, E can vary for each type of layer, but the systems and methods described herein can be configured to utilize E=16, which provides excellent results across a variety of use cases.

[0039] Regarding the bias parameters of the layer, the systems and methods described herein can implement one of two possible options. The first option is to treat the bias parameters as in ordinary neural networks and not include them in the vector quantization process. In some embodiments, this approach may be appropriate because in most cases, the number of bias parameters is significantly lower than the number of weighting parameters. Therefore, ignoring the bias does not impair the effectiveness of the vector quantization technique. Regarding the second option, the systems and methods described herein can divide the bias parameters into multiple partitions, similar to how the linear layer is treated. By doing so, the systems and methods described herein can cause the bias parameters to undergo the same quantization process.

[0040] The systems and methods described herein can be configured to apply vector quantization to a deep neural network during a training phase. In a forward pass, the systems and methods described herein can quantize the weights of each supporting layer using a corresponding codebook until the last layer of the network. In some embodiments, the quantization process involves finding the nearest codeword in the codebook in each partition in the weight matrix, replacing the original weight with the corresponding codeword, and then performing a convolution operation.

[0041] The systems and methods described herein can be configured to calculate a loss value when reaching the last layer, which loss value can be composed of two components. The first component can be the normal loss of the network, such as the cross entropy loss of a classification task. The second component can be the distance between the quantized weights and the values ​​in the codebook. The distance component can be used as a regularization term that encourages the model to adhere to the representative codewords in the codebook. The loss component can be calculated independently for each layer and applied to both the codebook value and the weight of each layer separately.

[0042] The systems and methods described herein can be configured to pass, at each layer during a backward pass, the received gradients from the next layer to the previous layer, thereby allowing the model to efficiently learn from quantized representations.

[0043] In some embodiments, the systems and methods described herein can be configured to accelerate neural network computations by exploiting repetitive patterns present in network weights. The systems and methods described herein can be configured to address the challenge of redundancy in computations to exploit the reuse of previously computed results. The systems and methods described herein can be configured to use weights for each layer derived from a compact codebook, resulting in multiple occurrences of each code within the codebook, which can be applicable to vector quantized deep neural networks and other techniques, such as product quantization.

[0044] The systems and methods described herein can be configured to utilize the repetition of code to optimize the computation process, thereby reducing redundant computation and improving efficiency. The systems and methods described herein can be configured to effectively utilize the reuse of intermediate results, thereby significantly accelerating the overall reasoning and training phase of the neural network.

[0045] The systems and methods described herein can be configured to reduce floating point operations (FLOPs) for common layers in deep learning architectures. The systems and methods described herein can be configured to significantly reduce the number of FLOPs by utilizing advanced computational optimization techniques, which can directly translate into faster inference times, making them invaluable for resource-constrained environments such as edge devices and mobile platforms. The systems and methods described herein can be configured to avoid redundant computations and efficiently reuse the results of previous computations by identifying and utilizing repeated patterns and redundancies in layer computations. The systems and methods described herein can be configured to allow seamless integration into a variety of deep learning architectures, thereby ensuring consistent FLOP reductions across different tasks and applications.

[0046] The systems and methods described herein can be configured to improve computing in resource-constrained environments. The systems and methods described herein can be configured to enable the adoption of artificial intelligence (AI) in remote locations, embedded systems, and edge computing scenarios where resources are limited.

[0047] The systems and methods described herein can be configured to provide improved model development speed. The systems and methods described herein can be configured to reduce computational overhead, which can increase productivity in AI research, thereby enabling rapid progress in this field.

[0048] The systems and methods described herein can be configured to provide energy-efficient AI. The systems and methods described herein can be configured to reduce computing requirements, which may be beneficial for large-scale AI deployment, thereby reducing power consumption and promoting a greener AI infrastructure.

[0049] The systems and methods described herein can be configured to provide improvements in the use of real-time AI applications. The systems and methods described herein can be configured to improve real-time object detection, natural language processing, and autonomous systems based on the efficiency gains provided herein, which can ensure faster and more accurate responses in dynamic environments.

[0050] In some embodiments, the systems and methods described herein can be configured to introduce streamlined computational methods to accelerate the performance of vector quantized deep neural networks. The systems and methods described herein can be configured to reduce the memory usage and computational power requirements of running vector quantized deep neural networks at inference time by optimizing computations, thereby enabling resource-efficient and high-performance AI computing.

[0051] In some embodiments, the systems and methods described herein can be configured to provide efficient computation for deep neural networks with repeated weights, particularly in the context of vector quantized deep neural networks. The systems and methods described herein can be configured to represent weights using a relatively small codebook that can be significantly smaller than the original weights and characterized by a large amount of repetition (e.g., which can align well with vector quantized deep neural networks, where compact codebooks and repeated codewords are common).

[0052] The systems and methods described herein can be configured to modify conventional calculations in deep neural networks. The systems and methods described herein can be configured so that instead of directly calculating the product of the weights and the input (e.g., or performing a convolution operation), the systems and methods described herein can be configured to multiply the entire codebook with the input (e.g., or convolution). Since the codebook is much smaller than the original weights, these calculations can be performed significantly faster. The systems and methods described herein can be configured to retrieve the results of the corresponding codewords from the first step for each repeated partition of the weights, and concatenate the results to generate the desired results.

[0053] For example, Figure 3 As generally illustrated in , for linear layers (eg, where other types of layers correspond to similar techniques), consider a layer with N in *N out parameters and batch size is n b In the conventional approach, the weight matrix is ​​multiplied by the input and the results are summed along the output dimension, producing N b *N out output. Therefore, it consumes N b *N in *N out In some embodiments, the systems and methods described herein may be configured to maintain a codebook containing E entries of size L. The systems and methods described herein may be configured to store N out *(N in / L) indexes are stored into the codebook to construct the weight matrix W. Each index may include an integer between 1 and E, and thus uses a single byte of storage (e.g., which is in contrast to the 4-byte or 8-byte weight values ​​of conventional methods). The systems and methods described herein may be configured to multiply the input by the entire codebook and sum along the L dimension, producing N b *(N in / L)*L*E Flops and size is N b *(N in / L)*E output, called R. The entire computation is simplified to lookup from R. The systems and methods described herein can be configured to construct a size N by looking up from R and summing along the output dimension. b *N out *(N in / L) to generate a matrix of size N b *N out Therefore, the total Flop is N b *N in *(E+N out / L). For example, for a linear layer of size 768*768, the systems and methods described herein use a codebook of 16 entries with L=8, which can reduce Flop by about 6.8 times.

[0054] By utilizing this technology, the systems and methods described herein can be configured to achieve significant computational efficiencies for vector quantized deep neural networks. The reduction in computation allows for faster inference times, thereby providing a valuable tool for resource-efficient and high-performance AI applications in different real-world scenarios.

[0055] In some embodiments, the systems and methods described herein may be configured to provide neural network calculations. The systems and methods described herein may be configured to receive a codebook having a plurality of entries. The codebook may include a dimension corresponding to the number of inputs of a layer of the neural network. Each of the plurality of entries of the codebook may correspond to an index value. Each index value may include an integer between 1 and an upper limit. The upper limit may correspond to the total number of entries of the plurality of entries of the codebook. The corresponding layer of the neural network may include one of the plurality of linear layers of the neural network. The corresponding layer of the neural network may include a fully connected layer. The neural network may include a convolutional neural network, a vector quantized deep neural network, or other suitable network. The neural network may be associated with at least one aspect of controlling a vehicle and / or any other suitable application described herein and / or any other suitable application.

[0056] The systems and methods described herein may be configured to multiply a corresponding input of a layer of the neural network by each of a plurality of entries of a codebook. The systems and methods described herein may be configured to determine an intermediate value for a corresponding input of a layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook.

[0057] The systems and methods described herein may be configured to store intermediate values ​​associated with respective inputs of a layer of a neural network in a lookup table. The lookup table may include a plurality of intermediate values ​​corresponding to other inputs of a layer of a neural network. The systems and methods described herein may be configured to use the lookup table to determine each output of a layer of a neural network.

[0058] Figure 1 A system 100 for training a neural network is shown. The system 100 may include an input interface for accessing training data 102 for the neural network. For example, Figure 1 As illustrated in , the input interface may be constituted by a data storage interface 104, which can access the training data 102 from a data storage device 106. For example, the data storage interface 104 may be a memory interface or a persistent storage interface, such as a hard disk or SSD interface, but may also be a personal area network, a local area network, or a wide area network interface, such as a Bluetooth, Zigbee, or Wi-Fi interface, or an Ethernet or fiber optic interface. The data storage device 106 may be an internal data storage device of the system 100, such as a hard disk drive or SSD, but may also be an external data storage device, such as a network accessible data storage device.

[0059] In some embodiments, data storage 106 may further include a data representation 108 of an untrained version of the neural network, which may be accessed by system 100 from data storage 106. However, it will be appreciated that training data 102 and data representation 108 of the untrained neural network may also each be accessed from a different data storage, such as via a different subsystem of data storage interface 104. Each subsystem may be of the type described above with respect to data storage interface 104.

[0060] In some embodiments, the data representation 108 of the untrained neural network may be generated internally by the system 100 based on the design parameters of the neural network, and thus may not be explicitly stored on the data storage device 106. The system 100 may further include a processor subsystem 110 that may be configured to provide an iterated function as a replacement for the stack of layers of the neural network to be trained during operation of the system 100. Here, the corresponding layers of the stack of layers being replaced may have mutually shared weights and may receive as input the output of the previous layer (or, for the first layer of the stack of layers, the initial activation) and a portion of the input of the stack of layers.

[0061] The processor subsystem 110 may further be configured to iteratively train the neural network using the training data 102. Here, the iterations of training performed by the processor subsystem 110 may include a forward propagation portion and a backward propagation portion. The processor subsystem 110 may be configured to perform the forward propagation portion by determining an equilibrium point of the iterative function at which the iterative function converges to a fixed point and by providing the equilibrium point as a substitute for the output of the stack of layers in the neural network, in addition to other operations that may be performed to define the forward propagation portion, wherein determining the equilibrium point includes using a numerical root finding algorithm to find a root solution of the iterative function minus its input.

[0062] The system 100 may further include an output interface for outputting a data representation 112 of the trained neural network, which may also be referred to as trained model data 112. Figure 1 As illustrated in , the output interface may be constituted by a data storage interface 104, wherein in these embodiments, the interface is an input / output ("IO") interface, via which the trained model data 112 may be stored in a data storage device 106. For example, a data representation 108 defining an "untrained" neural network may be at least partially replaced by a data representation 112 of a trained neural network during or after training, as the parameters of the neural network (such as weights, hyperparameters, and other types of parameters of the neural network) may be adapted to reflect the training of the training data 102. This is also Figure 1 106 by reference numerals 108, 112 referring to the same data records on the data storage device 106. In some embodiments, the data representation 112 may be stored separately from the data representation 108 defining the "untrained" neural network. In some embodiments, the output interface may be separate from the data storage interface 104, but may generally be of the type described above for the data storage interface 104.

[0063] Figure 2 A data annotation / augmentation system 200 configured to provide an implemented multimodal artificial intelligence question answering is generally illustrated. The system 200 may include at least one computing system 202. The computing system 202 may include at least one processor 204 operably connected to a memory unit 208. The processor 204 may include one or more integrated circuits that implement the functionality of a central processing unit (CPU) 206. The CPU 206 may be a commercially available processing unit that implements an instruction set, such as one of the x86, ARM, Power, or MIPS instruction set families.

[0064] During operation, CPU 206 may execute stored program instructions retrieved from memory unit 208. The stored program instructions may include software that controls the operation of CPU 206 to perform the operations described herein. In some embodiments, processor 204 may be a system on a chip (SoC) that integrates the functionality of CPU 206, memory unit 208, network interfaces, and input / output interfaces into a single integrated device. Computing system 202 may implement an operating system for managing various aspects of operation.

[0065] Memory unit 208 may include volatile memory and non-volatile memory for storing instructions and data. Non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when computing system 202 is deactivated or powered off. Volatile memory may include static and dynamic random access memory (RAM) that stores program instructions and data. For example, memory unit 208 may store machine learning model 210 (e.g., in Figure 2 210), or an algorithm, a training data set 212 for the machine learning model 210, and an original source data set 216.

[0066] The computing system 202 may include a network interface device 222 configured to provide communications with external systems and devices. For example, the network interface device 222 may include a wired and / or wireless Ethernet interface as defined by the family of Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards. The network interface device 222 may include a cellular communication interface for communicating with a cellular network (e.g., 3G, 4G, 5G). The network interface device 222 may be further configured to provide a communication interface to an external network 224 or cloud.

[0067] The external network 224 may be referred to as the World Wide Web or the Internet. The external network 224 may establish a standard communication protocol between computing devices. The external network 224 may allow information and data to be easily exchanged between computing devices and the network. One or more servers 230 may communicate with the external network 224.

[0068] Computing system 202 may include input / output (I / O) interface 220, which may be configured to provide digital and / or analog input and output. I / O interface 220 may include an additional serial interface (eg, a universal serial bus (USB) interface) for communicating with external devices.

[0069] The computing system 202 may include a human-machine interface (HMI) device 218, which may include any device that enables the system 200 to receive control inputs. Examples of input devices may include human-machine interface inputs such as keyboards, mice, touch screens, voice input devices, and other similar devices. The computing system 202 may include a display device 232. The computing system 202 may include hardware and software for outputting graphics and text information to the display device 232. The display device 232 may include an electronic display screen, a projector, a printer, or other suitable devices for displaying information to a user or operator. The computing system 202 may be further configured to allow interaction with a remote HMI and a remote display device via a network interface device 222.

[0070] One or more computing systems may be used to implement system 200. Although this example depicts a single computing system 202 that implements all of the described features, it is contemplated that various features and functions may be separated and implemented by multiple computing units that communicate with each other. The particular system architecture selected may depend on a number of factors.

[0071] The system 200 may implement a machine learning model 210 (e.g., which may be referred to as a machine learning algorithm 210) configured to analyze a raw source data set 216. The raw source data set 216 may include raw or unprocessed sensor data, which may represent an input data set for a machine learning system. The raw source data set 216 may include videos, video segments, images, text-based information, and raw or partially processed sensor data (e.g., a radar map of an object). In some embodiments, the machine learning model 210 may be a neural network algorithm designed to perform a predetermined function. For example, a neural network algorithm may be configured to identify pedestrians in a video image in an automotive application.

[0072] The computer system 200 may store a training data set 212 for the machine learning model 210. The training data set 212 may represent a set of previously constructed data used to train the machine learning model 210. The training data set 212 may be used by the machine learning model 210 to learn weighting factors associated with a neural network algorithm. The training data set 212 may include a set of source data having corresponding achievements or results that the machine learning model 210 attempts to replicate through a learning process.

[0073] The machine learning model 210 can operate in a learning mode using the training data set 212 as input. The machine learning model 210 can be executed over multiple iterations using data from the training data set 212. With respect to each iteration, the machine learning model 210 can update the internal weighting factors based on the results obtained. For example, the machine learning model 210 can compare the output results (e.g., annotations) with those included in the training data set 212. Since the training data set 212 includes the expected results, the machine learning model 210 can determine when the performance is acceptable. After the machine learning model 210 reaches a predetermined performance level (e.g., 100% consistent with the results associated with the training data set 212), the machine learning model 210 can be executed using data that is not in the training data set 212. The trained machine learning model 210 can be applied to a new data set to perform any suitable function of any of the applications described herein or other suitable applications.

[0074] The machine learning model 210 may be configured to identify specific features in the raw source data 216. The raw source data 216 may include multiple instances or input data sets for which various predictions are desired. The machine learning model 210 may be programmed to process the raw source data 216 to identify the presence of specific features. The machine learning model 210 may be configured to use the raw source data 216 to predict one or more answers to one or more questions. The raw source data 216 may be derived from a variety of sources. For example, the raw source data 216 may be actual input data collected by the machine learning system. The raw source data 216 may be machine generated for testing the system.

[0075] In this example, the machine learning model 210 can process the raw source data 216 and output a prediction. The machine learning model 210 can generate a confidence level (e.g., a determination value) or factor for each output generated. For example, a confidence value exceeding a predetermined high confidence threshold can indicate that the machine learning model 210 is confident that the prediction will result in the desired outcome. A confidence value less than a low confidence threshold can indicate that the machine learning model 210 has some uncertainty that the prediction will result in the desired outcome.

[0076] In some embodiments, system 200 may provide neural network calculations. System 200 may receive a codebook having a plurality of entries. The codebook may include a dimension corresponding to the number of inputs of a layer of a neural network associated with machine learning model 210. Each of the plurality of entries of the codebook may correspond to an index value. Each index value may include an integer between 1 and an upper limit. The upper limit may correspond to the total number of entries of the plurality of entries of the codebook. The corresponding layer of the neural network may include one of the plurality of linear layers of the neural network. The corresponding layer of the neural network may include a fully connected layer. The neural network may include a convolutional neural network, a vector quantized deep neural network, or other suitable network. The neural network may be associated with at least one aspect of controlling a vehicle and / or any other suitable application described herein and / or any other suitable application.

[0077] The system 200 may multiply the corresponding input of the layer of the neural network by each of the plurality of entries of the codebook. The system 200 may determine an intermediate value for the corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook.

[0078] The system 200 may store intermediate values ​​associated with corresponding inputs of a layer of the neural network in a lookup table. The lookup table may include multiple intermediate values ​​corresponding to other inputs of the layer of the neural network. The system 200 may use the lookup table to determine each output of a layer of the neural network.

[0079] It should be understood that the systems and methods described herein may be configured to perform any suitable functions, such as those described herein with respect to Figure 6-11 Those features described.

[0080] Figure 4 is a flow chart generally illustrating a neural network computation method 400 in accordance with the principles of the present disclosure. At 402, the method 400 receives a codebook having a plurality of entries. For example, the system 200 may receive the codebook.

[0081] At 404, method 400 multiplies the corresponding input of the layer of the neural network by each entry in the plurality of entries of the codebook. For example, system 200 may multiply the corresponding input of the layer of the neural network by each entry in the plurality of entries of the codebook.

[0082] At 406, method 400 determines an intermediate value for a corresponding input of a layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each entry of the plurality of entries of the codebook. For example, system 200 can determine the intermediate value.

[0083] At 408, method 400 stores the intermediate values ​​associated with the corresponding inputs of the layers of the neural network in a lookup table. For example, system 200 can store the intermediate values ​​in the lookup table. The lookup table can include multiple intermediate values ​​corresponding to other inputs of the layers of the neural network.

[0084] At 410, method 400 uses a lookup table to determine each output of a layer of a neural network. For example, system 200 can use a lookup table to determine each output of a layer of a neural network.

[0085] Figure 5 A schematic diagram of the interaction between a computer-controlled machine 500 and a control system 502 is depicted. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 may include one or more actuators, and the sensor 506 may include one or more sensors. The sensor 506 is configured to sense a condition of the computer-controlled machine 500. The sensor 506 may be configured to encode the sensed condition into a sensor signal 508 and transmit the sensor signal 508 to the control system 502. Non-limiting examples of the sensor 506 include video, radar, LiDAR, ultrasound, and motion sensors. In some embodiments, the sensor 506 is an optical sensor configured to sense an optical image of the environment near the computer-controlled machine 500.

[0086] The control system 502 is configured to receive sensor signals 508 from the computer controlled machine 500. As described below, the control system 502 may be further configured to calculate actuator control commands 510 based on the sensor signals and transmit the actuator control commands 510 to the actuators 504 of the computer controlled machine 500.

[0087] like Figure 5 As shown in , the control system 502 includes a receiving unit 512. The receiving unit 512 can be configured to receive a sensor signal 508 from the sensor 506 and transform the sensor signal 508 into an input signal x. In an alternative embodiment, without the receiving unit 512, the sensor signal 508 is directly received as an input signal x. Each input signal x can be a part of each sensor signal 508. The receiving unit 512 can be configured to process each sensor signal 508 to generate each input signal x. The input signal x can include data corresponding to an image recorded by the sensor 506.

[0088] The control system 502 includes a classifier 514. The classifier 514 can be configured to classify the input signal x into one or more labels using a machine learning (ML) algorithm (such as the neural network described above). The classifier 514 is configured to be parameterized by parameters (such as those described above) (e.g., parameter θ). The parameter θ can be stored in and provided by the non-volatile storage device 516. The classifier 514 is configured to determine the output signal y based on the input signal x. Each output signal y includes information that assigns one or more labels to each input signal x. The classifier 514 can transmit the output signal y to a conversion unit 518. The conversion unit 518 is configured to convert the output signal y into an actuator control command 510. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504, and the actuator 504 is configured to actuate the computer-controlled machine 500 in response to the actuator control command 510. In some embodiments, actuator 504 is configured to actuate computer-controlled machine 500 directly based on output signal y.

[0089] When the actuator control command 510 is received by the actuator 504, the actuator 504 is configured to perform an action corresponding to the associated actuator control command 510. The actuator 504 may include control logic configured to transform the actuator control command 510 into a second actuator control command for controlling the actuator 504. In one or more embodiments, the actuator control command 510 may be used to control a display instead of or in addition to the actuator.

[0090] In some embodiments, the control system 502 includes the sensor 506 instead of or in addition to the computer-controlled machine 500 including the sensor 506. The control system 502 may also include the actuator 504 instead of or in addition to the computer-controlled machine 500 including the actuator 504.

[0091] like Figure 5 As shown in , the control system 502 also includes a processor 520 and a memory 522. The processor 520 may include one or more processors. The memory 522 may include one or more memory devices. The classifier 514 (e.g., ML algorithm) of one or more embodiments may be implemented by the control system 502, which includes a non-volatile storage device 516, a processor 520, and a memory 522.

[0092] The non-volatile storage 516 may include one or more persistent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, a cloud storage device, or any other device capable of persistently storing information. The processor 520 may include one or more devices selected from a high-performance computing (HPC) system, including a high-performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, or any other device that manipulates signals (analog or digital) based on computer-executable instructions residing in the memory 522. The memory 522 may include a single memory device or multiple memory devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0093] The processor 520 may be configured to read into the memory 522 and execute computer executable instructions residing in the non-volatile storage 516 and implementing one or more ML algorithms and / or methods of one or more embodiments. The non-volatile storage 516 may include one or more operating systems and applications. The non-volatile storage 516 may store data compiled and / or interpreted from a computer program created using a variety of programming languages ​​and / or technologies, including but not limited to Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL (alone or in combination).

[0094] When executed by the processor 520, the computer executable instructions of the non-volatile storage device 516 can cause the control system 502 to implement one or more of the ML algorithms and / or methods as disclosed herein. The non-volatile storage device 516 may also include ML data (including data parameters) that support the functions, features, and processes of one or more embodiments described herein.

[0095] The program code implementing the algorithm and / or method described herein can be distributed separately or jointly as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium having a computer-readable program instruction for making the processor execute various aspects of one or more embodiments. The inherently non-temporary computer-readable storage medium may include a volatile and non-volatile, and removable and non-removable tangible medium implemented by any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). The computer-readable storage medium may further include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM) or other optical storage, cassette, magnetic tape, disk storage device or other magnetic storage device, or any other medium that can be used to store desired information and can be read by a computer. The computer-readable program instruction can be downloaded from the computer-readable storage medium to a computer, another type of programmable data processing device or another device, or downloaded to an external computer or external storage device via a network.

[0096] The computer-readable program instructions stored in the computer-readable medium can be used to guide a computer, other types of programmable data processing devices, or other devices to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions, actions, and / or operations specified in the flowchart or figure. In certain alternative embodiments, the functions, actions, and / or operations specified in the flowchart and figure can be reordered, processed serially, and / or processed simultaneously according to one or more embodiments. In addition, any of the flowchart and / or figure may include more or fewer nodes or blocks than those illustrated in accordance with one or more embodiments.

[0097] The process, method or algorithm may be implemented in whole or in part using suitable hardware components, such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components.

[0098] Figure 6A schematic diagram of a control system 502 configured to control a vehicle 600 is depicted, which may be an at least partially autonomous vehicle or an at least partially autonomous robot. The vehicle 600 includes an actuator 504 and a sensor 506. The sensor 506 may include one or more video sensors, cameras, radar sensors, ultrasonic sensors, LiDAR sensors, and / or position sensors (e.g., GPS). One or more of the one or more specific sensors may be integrated into the vehicle 600. Alternatively or in addition to the one or more specific sensors identified above, the sensor 506 may include a software module configured to determine the state of the actuator 504 when executed. A non-limiting example of a software module includes a weather information software module that is configured to determine the current or future state of the weather near the vehicle 600 or other location.

[0099] The classifier 514 of the control system 502 of the vehicle 600 may be configured to detect an object near the vehicle 600 based on the input signal x. In such an embodiment, the output signal y may include information characterizing the proximity of the object to the vehicle 600. The actuator control command 510 may be determined based on this information. The actuator control command 510 may be used to avoid a collision with the detected object.

[0100] In some embodiments, the vehicle 600 is an at least partially autonomous vehicle, and the actuator 504 can be implemented in the brakes, propulsion system, engine, transmission system, or steering device of the vehicle 600. The actuator control command 510 can be determined so that the actuator 504 is controlled so that the vehicle 600 avoids colliding with the detected object. The detected objects can also be classified according to what the classifier 514 thinks they are most likely to be, such as pedestrians or trees. The actuator control command 510 can be determined based on the classification. In the case where a hostile attack may occur, the above system can be further trained to better detect objects or identify changes in the lighting conditions or angles of the sensors or cameras on the vehicle 600.

[0101] In some embodiments in which the vehicle 600 is an at least partially autonomous robot, the vehicle 600 may be a mobile robot configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In such embodiments, the actuator control commands 510 may be determined such that a propulsion unit, a steering unit, and / or a braking unit of the mobile robot may be controlled such that the mobile robot may avoid colliding with an identified object.

[0102] In some embodiments, vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. In such embodiments, vehicle 600 may use an optical sensor as sensor 506 to determine the state of plants in the environment near vehicle 600. Actuator 504 may be a nozzle configured to spray a chemical. Depending on the identified species and / or the state of the identified plant, actuator control commands 510 may be determined to cause actuator 504 to spray the plant with an appropriate amount of an appropriate chemical.

[0103] Vehicle 600 can be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include washing machines, stoves, ovens, microwave ovens, or dishwashers. In such a vehicle 600, sensor 506 can be an optical sensor configured to detect the state of an object to be processed by the household appliance. For example, in the case where the household appliance is a washing machine, sensor 506 can detect the state of the laundry in the washing machine. Actuator control commands 510 can be determined based on the detected state of the laundry.

[0104] Figure 7 A schematic diagram of a control system 502 is depicted, the control system 502 being configured to control a system 700 (e.g., a manufacturing machine), such as a punch cutter, a cutter, or a gun drill, of a manufacturing system 702 (e.g., a portion of a production line). The control system 502 may be configured to control an actuator 504 configured to control the system 700 (e.g., a manufacturing machine).

[0105] The sensor 506 of the system 700 (e.g., a manufacturing machine) may be an optical sensor configured to capture one or more properties of the manufactured product 704. The classifier 514 may be configured to determine a state of the manufactured product 704 from one or more of the captured properties. The actuator 504 may be configured to control the system 700 (e.g., a manufacturing machine) for subsequent manufacturing steps of the manufactured product 704 based on the determined state of the manufactured product 704. The actuator 504 may be configured to control the function of the system 700 (e.g., a manufacturing machine) on the subsequent manufactured product 706 of the system 700 (e.g., a manufacturing machine) based on the determined state of the manufactured product 704.

[0106] Figure 8 A schematic diagram of a control system 502 configured to control a power tool 800 , such as a drill or driver, having an at least partially autonomous mode is depicted. The control system 502 may be configured to control an actuator 504 configured to control the power tool 800 .

[0107] The sensor 506 of the power tool 800 can be an optical sensor configured to capture one or more properties of the working surface 802 and / or the fastener 804 driven into the working surface 802. The classifier 514 can be configured to determine the state of the working surface 802 and / or the fastener 804 relative to the working surface 802 from one or more of the captured properties. The state can be that the fastener 804 is flush with the working surface 802. Alternatively, the state can be the hardness of the working surface 802. The actuator 504 can be configured to control the power tool 800 so that the driving function of the power tool 800 is adjusted according to the determined state of the fastener 804 relative to the working surface 802 or one or more captured properties of the working surface 802. For example, if the state of the fastener 804 is flush with the working surface 802, the actuator 504 can interrupt the driving function. As another non-limiting example, the actuator 504 can apply additional or less torque according to the hardness of the working surface 802.

[0108] Fig. 9 Depicted is a schematic diagram of a control system 502 configured to control an automated personal assistant 900. The control system 502 may be configured to control an actuator 504, which is configured to control the automated personal assistant 900. The automated personal assistant 900 may be configured to control a household appliance, such as a washing machine, stove, oven, microwave, or dishwasher.

[0109] Sensor 506 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive a video image of gesture 904 of user 902. The audio sensor may be configured to receive a voice command of user 902.

[0110] The control system 502 of the automated personal assistant 900 can be configured to determine an actuator control command 510 configured to control the system 502. The control system 502 can be configured to determine the actuator control command 510 based on the sensor signal 508 of the sensor 506. The automated personal assistant 900 is configured to transmit the sensor signal 508 to the control system 502. The classifier 514 of the control system 502 can be configured to execute a gesture recognition algorithm to identify a gesture 904 made by the user 902, determine the actuator control command 510, and transmit the actuator control command 510 to the actuator 504. The classifier 514 can be configured to retrieve information from the non-volatile storage device in response to the gesture 904, and output the retrieved information in a form suitable for receipt by the user 902.

[0111] Fig.10A schematic diagram of a control system 502 configured to control a surveillance system 1000 is depicted. The surveillance system 1000 can be configured to physically control access through a door 1002. A sensor 506 can be configured to detect scenes relevant to deciding whether to grant access. The sensor 506 can be an optical sensor configured to generate and transmit image and / or video data. Such data can be used by the control system 502 to detect a person's face.

[0112] The classifier 514 of the control system 502 of the monitoring system 1000 can be configured to interpret the image and / or video data by matching known identities of people stored in the non-volatile storage device 516 to determine the identity of the person. The classifier 514 can be configured to generate an actuator control command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504. In this embodiment, the actuator 504 can be configured to lock or unlock the door 1002 in response to the actuator control command 510. In some embodiments, non-physical, logical access control is also possible.

[0113] The monitoring system 1000 may also be a surveillance system. In such an embodiment, the sensor 506 may be an optical sensor configured to detect a scene under surveillance, and the control system 502 is configured to control the display 1004. The classifier 514 is configured to determine a classification of the scene, such as whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to transmit an actuator control command 510 to the display 1004 in response to the classification. The display 1004 may be configured to adjust the displayed content in response to the actuator control command 510. For example, the display 1004 may highlight an object that is considered suspicious by the classifier 514. Using embodiments of the disclosed system, the surveillance system can predict objects that appear at certain times in the future.

[0114] Fig.11 A schematic diagram of a control system 502 configured to control an imaging system 1100 (e.g., an MRI device, an x-ray imaging device, or an ultrasound device) is depicted. The sensor 506 may be, for example, an imaging sensor. The classifier 514 may be configured to determine a classification of all or part of a sensed image. The classifier 514 may be configured to determine or select an actuator control command 510 in response to a classification obtained by a trained neural network. For example, the classifier 514 may interpret an area of ​​the sensed image as a potential anomaly. In this case, the actuator control command 510 may be determined or selected so that the display 1102 displays the imaging and highlights the potential abnormal area.

[0115] Fig. 12AA simplified diagram 1200 of a vector quantized linear system according to the present disclosure is depicted. In some embodiments, the simplified diagram 1200 includes an index matrix 1202, which may include model parameters of a neural model. The index matrix 1202 may work with a codebook 1204, where the codebook has E entries, each of size 1. The index matrix 1202 works with the codebook 1204 to produce intermediate results in a weight matrix 1206. Each parameter in the weight matrix 1206 is an entry in the codebook 1204. Once the weight matrix 1206 is generated based on the index matrix 1202 and the codebook 1204, the weight matrix 1206 may be used as a normal linear layer in the neural model. In this article, the required storage space may be expressed as 4*E+Nin*Nout, and the required processing power may be expressed as Flop=Nin*Nout.

[0116] Fig. 12B A simplified diagram 1250 of a vector quantized linear system according to the present disclosure is depicted. In some embodiments, the simplified diagram 1250 includes an index matrix 1252 of model parameters. In some embodiments, the simplified diagram 1250 may further include a multiplication lookup matrix 1254 of intermediate results. To generate the multiplication lookup matrix 1254, every L consecutive input values ​​are multiplied by all codebook entries and every L values ​​are summed. To generate an output 1256, a lookup process is performed based on the multiplication lookup matrix 1254.

[0117] In some embodiments, a method for generating a vector quantized deep neural network includes: receiving a training data set including one or more images; training a neural model with the training data set to generate a first layer having weighted parameters; dividing the first layer into a first predetermined number of segments based on the first layer being a layer of the first type; generating a codebook by replacing the weighted parameters in each segment of the first layer with a codeword based on finding a representative vector most closely related to the weighted parameters of each segment in a vectorization dictionary, wherein the codebook includes a plurality of codewords equal to the first predetermined number of segments and each codeword includes a representative vector; and in response to updating the neural model with the codebook, outputting a trained neural model including the codebook that replaces the first layer.

[0118] In some embodiments, the method further includes: dividing the second layer into a second predetermined number of segments based on the second layer being a layer of the second type. In some embodiments, the method further includes: updating the codebook by replacing the weighted parameters in each segment of the second layer with a codeword based on finding the representative vector most closely related to the weighted parameters of each segment in the vectorization dictionary; and, in response to updating the codebook, outputting a trained neural model including the codebooks that replace the first layer and the second layer. In some embodiments, the method further includes: pruning redundant parameters from the neural model based on the codebook. In some embodiments, the first layer of the first type can be at least one of the following: a linear layer, a 1x1 convolutional layer, and a 3x3 convolutional layer. In some embodiments, the first layer is a layer of the linear type, and the first predetermined number of segments is 8. In some embodiments, the second layer is a layer of the convolutional type, and the second predetermined number of segments is 16. In some embodiments, after training and vectorization, the neural model is utilized by at least one of the following: a mobile device, an Internet of Things device, a wearable device. In some embodiments, the segments of the first layer do not overlap. In some embodiments, the one or more images are at least one of the following: a number, text, audio, a vector image, a bitmap image, and a sensor signal.

[0119] In some embodiments, a device for generating a vector quantized deep neural network includes one or more processors configured to: receive a training data set including one or more images; train a neural model with the training data set to generate a first layer with weighted parameters; divide the first layer into a first predetermined number of segments based on the first layer being a layer of the first type; generate a codebook by replacing the weighted parameters in each segment of the first layer with a codeword based on finding a representative vector most closely related to the weighted parameters of each segment in a vectorization dictionary, wherein the codebook includes a plurality of codewords equal to the first predetermined number of segments and each codeword includes a representative vector; and in response to updating the neural model with the codebook, output a trained neural model including the codebook that replaces the first layer.

[0120] In some embodiments, the one or more processors are further configured to: divide the second layer into a second predetermined number of segments based on the second layer being a layer of the second type. In some embodiments, the one or more processors are further configured to: update the codebook by replacing the weighting parameters in each segment of the second layer with a codeword based on finding a representative vector that is most closely related to the weighting parameters of each segment in the vectorization dictionary; and, in response to updating the codebook, output a trained neural model that includes replacing the codebooks of the first and second layers. In some embodiments, after training and vectorization, the neural model is utilized by at least one of the following: a mobile device, an Internet of Things device, a wearable device. In some embodiments, the segments of the first layer do not overlap. In some embodiments, the one or more images are at least one of the following: a number, text, audio, a vector image, a bitmap image, and a sensor signal.

[0121] In some embodiments, a system for generating a vector quantized deep neural network includes one or more processors configured to: receive a training data set including one or more images; train a neural model with the training data set to generate a first layer having weighted parameters; divide the first layer into a first predetermined number of segments based on the first layer being a layer of a first type; generate a codebook by replacing the weighted parameters in each segment of the first layer with a codeword based on finding a representative vector most closely related to the weighted parameters of each segment in a vectorization dictionary, wherein the codebook includes a plurality of codewords equal to the first predetermined number of segments and each codeword includes a representative vector; and in response to updating the neural model with the codebook, output a trained neural model including the codebook that replaces the first layer.

[0122] In some embodiments, the one or more processors are further configured to: divide the second layer into a second predetermined number of segments based on the second layer being a layer of the second type. In some embodiments, the one or more processors are further configured to: update the codebook by replacing the weighting parameters in each segment of the second layer with a codeword based on finding a representative vector that is most closely related to the weighting parameters of each segment in the vectorization dictionary; and, in response to updating the codebook, output a training neural model including the codebook that replaces the first layer and the second layer. In some embodiments, the first layer of the first type can be at least one of the following: a linear layer, a 1x1 convolutional layer, and a 3x3 convolutional layer.

[0123] In some embodiments, a method for neural network computation includes receiving a codebook having a plurality of entries, multiplying a corresponding input of a layer of the neural network by each of the plurality of entries of the codebook, and determining an intermediate value for the corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook. The method also includes storing the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network. The method also includes determining each output of the layer of the neural network using the lookup table.

[0124] In some embodiments, the codebook has a dimension corresponding to the number of inputs of the layer of the neural network. In some embodiments, each entry in the plurality of entries of the codebook corresponds to an index value. In some embodiments, each index value is an integer between 1 and an upper limit. In some embodiments, the upper limit corresponds to the total number of entries in the plurality of entries of the codebook. In some embodiments, the layer of the neural network is one of a plurality of linear layers of the neural network. In some embodiments, the layer of the neural network comprises a fully connected layer. In some embodiments, the neural network comprises a convolutional neural network. In some embodiments, the neural network comprises a vector quantized deep neural network. In some embodiments, the neural network is associated with controlling at least one aspect of a vehicle.

[0125] In some embodiments, a system for neural network computation includes a processor and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: receive a codebook having a plurality of entries; multiply a corresponding input of a layer of the neural network by each of the plurality of entries of the codebook; determine an intermediate value for a corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook; store the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network; and determine each output of the layer of the neural network using the lookup table.

[0126] In some embodiments, the codebook has a dimension corresponding to the number of inputs of the layer of the neural network. In some embodiments, each entry in the plurality of entries of the codebook corresponds to an index value. In some embodiments, each index value is an integer between 1 and an upper limit. In some embodiments, the upper limit corresponds to the total number of entries in the plurality of entries of the codebook. In some embodiments, the layer of the neural network is one of a plurality of linear layers of the neural network. In some embodiments, the layer of the neural network comprises a fully connected layer. In some embodiments, the neural network comprises a convolutional neural network. In some embodiments, the neural network comprises a vector quantized deep neural network.

[0127] In some embodiments, an apparatus includes a processor and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: retrieve a codebook having a plurality of entries and a dimension corresponding to the number of inputs of a layer of a neural network, wherein the neural network includes a vector quantized deep neural network; multiply a corresponding input of the layer of the neural network by each of the plurality of entries of the codebook; determine an intermediate value for the corresponding input of the layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each of the plurality of entries of the codebook; store the intermediate value associated with the corresponding input of the layer of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layer of the neural network; and determine each output of the layer of the neural network using the lookup table.

[0128] The process, method or algorithm disclosed herein can be delivered to / implemented by a processing device, a controller or a computer, which may include any existing programmable electronic control unit or a dedicated electronic control unit. Similarly, the process, method or algorithm can be stored in various forms as data and instructions that can be executed by a controller or a computer, including but not limited to information permanently stored on a non-writable storage medium (such as a ROM device) and information variably stored on a writable storage medium (such as a floppy disk, a tape, a CD, a RAM device and other magnetic and optical media). The process, method or algorithm can also be implemented in a software executable object. Alternatively, suitable hardware components, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a state machine, a controller or other hardware components or devices, or a combination of hardware, software and firmware components, can be used to implement the process, method or algorithm in whole or in part.

[0129] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms covered by the claims. The words used in the specification are descriptive words, not restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of various embodiments can be combined to form further embodiments of the present invention that may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages or being superior to other embodiments or prior art implementations with respect to one or more desired characteristics, it is recognized by those of ordinary skill in the art that one or more features or characteristics can be compromised to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, applicability, weight, manufacturability, ease of assembly, etc. Therefore, to the extent that any embodiment is described as not as good as other embodiments or prior art implementations of one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desired for specific applications.

Claims

1. A method for neural network computing, the method comprising: receiving a codebook having a plurality of entries; multiplying a corresponding input of a layer of the neural network by each of a plurality of entries of the codebook; determining an intermediate value for a corresponding input of a layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each entry of a plurality of entries of the codebook; storing intermediate values ​​associated with respective inputs of the layers of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layers of the neural network; as well as A lookup table is used to determine each output of a layer of the neural network.

2. The method of claim 1, wherein the codebook has a dimension corresponding to the number of inputs to the layers of the neural network. The method of claim 1 , wherein each entry of the plurality of entries of the codebook corresponds to an index value. The method according to claim 3 , wherein each index value is an integer between 1 and an upper limit. The method according to claim 4 , wherein the upper limit corresponds to a total number of entries of the plurality of entries of the codebook.

6. The method of claim 1, wherein the layer of the neural network is one of a plurality of linear layers of the neural network.

7. The method of claim 1, wherein the layers of the neural network include fully connected layers.

8. The method of claim 1, wherein the neural network comprises a convolutional neural network.

9. The method of claim 1, wherein the neural network comprises a vector quantized deep neural network.

10. The method of claim 1, wherein the neural network is associated with controlling at least one aspect of the vehicle.

11. A system for neural network computing, the system comprising: processor; as well as a memory comprising instructions that, when executed by a processor, cause the processor to: receiving a codebook having a plurality of entries; multiplying a corresponding input of a layer of the neural network by each of a plurality of entries of the codebook; determining an intermediate value for a corresponding input of a layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each entry of a plurality of entries of the codebook; storing intermediate values ​​associated with respective inputs of the layers of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layers of the neural network; as well as A lookup table is used to determine each output of a layer of the neural network.

12. The system of claim 11, wherein the codebook has a dimension corresponding to the number of inputs to the layers of the neural network.

13. The system of claim 11, wherein each entry of the plurality of entries of the codebook corresponds to an index value.

14. The system of claim 13, wherein each index value is an integer between 1 and an upper limit.

15. The system of claim 14, wherein the upper limit corresponds to a total number of entries of the plurality of entries of the codebook.

16. The system of claim 11, wherein the layer of the neural network is one of a plurality of linear layers of the neural network.

17. The system of claim 11, wherein the layers of the neural network include fully connected layers.

18. The system of claim 11, wherein the neural network comprises a convolutional neural network.

19. The system of claim 11, wherein the neural network comprises a vector quantized deep neural network.

20. An apparatus comprising: processor; as well as a memory comprising instructions that, when executed by a processor, cause the processor to: retrieving a codebook having a plurality of entries and a dimension corresponding to a number of inputs to a layer of a neural network, wherein the neural network comprises a vector quantized deep neural network; multiplying a corresponding input of a layer of the neural network by each of a plurality of entries of the codebook; determining an intermediate value for a corresponding input of a layer of the neural network based on a sum of each result of multiplying the corresponding layer of the neural network by each entry of a plurality of entries of the codebook; storing intermediate values ​​associated with respective inputs of the layers of the neural network in a lookup table, the lookup table including a plurality of intermediate values ​​corresponding to other inputs of the layers of the neural network; as well as A lookup table is used to determine each output of a layer of the neural network.

Images