Recursive neural networks and systems for decoding encoded data

Abstract

Examples described herein utilize a multi-layer neural network, such as a multi-layer recurrent neural network to decode encoded data (e.g., data encoded using one or more encoding techniques). The neural network and / or recurrent neural network can have non-linear mapping and distributed processing capabilities, which can be advantageous in many systems that employ the neural network decoder and / or recurrent neural network. In this manner, the neural network or recurrent neural network described herein can be used to implement an error correction coding (ECC) decoder.

Description

Technical Field

[0001] The examples described in this paper relate to neural networks used for decoding encoded data. Examples of neural networks that can be used with error-correcting decoder (ECC) memories are described, where the neural network can be used to decode encoded data. Background Technology

[0002] Error correction decoding (ECC) can be used in a variety of applications, such as memory devices or wireless baseband circuit systems. Generally, error correction decoding techniques use extra bits to encode raw data to describe the raw bits intended to be stored, retrieved, and / or transmitted. These extra bits can be stored along with the raw bits. Therefore, L bits of raw data can exist to be stored and / or transmitted. An encoder can provide NL extra bits, such that the encoded data can be N bits in size. The raw bits can be stored as raw bits, or can be modified by the encoder to form the encoded N bits of stored data. A decoder can decode the N bits to retrieve and / or estimate the original L bits, and in some instances, these bits can be corrected according to ECC techniques.

[0003] Bit flips (e.g., changes in the charge of memory cells) occur in non-volatile memory devices where ECC can be applied. Therefore, memory devices can be operated with complex error correction techniques, leading to increasing area and power requirements; resulting in higher silicon costs and longer firmware development times. Summary of the Invention

[0004] Example devices are disclosed herein. In embodiments of this disclosure, a device includes a first-level circuitry configured to receive encoded data, combine the encoded data, and evaluate at least one nonlinear function using the combination of encoded data and a delayed version of the combination of encoded data to provide intermediate data. The device further includes at least a second-level circuitry configured to receive the intermediate data and combine the intermediate data using a predetermined set of weights to provide decoded data, the predetermined set of weights being at least partially based on an encoding technique associated with the encoded data.

[0005] Alternatively, the predetermined weight set is based on the training of a delayed version of the neural network on known encoded data and combinations of known encoded data.

[0006] Alternatively, the neural network is configured to use a predetermined set of weights to provide an estimate of the decoded data based on the received encoded data.

[0007] Alternatively, the device may further include a memory encoded with executable instructions to select a predetermined set of weights for the neural network based on an encoding technique associated with the encoded data.

[0008] Alternatively or concurrently, coding techniques include Reed-Solomon decoding, Bose-Chaudhuri-Hocquenghem (BCH) decoding, low-density parity-check (LDPC) decoding, polarization decoding, or combinations thereof.

[0009] Alternatively, the first-level circuit system and the second-level circuit system include a first plurality of multiplication / accumulation units, each of which is configured to multiply at least one bit of the encoded data with at least one bit of a predetermined set of weights and to sum a plurality of weighted bits of the encoded data.

[0010] Alternatively, the first-level circuit system further includes a first plurality of memory lookup units (MLUs), each of the first plurality of MLUs being configured to retrieve at least one intermediate data value corresponding to the output of a corresponding one of the first plurality of multiplication / accumulation units based on at least one nonlinear function.

[0011] Alternatively, the first-stage circuit system further includes a plurality of delay units configured to provide a delayed version of a combination of encoded data based on at least one intermediate data value provided by a first plurality of MLUs.

[0012] Alternatively, the device may further include a memory that communicates with a first-level circuit system and a second-level MAC circuit system, the memory being configured to store a plurality of predetermined weight sets, including the predetermined weight sets.

[0013] In another aspect of this disclosure, a method includes: receiving at a computing device including a neural network a signaling instruction indicating a set of data pairs, each including known encoded data and decoded data, wherein the known encoded data is encoded using an encoding technique; determining a set of weights for the neural network to decode the data using the encoding technique, in part based on the signaling, a coefficient multiplication result associated with the signaling, and a delayed version of the coefficient multiplication result associated with the signaling; receiving from a memory of the computing device a signaling instruction indicating data encoded using the encoding technique; using the weights to decode the data using the neural network; and writing the decoded data to or reading the decoded data from the memory or storage medium of the computing device.

[0014] Alternatively or additionally, it may further include: determining multiple sets of weights for a neural network using additional encoded data encoded with other encoding techniques, decoded data pairs, and a delayed version based on the result of multiplying additional coefficients of the additional encoded data and decoded data pairs encoded with other encoding techniques, wherein each set of weights corresponds to a different encoding technique; selecting a set of weights associated with an encoding technique; and providing the set of weights to a neural network or another neural network for decoding additional encoded data.

[0015] Alternatively or concurrently, it may further include: receiving additional input data encoded with another encoding technique; selecting a chosen set of weights from a plurality of weight sets associated with the other encoding technique; providing the chosen set of weights to the neural network; and using the chosen set of weights to decode the additional input data.

[0016] Alternatively or concurrently, coding techniques include Reed-Solomon decoding, Bosch-Joherry-Hokvenheim (BCH) decoding, low-density parity-check (LDPC) decoding, polarization decoding, or combinations thereof.

[0017] Alternatively, using a neural network to decode additional encoded data includes weighting the bits of the additional encoded data using at least some of the weights in the set of weights, summing the selected weighted bits of the additional encoded data to provide intermediate data, then weighting the bits of the intermediate data using at least some of the weights and summing the selected weighted bits of the intermediate data and the selected delayed version of the weighted bits of the intermediate data to provide decoded data.

[0018] Alternatively, determining the weight set may involve selecting weights that minimize the error function between the neural network's output and the known decoded data.

[0019] Alternatively, determining the weight set may involve using k-means clustering to determine the center vector for the neural network.

[0020] In another aspect of this disclosure, the method includes mixing encoded data received at a processing unit using multiple coefficients and additional coefficients, both of which are specific to an encoding technique associated with the encoded data. Mixing the encoded data includes: calculating, at a first layer of a MAC unit of a plurality of multiplication / accumulation processing units (MAC units), delayed versions of the encoded data and corresponding outputs of the first layer of the MAC unit using the multiple coefficients to produce a first processing result; calculating, at an additional layer of the MAC unit of the plurality of MAC units, delayed versions of the first processing result and at least a portion thereof using the additional coefficients to produce a second processing result; and providing output data, in part based on the second processing result, representing properly decoded encoded data, such that the output data is an estimate of decoded data based on the encoded data.

[0021] Alternatively, using multiple coefficients to mix the encoded data received at the processing unit further includes delaying the corresponding output of the first layer of the MAC unit at a corresponding delay unit associated with the first layer of the MAC unit to produce a delayed version of the corresponding output of the first layer of the MAC unit.

[0022] Alternatively, calculating a delayed version of the encoded data and the corresponding output of the first layer of the MAC unit at the first layer of the multiple multiplication / accumulation processing units (MAC units) with multiple coefficients includes multiplying the delayed version of the encoded data and the corresponding output of the first layer of the MAC unit with the corresponding coefficients among the multiple coefficients to produce a first processing result.

[0023] Alternatively or additionally, it further includes: obtaining signaling indicating encoded data from coupled memory at the processing unit, the signaling indicating encoded data including an indication from the touchscreen of the mobile communication device using encoding technology.

[0024] Alternatively or concurrently, coding techniques include Reed-Solomon decoding, Bosch-Joherry-Hokvenheim (BCH) decoding, low-density parity-check (LDPC) decoding, polarization decoding, or combinations thereof. Attached Figure Description

[0025] Figure 1A A schematic diagram of a computing system arranged according to the example described in this article.

[0026] Figure 1B A schematic diagram of a neural network arranged according to the examples described in this article.

[0027] Figure 1C A schematic diagram of a recurrent neural network arranged according to the examples described in this article.

[0028] Figure 2A-2C This is a schematic diagram of the processing units arranged in the system according to the example described herein.

[0029] Figure 3 A schematic diagram of the equipment arranged according to the examples described in this article.

[0030] Figure 4 and 5 This is a flowchart of the method for arranging the examples described in this article. Detailed Implementation

[0031] Multilayer neural networks and / or multilayer recurrent neural networks can be used to decode encoded data (e.g., data encoded using one or more encoding techniques). Such neural networks can possess non-linear mapping and distributed processing capabilities, which can be advantageous in many systems employing neural network decoders. In this way, the neural networks described herein can be used to implement error-correcting decoding (ECC) decoders.

[0032] The encoder may have L bits of input data (a1, a2, ..., aL). The encoder may encode the input data using an encoding technique to provide N bits of encoded data (b1, b2, ..., bN). The encoded data may be stored and / or transmitted, or some other action may be taken on the encoded data, which may introduce noise into the data. Therefore, the decoder may receive a version of the N bits of encoded data (x1, x2, ..., xN). The decoder may decode the received encoded data into an estimate of the L bits of raw data (y1, y2, ..., yL).

[0033] Examples of wireless baseband circuitry systems can utilize error-correcting decoding (e.g., low-density parity-check decoding, LDPC). The encoder can add a specific selection of NL bits to the raw L bits of data, which allows the decoder to decode the data and reduces and / or minimizes errors introduced by noise, interference, and other practical factors during data storage and transmission.

[0034] Various error-correcting decoding techniques exist, including low-density parity-check decoding (LDPC), Reed-Solomon decoding, Bosch-Joheri-Hokvenheim (BCH) decoding, and polarization decoding. However, using these decoding techniques may come at the cost of reduced efficiency in frequency and / or channel and / or storage resource utilization, as well as increased processing complexity. For example, using decoding techniques may increase the amount of data that can be stored and / or transmitted. Furthermore, processing resources may be required to implement encoding and decoding. In some instances, the decoder may be one of the processing blocks that consumes the most computational resources in the wireless baseband circuitry system and / or memory controller, which may reduce the desirability of existing decoding schemes in many emerging applications that highly require ultra-low power consumption and ultra-low latency, such as the Internet of Things (IoT) and / or the Internet of Touch.

[0035] The examples described herein utilize multilayer neural networks (NNs), such as multilayer recurrent neural networks (RNNs), to decode encoded data (e.g., data encoded using one or more encoding techniques). NNs and / or RNNs possess non-linear mapping and distributed processing capabilities, which can be advantageous in many systems employing neural network decoders. For example, in some non-volatile memory devices where bit flipping can reduce memory storage capacity, NNs and / or RNNs can provide a more robust decoder that can be trained to transform noisy encoded input data into decoded data (e.g., an estimate of the decoded data). Advantageously, such noisy encoded data can be decoded so that the NN or RNN reduces and / or improves errors that can be introduced by noise present in the input data. In the examples, such noise can be introduced when encoded data is stored in degraded memory (e.g., due to bit flipping).

[0036] Figure 1A This is a schematic illustration of an electronic device 110 arranged in system 100 according to an example described herein. Electronic device 110 is coupled via network 120 to memory 140 or memory 145, each of which can store coefficient data. The coefficient data stored in memory 140 or memory 145 may include coefficient data that can be mixed with noisy encoded input data received by electronic device 110 in the example described herein. The encoded input data may also be stored in memory 140 or 145. Electronic device 110 can obtain the encoded input data (e.g., m encoded input bits) from memory 140 or memory 145 to decode the data to be output into decoded output data.

[0037] Electronic device 110 further includes a processing unit 112 that can interact with mode-configurable control 105, which may be encoded with instructions executable by processing unit 112. In some embodiments, mode-configurable control 105 may be implemented as memory. As used herein, memory may refer to computer-readable media, which may include both storage media and communication media. Examples of computer-readable media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions. Mode-configurable control 105 includes executable instructions 115 for mixing encoded input data with coefficients. For example, mode configuration control 105 may be implemented using a circuit system (e.g., logic), one or more processors, microcontrollers, controllers, or other elements. Mode configuration control 105 may select certain weights and / or other parameters (e.g., from memory 145 that can store weights) and provide said weights and / or other parameters to Figure 2A-2COne or more of the multiplication / accumulation units and / or memory lookup units, details of which will be described in the description Figure 1A-1C This is presented subsequently. In some instances, the weights and / or other parameters stored in memory 145 may be associated with an encoding technique; for example, Reed-Solomon decoding, Bosch-Joheri-Hokvenheim (BCH) decoding, low-density parity-check (LDPC) decoding, polarization decoding, or combinations thereof. During operation, mode configuration control 105 may be used to select the weights and / or other parameters in memory 140 or memory 145 based on the indicated encoding technique. For example, header information or another flag associated with the encoded input data (e.g., obtained from memory 140 or 145) may indicate the encoding technique. In instances, mode configuration control 105 may select weights and / or other parameters from memory 140 or memory 145 to decode the encoded input data into decoded data.

[0038] Processing unit 112 may be implemented using one or more processors, for example, having any number of cores. In some instances, processing unit 112 may include a circuit system (including custom circuit systems) and / or firmware for performing the functions described herein. For example, the circuit system may include multiplication / accumulation units for performing the described functions, as described herein. Processing unit 112 may be of any type, including but not limited to microprocessors or digital signal processors (DSPs) or any combination thereof. For example, processing unit 112 may include cache hierarchies (e.g., L1 and L2 caches), cores, and registers. Example processor cores may include an arithmetic logic unit (ALU), bit manipulation units, multiplication units, accumulation units, adder units, lookup table units, memory lookup units, or any combination thereof. References herein include, for example, […]. Figure 2A , 2B Or 2C describes an instance of processing unit 112.

[0039] The mode-configurable control 105 may, for example, be encoded with executable instructions 115 for mixing encoded input data with coefficient data, for example, to mix noisy encoded input data with coefficient data at processing unit 112. For example, in the context of decoding noisy encoded input data from memory 140 or 145, the executable instructions 115 for mixing encoded input data with coefficient data may include instructions for obtaining noisy encoded input data from memory 140 or 145; and instructions for transforming the obtained noisy encoded input data into decoded data (e.g., an estimate of the decoded data) at processing unit 112. For example, the executable instructions 115 for mixing encoded input data with coefficient data may further include instructions for multiplying a portion of the noisy encoded input data with coefficient data to produce a coefficient multiplication result and accumulating the coefficient multiplication result to further multiply and accumulate another portion of the noisy encoded input data and coefficient data, examples of which are described herein. For example, to produce a coefficient multiplication result, the first layer of the multiplication / accumulation processing unit (MAC unit) can compute noisy encoded input data with multiple coefficients to produce such a coefficient multiplication result, or a first processing result of the first layer of the MAC unit. Continuing with the example, to provide output data, an additional layer of the MAC unit can compute a first processing result with additional coefficients to produce an additional coefficient multiplication result, or a second processing result of an additional layer of the MAC unit. The MLU of the last layer in the additional layers of the MAC unit can provide decoded data based on the second processing result. Therefore, the executable instructions 115 for mixing encoded input data with coefficient data can be included for different types of hardware implementations, such as... Figure 2A-2C The various sets of executable instructions shown in the document are used to implement such a mixture of noisy encoded input data and coefficient data.

[0040] User interface 114 can be implemented using any of a plurality of input devices, including, but not limited to, a touchscreen, keyboard, mouse, microphone, or a combination thereof. User interface 114 can receive input from a user, such as a processing mode selection for specifying a processing mode for processing unit 112. User interface 114 can transmit user input to mode configurable control 105. Example user interface 114 includes a serial interface controller or a parallel interface controller, which can be configured to communicate with external input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.).

[0041] Network 120 may include a wired network or direct wired connection, as well as wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR), and other wireless media.

[0042] Memory 140 and 145 (or mode-configurable control 105, if implemented as memory) can be implemented using any storage medium accessible by processing unit 112. For example, RAM, ROM, solid-state memory, flash memory, disk drive, system memory, optical storage device, or combinations thereof can be used to implement mode-configurable control 105 or memories 140 and 145. When coded data is stored in such memories, environmental or other noise can be introduced during the storage process. For example, noise can be introduced when coded data is stored in degraded (e.g., due to bit flipping) memories 140 or 145. Therefore, data obtained from memories 140 or 145 can be referred to as noisy coded input data. In some embodiments, the mode-configurable control 105 stores coefficients related to the encoding techniques described herein.

[0043] Electronic device 110 can be implemented using any of a variety of computing systems, including, but not limited to, one or more desktop computers, servers, laptops, or other computers. Electronic device 110 typically includes one or more processing units 112. Computing system 100 can be implemented as a mobile communication device using any user communication device, including, but not limited to, desktop computers, laptops, cellular phones, tablets, appliances, automobiles, or combinations thereof. Electronic device 110 may be programmed with a mobile application (e.g., processing unit 112 and computer-readable media encoded with instructions that, when executed, cause electronic device 110 to perform the described functions) for mixing noisy encoded input data with coefficient data. For example, electronic device 110 may be programmed to receive instructions from a touchscreen of a mobile communication device for encoding noisy encoded input data received in 5G wireless transmission.

[0044] It should be understood that the arrangement of the computing system of system 100 can be quite flexible, and although not shown, it should be understood that system 100 may include a number of electronic devices 110, which can be connected via network 120 and operate in combination with each other to perform the systems and methods described herein. In some instances, memory 145 and / or memory 140 may be implemented using the same media, and in other instances, different media may be used. For example, although memory 140 is in Figure 1A The image shows the memory 140 coupled to network 120, but it can be understood that the memory 140 can also be implemented by electronic device 110 as part of mode-configurable control 105. Additionally, although... Figure 1A The image shows a single user interface 114, but it will be understood that the electronic device 110 may further include any number of input devices, output devices, and / or peripheral components. For example, the user interface 114 may be an interface for a mobile communication device.

[0045] Figure 1B This is a schematic illustration of a neural network arranged according to the example described herein. The neural network 150 contains three levels (e.g., layers). Although Figure 1B The example shown has three levels, but in other instances, any number of levels can be used. The first level of neural network 150 contains nodes 168, 169, 172, and 174. The second level of neural network 150 contains combiners 152, 154, 156, and 158. The third level of neural network 150 contains combiners 160, 162, 164, and 166. In other instances, additional, fewer, and / or different components can be used.

[0046] Generally, a neural network comprising multiple node levels can be used. Nodes can be implemented using processing units (e.g., processing unit 112), which can execute one or more functions on input received from a previous level and provide the output of the functions to the next level of the neural network. Processing units can be implemented using, for example, one or more processors, controllers, and / or custom circuit systems, such as application-specific integrated circuits (ASICs) and / or field-programmable gate arrays (FPGAs). In some instances, processing units can use... Figure 2A-2C The neural network described herein may be implemented using any combination of one or more processing units. The processing units may be implemented as combiners and / or summers and / or any other structure for performing functions assigned to the processing units. In some instances, certain elements of the neural network described herein perform weighted sums, for example, implemented using one or more multiplication / accumulation units, which may be implemented using a processor and / or other circuitry. In one example, neural network 150 may be implemented using electronic device 110.

[0047] exist Figure 1B In an example, neural network 150 may have an input layer, which may be the first stage of a neural network containing nodes 168, 169, 172, and 174. Nodes 168, 169, 172, and 174 may implement linear functions that provide input signals (e.g., x1(n), x2(n), ..., xm(n)) to another stage of the neural network (e.g., a 'hidden stage' or 'hidden layer'). Therefore, in Figure 1BIn some instances, m-bit noisy, encoded input data may be provided to the input stage (e.g., input layer) of the neural network during operation. In some instances, the input data may be data encoded according to an encoding technique (e.g., low-density parity-check decoding (LDPC), Reed-Solomon decoding, Bosch-Joherry-Hokvenheim (BCH) and / or polarization decoding). The m-bit input data may be output from the first stage of the neural network 150 to the next stage of the neural network 150. In some instances, the connection between the first and second stages of the neural network 150 may not be weighted; for example, in some instances, processing units in the second stage may receive unaltered bits from the first stage. Each of the input bits may be provided to multiple processing units in the next stage. Although an input layer is shown, in some instances, an input layer may not exist.

[0048] The neural network 150 may have a next layer, which in some instances may be referred to as a 'hidden layer'. The next layer may contain combiners 152, 154, 156, and 158, but any number of elements may be used. Although the processing elements in the second stage of the neural network 150 are referred to as combiners, typically, the processing elements in the second stage can use the input data bits received at the processing element to perform a non-linear activation function. Any number of non-linear activation functions may be used. Examples of functions that may be used include the Gaussian function, for example... Instances of usable functions include quadratic functions, such as f(r) = (r... 2 +σ 2 ) 1 / 2 Instances of usable functions include inverse quadratic functions, such as f(r) = (r... 2 +σ 2 ) -1 / 2 Instances of usable functions include thin-plate spline functions, such as f(r) = r 2 log(r). Instances of usable functions include piecewise linear functions, such as... Instances of usable functions include cubic approximation functions, such as... In these instance functions, σ represents the true parameters (e.g., scaling parameters), and r is the distance between the input vector and the current vector. Distance can be measured using any of a variety of metrics, including the Euclidean norm.

[0049] Each element in the 'hidden layer' can receive selected bits (e.g., some or all) of the input data as input. For example, each element in the 'hidden layer' can receive the outputs of multiple selected elements (e.g., some or all elements) in the input layer as input. For example, each element in the 'hidden layer' can receive the outputs of multiple selected units (e.g., some or all units) in the input layer as input. For example, combiner 152 can receive the outputs of nodes 168, 169, 172, and 174 as input. Although in Figure 1B The example illustrates a single 'hidden layer,' but any number of 'hidden layers' can exist and be connected in series. While four elements are shown in a 'hidden layer,' any number can be used, and it can be the same as or different from the number of nodes in the input layer and / or any other hidden layer. Nodes in a hidden layer can use combinations of data received at the hidden layer nodes (e.g., elements) to evaluate at least one nonlinear function. In this way, hidden layers can provide intermediate data at the output of one or more hidden layers.

[0050] Neural network 150 may have an output layer. Figure 1B The output layer in an example may contain combiners 160, 162, 164, and 166, but any number of elements may be used. Although the processing elements in the output stage of the neural network 150 are referred to as combiners, typically, the processing elements in the output can use data bits received from the last 'hidden layer' in the neural network to perform any combination or other operations. Each element in the output layer may receive selected bits (e.g., some or all) of the data provided by the last 'hidden layer' as input. For example, combiner 160 may receive input from the outputs of combiners 152, 154, 156, and 158. Connections between the hidden layer and the output layer may be weighted. For example, a set of weights W may be specified. In some instances, a weight may exist for each connection between a hidden layer node and an output layer node. In some instances, a weight may exist for each hidden layer node that can be applied to the data provided by said node to each connected output node. Other distributions of weights may also be used. Before providing the output to the output layer, the weights can be multiplied by the output of the hidden layers. In this way, the output layer can perform a weighted summation of the inputs. Therefore, the output of the neural network 150 (e.g., the output of the output layer) can be referred to as a weighted sum. The output layer can thus use weights to combine intermediate data received from one or more hidden layers to provide output data.

[0051] In some instances, the neural network 150 can be used to provide L output bits representing processed data corresponding to m input bits. For example, in Figure 1BIn the example, m input bits (x1(n), x2(n), ... xn) are shown. m (n)) and provides L output bits (y1(n), y2(n), ... y L (n)). The neural network 150 can be trained such that the weights W used by the elements of the hidden layer and / or the functions provided cause the neural network 150 to provide output bits representing processed data corresponding to m encoded input bits. The input bits can be encoded using encoding techniques, and the weights and / or functions provided by the elements of the hidden layer can be selected according to the encoding technique. Therefore, the neural network 150 can be trained multiple times, once for each encoding technique that can be used to provide input data to the neural network 150.

[0052] Examples of trainable neural networks. Training generally refers to the process of determining the weights, functions, and / or other properties to be used by the neural network to generate input data to output data. In some instances, the neural network described herein can be trained to transform encoded input data into decoded data (e.g., an estimate of the decoded data). In some instances, the neural network described herein can be trained to transform noisy encoded input data into decoded data (e.g., an estimate of the decoded data). In this way, the neural network can be used to reduce and / or improve errors that can be introduced by noise present in the input data. In some instances, the neural network described herein can be trained to transform noisy encoded input data into encoded data with reduced noise. The encoded data with reduced noise can then be provided to any decoder (e.g., the neural network and / or other decoders) for decoding the encoded data. In this way, the neural network can be used to reduce and / or improve errors that can be introduced by noise.

[0053] Training can be supervised or unsupervised in various instances, as described herein. In some instances, training can occur using known pairs of expected input and desired output data. For example, training can utilize known pairs of encoded and decoded data to train a neural network to decode subsequent encoded data into decoded data. In some instances, training can utilize known pairs of noisy encoded and decoded data to train a neural network to decode subsequent noisy encoded data into decoded data. In some instances, training can utilize known pairs of noisy encoded and decoded data to train a neural network to provide encoded data with reduced noise compared to input noisy encoded data. Instances of training may include determining, for example... Figure 1BThe weights used in neural networks such as Neural Network 150. In some instances, the same neural network hardware used during training is used as will be used during operation. However, in other instances, different neural network hardware may be used during training, and weights, functions, or other properties determined during training may be stored for use by other neural network hardware during operation.

[0054] An instance of training can be described mathematically. For example, consider the input data at time (n), given as: X(n) = [x1(n), x2(n), ... x m (n)] T The center vector of each element in the hidden layer of neural network 150 (e.g., combiner 152, combiner 154, combiner 156, and combiner 158) can be denoted as C. i (where i = 1, 2, ..., H, and H is the number of elements in the hidden layer).

[0055] The output of each element in the hidden layer can then be given as:

[0056] h i (n)=f i (‖X(n)-C i ‖) where (i=1,2,……,H) (1)

[0057] The connection between the last hidden layer and the output layer can be weighted. Each element in the output layer can have a linear input-output relationship, allowing it to be summed (e.g., weighted summation). Therefore, the output of the i-th element in the output layer at time n can be written as:

[0058]

[0059] Where (i = 1, 2, ..., L), and where L is the number of output elements of the output layer, and W ij The connection weights are the j-th element in the hidden layer and the i-th element in the output layer.

[0060] Typically, neural network architectures (e.g., Figure 1B The neural network (150) may contain multiple elements and may have a center vector distributed in the input domain, such that the neural network can approximate a nonlinear multidimensional function and therefore approximate the inverse mapping between two codewords (e.g., from an N-bit input to an L-bit output). Typically, the choice of transfer function used by the elements in the hidden layers does not affect the mapping performance of the neural network, and therefore, in some instances, functions that are readily implementable in hardware can be used. For example, thin-plate spline functions and / or Gaussian functions can be used in a variety of instances and both provide sufficient approximation capabilities. Other functions may also be used.

[0061] Therefore, an instance of a neural network can be specified by attributes (e.g., parameters). In some instances, two sets of parameters can be used to specify the neural network: connection weights and a center vector (e.g., a threshold). The parameters can be determined from the selected input data (e.g., encoded input data) by solving an optimization function. An instance optimization function can be given as:

[0062]

[0063] Where M is the number of trained input vectors (e.g., trained encoded data inputs) and Y(n) is the output vector calculated from the sample input vectors using equations (1) and (2) above, and This corresponds to the desired (e.g., known) output vector. The output vector Y(n) can be written as:

[0064] Y(n)=[y1(n),y2(n),…………y L (n)] T

[0065] Various methods (e.g., gradient descent) can be used to solve optimization functions. However, in some instances, another method can be used to determine the parameters of a neural network, which may typically involve two steps: (1) determining the center vector C. i (i = 1, 2, ..., H), and (2) determine the weights.

[0066] In some instances, center vectors can be selected from a subset of available sample vectors. In such instances, the number of elements in the hidden layer can be relatively large to cover the entire input domain. Therefore, in some instances, it may be necessary to apply the k-means clustering algorithm. Typically, the k-means clustering algorithm distributes center vectors according to the natural measurement of attractors (e.g., the density of centers if the density of data points is high). The k-means clustering algorithm finds a set of cluster centers and partitions the training samples into subsets. Each cluster center can be associated with one of the H hidden layer elements in this network. The data can be partitioned in such a way that training points are assigned to the cluster with the nearest center. The cluster centers correspond to one of the minimum values ​​of the optimization function. An example optimization function used with the k-means clustering algorithm can be given as:

[0067]

[0068] Among them B jn This is the clustering partition or membership function that forms the HxM matrix. Each column can represent an available sample vector (e.g., known input data), and each row can represent a cluster. Each column can contain a single '1' in the row corresponding to the cluster closest to the training point, and zeros elsewhere.

[0069] The center of each cluster can be initialized with a different randomly selected training point. Then, each training instance can be assigned to the element closest to it. Once all training points have been assigned, the average position of the training points in each cluster can be determined, and the cluster center moves to that point. The cluster can then become the desired center for the hidden layer elements.

[0070] In some instances, for certain transfer functions (e.g., Gaussian functions), the scaling factor σ can be determined and may be determined before the connection weights are determined. The scaling factor can be chosen to cover the training points to allow for a smooth fit to the desired network output. Typically, this means that any point within the convex hull of the processing element center can significantly activate more than one element. To achieve this, each hidden layer element can significantly activate at least one other hidden layer element. A suitable method for determining the scaling parameter σ can be based on the P-nearest neighbor heuristic, which can be given as...

[0071] Where C j (where i = 1, 2, ..., H) is C i The nearest neighbor of P.

[0072] Connection weights can be determined additionally or alternatively during training. In instances of neural networks, for example... Figure 1B The neural network 150, which has a weighted hidden layer and output elements of summing units, can be transformed into a linear least squares problem once the center vector and scaling parameters are determined. The linear least squares problem can be written as follows:

[0073]

[0074] Where W = {W ij Let} be an LxH matrix of connection weights, and F be an HxM matrix of the outputs of the hidden layer processing elements, with its matrix elements using F. in =f i (‖X(n)-C i The calculation is performed using the formula (i = 1, 2, ..., H; n = 1, 2, ..., M), and... Let LxM be the desired (e.g., known) output matrix. The connection weight matrix W can be found from equation (5) and can be written as follows:

[0075]

[0076] Where F +This is the pseudoinverse of F. In this way, the above provides a batch processing method for determining the connection weights of a neural network. It can be applied, for example, to cases where all sets of input samples are available simultaneously. In some instances, each new set of samples can become available recursively, for example, in the Recursive Least Squares (RLS) algorithm. In such cases, the connection weights can be determined as follows.

[0077] First, the connection weights can be initialized to any value (e.g., random values ​​can be used). The output vector Y(n) can be computed using equation (2). The error term e for each output element in the output layer can be computed as follows. i (n):

[0078]

[0079] The connection weights can then be adjusted based on the errors, for example, as follows:

[0080] W ij (n+1)=W ij (n)+γe i (n)f j (‖X(n)-C i ‖) (7)

[0081] (i=1,2,…,L; j=1,2,…,M)

[0082] Where γ is a learning rate parameter that can be fixed or time-varying.

[0083] The total error can be calculated based on the output from the output layer and the desired (known) data:

[0084]

[0085] The process can be iterated by recalculating the new output vector, the error terms, and readjusting the connection weights. The process can continue until the weights that reduce the error to be equal to or less than the threshold error are identified.

[0086] therefore, Figure 1BThe neural network 150 can be trained to determine parameters (e.g., weights) for use by the neural network 150 to perform a specific mapping between input and output data. For example, the neural network 150 can be trained with a set of parameters to be used when decoding encoded data encoded using an encoding technique (e.g., Low-Density Parity-Check Decoding (LDPC), Reed-Solomon Decoding, Bosch-Joherry-Hokvenheim (BCH) and / or Polarization Decoding). For example, the neural network 150 (and / or another neural network) can be trained multiple times using different known input / output data pairs. Multiple trainings can result in multiple sets of connection weights. For example, a different set of weights can be determined for each of the multiple encoding techniques; for instance, one set of weights can be determined for use with decoding LDPC encoded data, and another set of weights can be determined for use with decoding BCH encoded data.

[0087] To recap, this is provided solely through examples. Figure 1B The structure of neural network 150 is described. Other multilayer neural network structures can be used in other instances. For example, see the following section about... Figure 1C The described method utilizes a multi-layer recurrent neural network structure as disclosed herein. Furthermore, the training process described herein is provided with the aid of examples. For instance, other training techniques (e.g., learning algorithms) can be used to solve local minima and / or vanishing gradient problems. The determined weights and / or vectors for each decoder can be obtained through an offline learning mode of the neural network, which can advantageously provide more resources and data. In some implementations, such as those described in the reference... Figure 2A-2C As described, the hidden layers of combiners (e.g., combiners 152, 154, 156, and 158) may include multiplication / accumulation (MAC) units, with each layer having an additional MAC unit. Such embodiments of intermediate processing results accumulated in corresponding processing elements (e.g., corresponding MAC units) may also include a memory lookup (MLU) unit configured to retrieve multiple coefficients and provide said multiple coefficients as connection weights (e.g., determined weights) of the corresponding layers of the MAC units to be mixed with the input data.

[0088] In a supervised learning instance, the input training samples are: [x1(n), x2(n), ..., x... m [b1(n), b2(n), ..., bn(n)] can be obtained by encoding samples [b1(n), b2(n), ..., bn(n)]. m [b(n)] is generated through some noisy channels and / or by adding noise. Supervised output samples can be used to generate [b1(n), b2(n), ..., bn] by the encoder. m The corresponding original code for [a1(n), a2(n), ... a2(n)] is [a1(n), a2(n), ... a2(n)]. L(n)]. Once these parameters are determined in offline mode, the desired codeword to be decoded can be obtained from the input data using a neural network (e.g., calculating equation (2)), which avoids the complex iterative and feedback decisions used in traditional error-correction decoding algorithms. In this way, the neural network described herein can provide a reduction in processing complexity and / or latency, because some of the complexity has been transferred to the offline training process for determining the weights and / or functions to be used. Furthermore, the same neural network (e.g., Figure 1B The neural network (150) can be used to decode input codewords encoded from any of a plurality of error-correcting encoders by selecting different weights obtained through training for the error correction technique employed. In this way, the neural network can act as a universal decoder for multiple encoder types.

[0089] Figure 1C This is a schematic illustration of a recurrent neural network arranged according to the example described herein. The neural network 170 comprises three stages (e.g., layers): input node 171; combiner stages 173 and 175; and output node 177. Although in Figure 1C The example shows three levels, but in other instances, any number of levels can be used, for example, as shown in the reference. Figure 2A-2C As described. In some implementations, the neural network 170 may have multiple combiner stages, such that the output from one combiner stage is provided to another combiner stage, until it is provided to the output node 177. (See reference...) Figure 2A As described, for example, multiple combiner stages may exist in a neural network 170. (e.g.) Figure 1C As depicted herein, delay units 175a, 175b, and 175c may be optional components of neural network 170. When utilizing such delay units 175a, 175b, and 175c as described herein, neural network 170 may be referred to as a recurrent neural network.

[0090] The first stage of the neural network 170 includes an input node 171. The input node 171 can receive input data at various input points of the recurrent neural network. In some instances, the input node 171 may contain multiple input nodes, for example... Figure 1B The input nodes are 168, 169, 172, and 174. The second stage of the neural network 170 is a combiner stage, which includes combiner units 173a, 173b, and 173c; and delay units 175a, 175b, and 175c. Therefore, combiner unit 173 and delay unit 175 can be collectively referred to as the combiner stage. In some embodiments, combiner units 173a, 173b, and 173c may, for example, correspond to... Figure 1B Combiners 152, 154, and 156. Therefore, as regarding... Figure 1BAs described, processing unit 112 implements such a combiner. Typically, processing unit 112, which implements combiner units 173a-c and delay units 175a-c in the second stage, can use input data from input node 171 (e.g., input signals X1(n), X2(n), and X3(n)) to perform a nonlinear activation function. The third stage of neural network 170 includes output node 177. In some instances, output node 177 may contain... Figure 1B Combiners 160, 162, and 164 are used. Therefore, in some instances, output node 177 may be referred to as a combiner level. In other instances, additional, fewer, and / or different components may be used.

[0091] The recurrent neural network 170 includes delay units 175a, 175b, and 175c, which generate delayed versions of the output from the respective combiner units 173a-c based on such output data received from the combiner units 173a-c. In an example, the output data of the combiner units 173a-c can be represented as h(n); and therefore, each of the delay units 175a-c delays the output data of the combiner units 173a-c to produce a delayed version of the output data from the combiner units 173a-c, which can be represented as h(nt). In various embodiments, the delay amount t can also be different, for example, one clock cycle, two clock cycles, or one hundred clock cycles. That is, the delay unit 175 can receive a clock signal and use the clock signal to identify the delay amount. Figure 1C In the example, the delayed version is delayed by one time period, where '1' represents the time period. The time period can correspond to any number of time units, such as a time period defined by a clock signal or a time period defined by another element of the neural network 170.

[0092] exist Figure 1CContinuing with the example, each delay unit 175a-c provides a delayed version of the output data from combiner units 173a-c as input to combiner unit 173a-c, optionally as a recurrent neural network operation. Such delay units 175a-c may provide corresponding delayed versions of the output data from nodes of combiner units 173a-c to corresponding input units / nodes of combiner unit 173a-c. While utilizing the delayed versions of the output data from combiner units 173a-c, the recurrent neural network 170 may train weights at combiner units 173a-c, said weights incorporating the time-varying aspects of the input data to be processed by such recurrent neural network 170. In some instances, once trained, input node 171 receives encoded input data to be decoded in recurrent neural network 170. For example, each stream of input data may correspond to a different obtained set of encoded data representing a time signal. Because RNN 170 incorporates delayed versions of the output data from combiner units 173a-c, these delayed versions provide feedback information representing temporal properties. RNN 170 provides outputs that are decoded more quickly, incorporating these temporal properties into the decoded data used to compute the output. In this example, the decoded output data may represent an estimate of the encoded data decoded from the original data encoded according to an associated coding technique.

[0093] Typically, a recurrent neural network may contain multiple node levels. Nodes can be implemented using processing units (e.g., processing unit 112), which can execute one or more functions on input received from a previous level and provide the output of the functions to the next level of the recurrent neural network. Processing units can be implemented using, for example, one or more processors, controllers, and / or custom circuit systems, such as application-specific integrated circuits (ASICs) and / or field-programmable gate arrays (FPGAs). In some instances, processing units can use... Figure 2A-2C The processing units 112 described herein may be implemented in any combination. The processing units may be implemented as combiners and / or summers and / or any other structure for performing functions assigned to the processing units. In some instances, certain elements of the neural network described herein perform weighted sums, for example, implemented using one or more multiplication / accumulation units, which may be implemented using a processor and / or other circuitry. In one instance, the neural network 170 may be implemented by electronic device 110 using... Figure 2A-2C The described one or more processing units may be used in any combination to implement this.

[0094] This can be mathematically described as an instance of training and inference in a recurrent neural network. Again, as an example, consider the input data at time (n), given as: X(n) = [x1(n), x2(n), ... x... m (n)] T Each element in the hidden layer of the recurrent neural network 170 (e.g., containing...) Figure 1B The center vector of combiner unit 173 (combiner unit 152, combiner 154, combiner 156 and combiner 158) can be denoted as C. i (where i = 1, 2, ..., H, and H is the number of elements in the hidden layer).

[0095] The output of each element in the hidden layer can then be given as:

[0096] h i (n)=f i (‖X(n)+h i (nt)-C i ‖) where (i=1,2,……,H) (9)

[0097] t can be a delay at delay unit 175, such that the output of combiner unit 173 contains a delayed version of the output of combiner unit 173. In some instances, this may be referred to as feedback of combiner unit 173. Therefore, each of the connections between the last hidden layer and the output layer can be weighted. Each element in the output layer can have a linear input-output relationship, such that it can be summed (e.g., weighted summation). Therefore, the output of the i-th element in the output layer at time n can be written as:

[0098]

[0099] Where (i = 1, 2, ..., L), and where L is the number of output elements of the output layer, and W ij The connection weights are the j-th element in the hidden layer and the i-th element in the output layer.

[0100] Alternatively or concurrently, although a single stage (e.g., a second stage) of a combiner comprising combiner units 173a-c and delay units 175a-c has been described, Figure 1C However, it is understood that multiple levels similar to the combiner level can be contained in a neural network 170 with different types of combiner units and different types of delay units with different delays, for example, as will now be referred to. Figure 2A-2C As described. Furthermore, the same neural network (e.g., Figure 1CThe recurrent neural network 170 can be used to decode input codewords encoded from any of a plurality of error-correcting encoders by selecting different weights obtained through training for the adopted error correction technique. In this way, the recurrent neural network can act as a general decoder for multiple encoder types, similar to the neural network 150. Advantageously, the recurrent neural network 170 utilizes a delayed version of the output data from the combiner units 173a-c to provide feedback information representing temporal properties. Thus, for example, the RNN 170 can provide output data that is decoded faster than the neural network 150. For example, if the encoding technique is susceptible to noise (e.g., bit flipping) during storage in memory 140 or memory 145, the resulting encoded data to be decoded by the RNN 170 may contain noise that produces time-varying effects (e.g., temporal properties) on the resulting encoded data. Therefore, the feedback information included in the delayed version of the output data can improve the efficiency of RNN 170 (e.g., compared to neural network 150) to decode the data into decoded data, such as an estimate of encoded data that has been decoded according to a decoding technique complementary to the encoding technique.

[0101] Figure 2A This is a schematic illustration of a processing unit 112 arranged in system 200 according to an example described herein. Such hardware implementations (e.g., system 200) can be used, for example, to implement one or more neural networks, such as... Figure 1C The recurrent neural network 170. Alternatively, in some embodiments, the processing unit 112 may receive input data 210a, 210b, and 210c from such a computing system. The input data 210a, 210b, and 210c may be encoded data obtained from a sensor or data stored in memory 145. In some instances, encoded data stored in memory 140 or memory 145 may be obtained as input data to be provided to the electronic device 110 in which the processing unit 112 is implemented. Therefore, the obtained input data may be m-bit noisy encoded input data provided to the input stage (e.g., input layer) of the processing unit, where input data 210a X1(i) corresponds to the first bit; input data 210b X2(i) corresponds to the second bit; and input data 210c X... m(i) Corresponding to the m-th bit. In some instances, the received input data may be data encoded according to an encoding technique (e.g., low-density parity-check decoding (LDPC), Reed-Solomon decoding, Bosch-Joherry-Hockvanheim (BCH) and / or polarization decoding). Alternatively, in some embodiments, the input data may be noisy due to noise (e.g., bit flips) experienced during the storage of the encoded data in memory 140 or memory 145. Therefore, one or more processing units 112 may implement decoding functionality or decoding operations for the received noisy encoded input data at the electronic device 110.

[0102] As indicated in the representation of the input data signal, the input data 210a X1(i,i-1) comprises the current portion of the input data at time i and the previous portion of the input data at time i-1. For example, the current portion of the input data may be a sample obtained at a specific time period (e.g., at time i), while the previous portion of the input data may be a sample obtained at a time period prior to the specific time period (e.g., at time i-1). Therefore, the previous portion of the input data may be referred to as a time-delayed version of the current portion of the input data. For example, the portion of the input data at each time period can be obtained in vector or matrix format. In an example, the current portion of the input data at time i may be a single value; and the previous portion of the input data at time i-1 may be a single value. Therefore, the input data 210a X1(i,i-1) may be a vector. In some instances, the current portion of the input data at time i may be a vector value; and the previous portion of the input data at time i-1 may be a vector value. Therefore, the input data 210a X1(i,i-1) may be a matrix.

[0103] Processing unit 112 may include multiplication / accumulation (MAC) units 212a-c, 216a-b and 220; delay units 213a-c, 217a-b and 221; and memory lookup units (MLUs) 214a-c, 218a-b and 222, which can produce output data (e.g., B(1)) 230 when mixed with input data obtained from memory 145. Each set of MAC units and MLU units with different element numbers may be referred to as the corresponding level of the combiner acting on processing unit 112. For example, the first level of the combiner includes MAC units 212a-c and MLUs 214a-c that combine the operation of delay units 213a-c to form a first level or "layer", as per [reference to...]. Figure 1BThe referenced layer has "hidden" layers as various combiner levels. Continuing with the example, the second level of the combiner includes MAC units 216a-b and MLU 218a-b that combine the operations of delay units 217a-b to form a second level or a second layer of hidden layers. Furthermore, the third level of the combiner can be a single combiner containing MAC unit 220 and MLU 222 that combine the operations of delay unit 221 to form a third level or a third layer of hidden layers.

[0104] In instances where the input data is encoded according to an encoding technique, in some instances, the output data 230B(1) may be decoded data (e.g., an estimate of the decoded data) corresponding to the encoded input data. For example, the output data may be data corresponding to the encoded input data but with reduced and / or modified noise. In operation, the processing unit 112 may provide instructions 115 stored in the mode-configurable control 105 to cause the processing unit 112 to configure multiplication units 212a-c, 216a-c, and 220 to multiply and / or accumulate the input data 210a, 210b, and 210c and the delayed versions of the processing results from delay units 213a-c, 217a-b, and 221 (e.g., the corresponding outputs of the corresponding layers of the MAC unit) with coefficient data to produce output data 230B(1). For example, mode configurable control 105 can execute instructions that cause memory 145 to provide weights and / or other parameters associated with encoding techniques stored in memory 145 to MLUs 214a-c, 218a-b, and 222 as weights for MAC units 212a-c, 216a-b, and 220, and delay units 213a-c, 217a-b, and 221. During operation, mode configuration control 105 can be used to select weights and / or other parameters in memory 145 based on the indicated encoding technique for processing unit 112.

[0105] As indicated in the representation of the corresponding outputs of the corresponding layers of the MAC units (e.g., the outputs of MLUs 214a-c, 218a-b, and 222), the input data to each MAC unit 212a-c, 216a-b, and 220 comprises the current portion of the input data at time i and a delayed version of the processing result at time i-1. For example, the current portion of the input data may be a sample obtained at a specific time period (e.g., at time i), while the delayed version of the processing result (e.g., at time i-1) may be obtained from the outputs of delay units 213a-c, 217a-b, and 221, representing the time period prior to the specific time period (e.g., as a result of the introduced delay). Therefore, when using such input data obtained from both the current period and at least one previous period, the output data 230B(1) can represent a Markov process, such that the causal relationship between at least the data from the current time period and the previous time period can improve the accuracy of the weight estimation used for training coefficient data to be utilized by the MAC unit and MLU of processing unit 112 or for inference of encoded input data or noisy encoded input data by processing unit 112. Thus, when utilizing delayed versions of the output data from 213a-c, 217a-b, and 221, the recurrent neural network 170 provides decoded data (e.g., an estimate of the decoded data), which may have reduced and / or modified noise. Therefore, the recurrent neural network 170 can operate more efficiently to decode noisy encoded input data, such as data that may have been stored in memory cells of a degraded memory device in which bit flips have occurred.

[0106] In an instance of executing such instruction 115 for mixing encoded input data with coefficients, at the first layer of MAC units 212a-c and MLUs 214a-c, the multiplication / accumulation unit 212a-c is configured to multiply or accumulate at least two operands from corresponding input data 210a, 210b, or 210c with operands from corresponding delay units 213a-c to produce a multiplication result provided to MLUs 214a-c. For example, the multiplication / accumulation unit 212a-c may perform a multiplication-accumulation operation such that three operands M, N, and T are multiplied and then added to P to produce a new version of P stored in its corresponding MLU 214a-c. Therefore, MLU 214a latches the multiplication result until the stored multiplication result is provided to the next layer of the MAC unit. MLUs 214a-c, 218a-b, and 222 can be implemented by any number of processing elements operating as memory lookup units, such as D, T, SR, and / or JK latches.

[0107] Figure 2AThe MLUs 214a-c, 218a-b, and 222 shown can typically perform a predetermined nonlinear mapping from input to output. For example, MLUs 214a-c, 218a-b, and 222 can be used to evaluate at least one nonlinear function. In some instances, the contents and sizes of the various MLUs 214a-c, 218a-b, and 222 depicted can differ and can be predetermined. In some instances, Figure 2A One or more of the MLUs 214a-c, 218a-b, and 222 shown can be replaced by a single merged MLU (e.g., table lookup). Examples of nonlinear mappings (e.g., functions) that can be performed by MLUs 214a-c, 218a-b, and 222 include Gaussian functions, piecewise linear functions, sigmoid functions, thin-plate spline functions, multiple quadratic functions, cubic approximations, and inverse multiple quadratic functions. References have also been made. Figure 1B Instances of the description function. In some instances, the selected MLUs 214a-c, 218a-b, and 222 can be bypassed and / or deactivated, which allows the MLU and its associated MAC unit to be treated as unity-gain elements.

[0108] Additionally, in this example, MLU 214a provides the processing result to delay unit 213a. Delay unit 213a delays the processing result (e.g., h1(i)) to produce a delayed version of the processing result (e.g., h1(i-1)), which is then output as operand T to MAC unit 212a. Although Figure 2A The delay units 213a-c, 217a-b, and 221 are depicted as introducing a delay of '1', but it will be understood that different amounts of delay can be introduced into the output of the first layer of the MAC unit. For example, a clock signal that introduces a sample delay of '1' (e.g., h1(i-1)) can alternatively introduce a sample delay of '2', '4', or '100'. In various embodiments, delay units 213a-c, 217a-b, and 221 can correspond to any number of processing units that can use clock signals or other time-oriented signals to introduce delays into the processing circuitry system, such as flip-flops (e.g., D flip-flops) and / or one or more various logic gates (e.g., AND, OR, NOR, etc.) that operate as delay units.

[0109] In an example of the first hidden layer of a recurrent neural network, MLU 214a-c may retrieve coefficient data stored in memory 145, which may be weights associated with the weights of the first layer to be applied to the MAC unit for both data from the current cycle and data from previous cycles (e.g., a delayed version of the first layer processing result). For example, MLU 214a may be a lookup table for retrieving one or more coefficients to be applied to both operands M and N (e.g., specific coefficients associated with a first frequency) and additional coefficients to be applied to operand T. MLU 214a-c also provides the resulting multiplication processing result to the next layer of MAC units 216a-b and MLU 218a-b. Additional layers of MAC units 216a, 216b and MAC unit 220, which operate in conjunction with MLU 218a, 218b and MLU 222 respectively, may continue to process the multiplication result to produce output data 230B (1). Using this type of circuit system arrangement, output data 230B (1) can be generated from input data 210a, 210b and 210c.

[0110] Advantageously, for example, compared to Figure 2B The processing unit 112 of the system 200 may utilize a reduced number of MAC units and / or MLUs. Each subsequent layer may have a reduced portion of MAC units, delay units, and MLUs. As depicted, in Figure 2A For example, when m=3, the second layer of MAC units 216a-b, delay units 217a-b, and MLU 218a-b can contain m-1 MAC units and MLUs. Therefore, the last layer in processing unit 112 (containing MAC unit 220, delay unit 221, and MLU 222) contains only one MAC, one delay unit, and one MLU.

[0111] For example, coefficient data from memory 145 can be mixed with input data 210a-210c and a delayed version of the processing result to produce output data 230B(1). For instance, the relationship between the coefficient data based on input data 210a-c and the delayed version of the processing result and the output data 230B(1) can be expressed as:

[0112]

[0113] Where a (m) a (m-1) a 1The coefficients are for the first layer of multiplication / accumulation units 212a-c and the outputs of delay units 213a-c, respectively; the second layer of multiplication / accumulation units 216a-b and the outputs of delay units 217a-b; and the last layer of multiplication / accumulation unit 220 and the output of delay unit 221; and where f(·) is a mapping relationship that can be performed by memory lookup units 214a-c and 218a-b. As described above, memory lookup units 214a-c and 218a-b retrieve coefficients to mix with the corresponding delayed versions of the input data and each layer of the MAC unit. Therefore, output data can be provided by manipulating the input data and the delayed versions of the MAC unit with the corresponding multiplication / accumulation units using the coefficient set stored in memory. The coefficient set can be associated with the desired encoding technique. The resulting mapped data can be manipulated by additional multiplication / accumulation units and additional delay units using additional coefficient sets stored in memory associated with the desired encoding technique. The set of coefficients multiplied at each stage of the processing unit 112 can represent or provide an estimate of the processing of input data in specially designed hardware (e.g., an FPGA).

[0114] Furthermore, it can be shown, as expressed by equation (11), that in some instances, system 200 can approximate any nonlinear mapping with arbitrarily small errors, and the mapping of system 200 can be determined by coefficient a. (m) a (m-1) a 1 Determined. For example, if such coefficient data is specified, any mapping and processing between input data 210a-210c and output data 230 can be implemented by system 200. For example, the coefficient data can represent a nonlinear mapping from input data 210a-c to output data B(1)230. In some instances, the nonlinear mapping of the coefficient data can represent a Gaussian function, a piecewise linear function, a sigmoid function, a thin-plate spline function, a multiple quadratic function, a cubic approximation, an inverse multiple quadratic function, or a combination thereof. In some instances, some or all of the memory lookup units 214a-c, 218a-b can be deactivated. For example, one or more of the memory lookup units 214a-c, 218a-b can operate as gain units with unity gain. Such relationships, as derived from the circuit system arrangement depicted in system 200, can be used to train the entity of computation system 200 to generate coefficient data. For example, using equation (11), the entity of computation system 200 can compare input data with output data to generate coefficient data.

[0115] Each of the multiplication / accumulation units 212a-c, 216a-b, and 220 may contain multiple multipliers, multiple accumulators, and / or multiple adders. An ALU can be used to implement any of the multiplication / accumulation units 212a-c, 216a-b, and 220. In some instances, any of the multiplication / accumulation units 212a-c, 216a-b, and 220 may contain one multiplier and one adder, each performing multiple multiplications and multiple additions, respectively. The input-output relationship of the multiplication / accumulation units 212a-c, 216a-b, and 220 can be represented as follows:

[0116]

[0117] Where "I" represents the number of times the multiplication is performed in the unit, and C i For coefficients that can be accessed from, for example, memory 145, and B in (i) represents the factors from input data 210a-c or the outputs from multiplication / accumulation units 212a-c, 216a-b, and 220. In this example, the output B of the set of multiplication / accumulation units... out Equal to the sum of coefficient data C i The output B of another set of multiplication / accumulation units. in (i). B in (i) can also be input data, such that the output B of the set of multiplication units / accumulation units out Equal to the sum of coefficient data C i Multiply by the input data.

[0118] Therefore, system 200 (e.g., a hardware implementation of recurrent neural network 170) can be used to process input codewords (e.g., x1(n), x2(n), ... x m (n) is converted to the output codeword (e.g., B(1)). References have been made in this paper. Figure 1C Describe an example of the transformation. For instance, the input codeword may correspond to noisy encoded input data. A hardware implementation of a recurrent neural network 170 may utilize multiplication with corresponding weights (e.g., weights obtained during training) and the MLU to provide an output codeword. The output codeword may correspond to a version of the decoded data and / or encoded input data with reduced and / or altered noise.

[0119] The mode configuration control 105 can be implemented using a circuit system (e.g., logic), one or more processors, microcontrollers, controllers, or other components. The mode configuration control 105 can select certain weights and / or other parameters from memory 145 and provide said weights and / or other parameters to... Figure 2AOne or more of the MAC units and / or MLUs. In some instances, weights and / or other parameters stored in memory 145 may be associated with an encoding technique. During operation, mode configuration control 105 may be used to select weights and / or other parameters associated with an encoding technique (e.g., Reed-Solomon decoding, BCH decoding, LDPC decoding, and / or polarization decoding) in memory 145. The hardware implementation of recurrent neural network 170 may then utilize the selected weights and / or other parameters to act as a decoder for data encoded with said encoding technique. Mode configuration control 105 may select different weights and / or other parameters associated with different encoding techniques stored in memory 145 to change the operation of the hardware implementation of recurrent neural network 170 to act as a decoder for different encoding techniques. In this way, the hardware implementation of recurrent neural network 170 can flexibly act as a decoder for multiple encoding techniques.

[0120] Figure 2B This is a schematic illustration of a processing unit 112 arranged in system 250 according to an example described herein. Such hardware implementations (e.g., system 250) can be used, for example, to implement one or more neural networks, such as... Figure 1C The recurrent neural network 170. Alternatively, in some embodiments, the processing unit 112 may receive input data 210a, 210b, and 210c from the computing system. Figure 2B Elements described similarly can be as described regarding Figure 2A As described, but may also include, for example, information about... Figure 2B Additional features described. For example, Figure 2B Descriptions can be as follows: Figure 2A The MAC units 262a-c and 263a-c operate as described by the MAC units 212a-c and delay units 213a-c. Therefore, their digital indicators are relative to... Figure 2A Offset 50 Figure 2B The components similarly include those of processing element 112; for example, MAC unit 266a operates similarly to MAC unit 216a. The system 250 including processing element 112 also includes components not in... Figure 2A Additional features highlighted in the processing element 112. For example, Figure 2B The processing unit 112 further includes MAC units 266c and 270b-c; delay units 267c and 271b-c; and MLUs 268c and 272b-c, such that the output data is provided as 275a-c, rather than separately in Figure 2AProvided as B(1)230. Advantageously, the system 250, including processing element 112, can process input data 260a-c to produce output data 275a-c with greater accuracy. For example, the output data 275a-c can be processed by additional coefficients retrieved at MLU 268c and multiplied and / or accumulated by additional MAC units 266c and 270b-c and additional delay units 267c and 271b-c. For example, such additional processing can yield more accurate output data about providing an estimate as decoded data (e.g., compared to from...). Figure 2A The processing unit 112 obtains the output data. In embodiments where board space (e.g., printed circuit board) is not a major factor in the design, compared to Figure 2A The implementation scheme of the processing unit 112, Figure 2B The implementation of the processing unit 112 may be as desired; in some implementations, due to having a higher... Figure 2B The processing unit 112 has fewer components. Figure 2A The implementation of the processing unit described above can occupy less board space.

[0121] Although Figure 2A and 2B In the context of this document, processing element 112 is described as a single processing element 112, but features can also be implemented in... Figure 1A In the processing element 112, so that Figure 2A The description of a single processing element 112 in 2B is interchangeable with the processing element implemented in the examples described herein. Therefore, it can be understood that although the reference... Figure 2A Alternatively, a single processing element 112 may be present in 2B, but electronic device 110 may have multiple processing elements 112, similar to... Figure 1A , and also has such as about Figure 2A Or the same features described by a single processing element 112 of 2B.

[0122] Figure 2C This is a schematic illustration of a processing unit 112 arranged in system 280 according to an example described herein. Such hardware implementations (e.g., system 280) can be used, for example, to implement one or more neural networks, such as... Figure 1C The recurrent neural network 170. Alternatively, in some embodiments, the processor unit 112 may be implemented as... Figure 1A Processing unit 112 in the example. Figure 2C Elements described similarly can be as described regarding Figure 2B The operation described differs in that Figure 2B The delay units are 263a-c, 267a-c, and 271a-c. For example, Figure 2C Descriptions can be as follows: Figure 2B The MAC unit 282a-c and delay unit 283a-c operate as described by the MAC unit 262a-c and delay unit 263a-c. Therefore, their digital indicators are relative to... Figure 2B Offset 20 Figure 2C The components similarly include the components of processing element 112; for example, MAC unit 286a operates similarly to MAC unit 266a.

[0123] The system 280, which includes processing element 112, also includes elements not in... Figure 2B Additional features highlighted in the processing element 112. Unlike Figure 2B , Figure 2C Delay units 283a, 283b, and 283c are depicted. Therefore, Figure 2C The processing unit 112 may include different arrangements of the inputs and outputs of the delay units, as illustrated by delay units 283a, 283b, and 283c. For example, the output of MLU 288b may be provided to delay unit 283b to produce a delayed version of the processing result from the second layer of the MAC unit, as input to the first layer of the MAC unit, for example, as input to MAC unit 282b. Therefore, the processing unit 112 of system 280 is illustrative; the delayed version of the processing result may be provided as input to other hidden layers, unlike... Figure 2B The processing unit 112 of the system 250, the Figure 2B The corresponding delayed version is provided as input to the same layer in which the delayed version is generated (e.g., the output of MLU 268b is provided to delay unit 267b to generate a delayed version for MAC unit 266b in the same layer as the result of processing its output). Thus, in this example, even output B(n) 295c can be provided from the last hidden layer to the first hidden layer (e.g., as input to MAC unit 282c).

[0124] Advantageously, for example, compared to Figure 2A Alternatively, the processing unit 112 of type 2B can be used as input to provide processing results to different or additional hidden layers. Such a delayed version can better compensate for implementation. Figure 2C"Higher-order" memory effects in the recurrent neural network 170 of one or more processing units 112. For example, higher-order memory effects model the effects of leading and lagging envelope signals used during the training of the recurrent neural network 170 to transform noisy encoded input data obtained at processing unit 112 into decoded data (e.g., an estimate of the decoded data). The acquired encoded input data, which has noise (e.g., bit flips) while stored in memory 140 or memory 145 or other noise experienced during storage (e.g., shot or voltage noise), can be noisy encoded input data to be decoded by the recurrent neural network 170. In an example, the recurrent neural network 170 that estimates the output data may contain different delayed versions of the processing results corresponding to such leading and lagging envelopes (e.g., various envelopes of encapsulated noise). Thus, processing unit 112 is implemented with such higher-order memory effects, for example, for inference of the recurrent neural network 170 to provide output data 295a-c based on input data 281a-c.

[0125] Although Figure 2A , 2B In the context of 2C, processing element 112 is described as a single processing element 112, but features can also be implemented in Figure 1A In the processing element 112, so that Figure 2A , 2B The description of the single processing element 112 in 2C is interchangeable with the processing element implemented in the examples described herein. Therefore, it can be understood that although the reference... Figure 2A , 2B The electronic device 110 may have multiple processing elements 112, similar to a single processing element 112 in 2C. Figure 1A , and also has such as about Figure 2A , 2B Or the same features described by a single processing element 112 of 2C.

[0126] Figure 3 This is a schematic illustration of a device 300 (e.g., an integrated circuit, memory device, memory system, electronic device or system, smartphone, tablet computer, computer, server, appliance, vehicle, etc.) according to embodiments of the present disclosure. Device 300 may generally include a host 302 and a memory system 304.

[0127] Host 302 may be a host system, such as a personal laptop, desktop computer, digital camera, mobile phone, or memory card reader, as well as various other types of host systems. Host 302 may include multiple memory access devices (e.g., multiple processors). Host 302 may also be a memory controller, for example, where memory system 304 is a memory device (e.g., a memory device with an on-die controller).

[0128] The memory system 304 may be a solid-state drive (SSD) or other type of memory, and may include a host interface 306, a controller 308 (e.g., a processor and / or other control circuitry), and multiple memory devices 314. The memory system 304, controller 308, and / or memory devices 314 may also be considered individually as "devices." The memory devices 314 may include multiple solid-state memory devices, such as NAND flash memory devices, which provide storage capacity for the memory system 304. Other types of memory may also be used.

[0129] Controller 308 may be coupled to host interface 306 and memory device 314 via multiple channels to transfer data between memory system 304 and host 302. Interface 306 may be in the form of a standardized interface. For example, when memory system 304 is used for data storage in device 300, interface 306 may be Serial Advanced Technology Attachment (SATA), Peripheral Component Interconnect High Speed ​​(PCIe), or Universal Serial Bus (USB), as well as other connectors and interfaces. Typically, interface 306 provides an interface for transmitting control, address, data, and other signals between memory system 304 and host 302 having a compatible receiver for interface 306.

[0130] The controller 308 can communicate with the memory device 314 (which in some embodiments may include multiple memory arrays on a single die) to control data read, write, and erase operations, as well as other operations. The controller 308 may include channels for coupling the controller 308 to each memory device 314. Figure 3 A discrete memory channel controller (not shown in the diagram). Controller 308 may include, for example, hardware and / or firmware (e.g., one or more integrated circuits) and / or software components for controlling access to memory device 314 and / or for facilitating data transfer between host 302 and memory device 314.

[0131] Controller 308 may include an ECC encoder 310 for encoding data bits written to memory device 314 using one or more encoding techniques. The ECC encoder 310 may include a single parity check (SPC) encoder and / or algebraic error correction circuitry, such as one of the following groups: Bosch-Joheri-Hockwinheim (BCH) ECC encoders and / or Reed-Solomon ECC encoders, and other types of error correction circuitry. Controller 308 may further include an ECC decoder 312 for decoding the encoded data, which may include error identification units, conversion of error units to erase, and / or corrective erase. Memory device 314 may, for example, include one or more output buffers that can read selected data from memory cells of memory device 314. The output buffers can provide output data that can be provided to ECC decoder 312 as encoded input data. Figure 1B Neural networks 150 and / or Figure 1C Neural network 170 (e.g., as by) Figure 2A-2C One or more processing units can be used to implement, for example Figure 3 The ECC decoder 312. In various embodiments, the ECC decoder 312 may be able to decode data for each type of encoder in the ECC encoder 310. For example, Figure 2A , 2B The memory 145 of the 2C can store parameters associated with multiple encoding techniques that can be used by the ECC encoder 310, such that the hardware implementation of the neural network 150 or the recurrent neural network 170 can be used as a 'universal decoder' to decode the input data encoded by the ECC encoder 310 using any of the multiple encoding techniques available for the ECC encoder.

[0132] The ECC encoder 310 and ECC decoder 312 may each be implemented using discrete components such as application-specific integrated circuits (ASICs) or other circuit systems, or the components may reflect the functionality provided by the circuit system within the controller 308, and may not necessarily have a discrete physical form separate from the rest of the controller 308. Although described as Figure 3 The components are located within the controller 308, but each of the ECC encoder 310 and ECC decoder 312 may be located outside the controller 308, or have multiple components located within the controller 308 and multiple components located outside the controller 308.

[0133] Memory device 314 may include multiple arrays of memory cells (e.g., non-volatile memory cells). For example, the array may be a flash array with a NAND architecture. However, embodiments are not limited to a particular type of memory array or array architecture. Floating-gate flash memory cells in a NAND architecture may be used, but embodiments are not limited thereto. Cells may be multi-level cells (MLCs), such as three-level cells (TLCs) that store three data bits per cell. Memory cells may, for example, be grouped into multiple blocks containing multiple physical pages. Multiple blocks may be contained in a plane of memory cells, and the array may contain multiple planes. As an example, the memory device may be configured to store 8KB (kilobytes) of user data per page, 128 pages of user data per block, 2048 blocks per plane, and 16 planes per device.

[0134] According to several embodiments, controller 308 can control the encoding of a plurality of received data bits based on the identification of allowed error bits and the ECC encoder 310 for the conversion of said error bits to erase. Controller 308 can also control the programming of a number of encoded received data bits into a group of memory cells in memory device 314.

[0135] Figure 3 The devices shown can be implemented in any of a variety of products employing processors and memory, including, for example, cameras, telephones, wireless devices, displays, chipsets, set-top boxes, gaming systems, vehicles, and appliances. Devices employing memory systems can benefit from the examples of neural networks described herein to perform their end-user functions.

[0136] As should be understood from the foregoing, although specific embodiments have been described herein for illustrative purposes, various modifications can be made while maintaining the scope of the claimed art. Certain details are set forth herein to provide an understanding of the described embodiments of the art. However, other examples can be practiced without various details among these specific details. In some examples, well-known circuits, control signals, timing protocols, neural network architectures, algorithms, and / or software operations are not shown in detail to avoid unnecessarily obscuring the described embodiments. Other embodiments can be utilized, and other changes can be made, without departing from the spirit or scope of the subject matter presented herein.

[0137] Figure 4 This is a flowchart of method 400 arranged according to the examples described herein. An example method may include block 402, which may be followed by block 404, which may be followed by block 406, which may be followed by block 408, and which may be followed by block 410. Additional, fewer, and / or different blocks may be used in other instances, and the order of the blocks may be different in other instances.

[0138] Box 402 describes "receiving a known encoded and decoded data pair, wherein the encoded data is encoded using an encoding technique." The known encoded and decoded data pair may be received by a computing device (e.g., electronic device 110), which includes a neural network, for example... Figure 1B Neural network 150 Figure 1C Neural network 170 (e.g., as by) Figure 2A-2C (implemented by one or more processing units), and / or Figure 3 The ECC decoder 312. Signaling indicating the data set can be provided to the computing device.

[0139] Box 404 follows box 402. Box 404 describes “determining a set of weights for a neural network to decode data encoded using an encoding technique.” For example, the encoded and decoded data pairs received in box 402 can be used to train a neural network (e.g., any of the neural networks described herein). The weights can be numerical values ​​that, when used by the neural network, allow the neural network to output decoded data corresponding to the encoded input data encoded using the encoding technique. The weights can, for example, be stored in… Figure 2A , 2B Or in memory 145 of 2C. In some instances, training may not be performed, and the initial set of weights may be provided only to the neural network, for example, for training based on another neural network.

[0140] In some instances, multiple sets of data pairs can be received (e.g., in box 402), each set corresponding to data encoded using a different encoding technique. Therefore, multiple sets of weights can be determined (e.g., in box 404), each set corresponding to a different encoding technique. For example, one set of weights can be determined for decoding data encoded using LDPC decoding, while another set of weights can be determined for decoding data encoded using BCH decoding.

[0141] Box 406 may follow box 404. Box 406 describes "receiving data encoded using an encoding technique". For example, data encoded using an encoding technique (e.g., signaling indicating data) may be retrieved from the memory of a computing device and / or received using a wireless communication receiver. Any of a variety of encoding techniques may be used to encode the data.

[0142] Box 408 follows box 406. Box 408 describes "decoding data using a set of weights". Decoded data can be determined by processing the encoded data received in box 406, which may have already been determined in box 404, using weights. For example, any neural network described herein (e.g., Figure 1B Neural network 150 Figure 1C Neural network 170 (e.g., as by) Figure 2A-2C(implemented by one or more processing units) and / or Figure 3 The ECC decoder 312 can be used to decode the encoded data. In some instances, a set of weights can be selected that is associated with the encoding technique used to encode the data received in box 406. The set of weights can be selected from a plurality of available sets of weights, each used to decode data encoded with a different encoding technique.

[0143] Box 410 may follow box 408. Box 410 describes "writing decoded data to or reading decoded data from memory". For example, the data decoded in box 408 may be written to memory, such as... Figure 3 The memory device 314. In some instances, instead of writing data to memory or, in addition to writing data to memory, decoded data can be transmitted to another device (e.g., using wireless communication technology). Although box 410 describes memory, in some instances, any memory device can be used.

[0144] In some instances, boxes 406-410 can be repeated for data encoded using different encoding techniques. For example, data encoded using one encoding technique (e.g., LDPC decoding) can be received in box 406. A set of weights can be selected for use with LDPC decoding and provided to the neural network for decoding in box 408. Decoded data can be obtained in box 410. Data encoded using a different encoding technique (e.g., BCH decoding) can then be received in box 406. Another set of weights can be selected for use with BCH decoding and provided to the neural network for decoding in box 408. Decoded data can be obtained in box 410. In this way, a neural network can be used to decode data encoded using multiple encoding techniques.

[0145] Figure 5 This is a flowchart of method 500 according to an example described herein. Example method 500 may use, for example... Figure 1A System 100 in Figure 1B Neural network 150 Figure 1C The neural network 170 (e.g., by) Figure 2A-2C (implemented by one or more processing units), and / or Figure 3 The ECC decoder 312, or any system or combination of systems depicted in the foregoing figures, may be used for implementation. In some instances, the boxes in instance method 500 may be, for example, Figure 1A The electronic device 110 is a computing device and / or combined with, for example, Figure 2A , 2BThe operations described in blocks 504-516 may also be stored as computer-executable instructions in a computer-readable medium, such as a mode-configurable control 105, which stores the executable instructions 115.

[0146] Example method 500 may begin with block 504, which begins execution of a routine that mixes encoded input data with coefficient data. The method may include block 508, which states “obtaining encoded data associated with an encoding technique.” As described herein, the processing unit may be configured to obtain multiple types of input data that can be encoded using various encoding techniques, such as data encoded using low-density parity-check decoding (LDPC), Reed-Solomon decoding, Bosch-Joheri-Hokvenheim (BCH), and / or polarization decoding. In the context of device 300, encoded data may be obtained from memory device 314, such as regarding… Figure 3 As described.

[0147] Box 508 may be followed by box 512, which states "retrieving multiple coefficients from memory, said multiple coefficients being associated with an encoding technique." As described herein, the processing unit may retrieve coefficients for mixing with encoded input data; for example, using a memory lookup unit (MLU). For example, the memory may store (e.g., in a database) the associations between the coefficients described herein and the encoding techniques. For example, the processing unit may request coefficients from a memory portion of a computing device, from a memory portion of an external computing device, or from a memory implemented in a cloud computing device. Conversely, multiple coefficients may be retrieved from memory as requested by the processing unit.

[0148] Box 512 can be followed by box 516, which states that "the encoded data is computed with multiple coefficients at the first layer of the multiplication / accumulation processing unit (MAC unit) to produce a first processing result." As described herein, the processing unit utilizes multiple coefficients such that mixing the coefficients with the encoded input data produces a response... Figure 2A , 2B Alternatively, a 2C circuit system processes input data with coefficients to produce output data. In some implementations of a recurrent neural network (e.g., recurrent neural network 170), a delayed version of the corresponding output of the first layer of the MAC unit is mixed with the encoded input data to produce output data (e.g., an estimate of the decoded data). For example, various ALUs in an integrated circuit can be configured as... Figure 2A , 2B Or, a 2C circuit system operation, thereby mixing the input data and the delayed version of the corresponding output of the first layer of the MAC unit with coefficients as described herein. For example, refer to Figure 2AAt the first level of the multiplication / accumulation processing unit (MAC unit), multiple coefficients can be used to calculate a delayed version of the input data and the corresponding output of the first level of the MAC unit to produce a first processing result. In some instances, various hardware platforms can be used for implementation. Figure 2A , 2B Or 2C circuit systems, such as ASICs, DSPs implemented as FPGAs, or systems-on-a-chip.

[0149] Box 516 can be followed by box 520, which states that "a first processing result is calculated at an additional layer of the MAC unit using additional coefficients to produce a second processing result." As described herein, the processing unit utilizes additional coefficients such that mixing the coefficients with certain processing results produces a reaction. Figure 2A , 2B Or, a 2C circuit system processes input data with coefficients to produce output data. In some implementations of a recurrent neural network (e.g., recurrent neural network 170), a first processing result and at least a delayed version of the first processing result are mixed to produce output data (e.g., an estimate of decoded data). For example, refer to... Figure 2A At the second layer of the multiplication / accumulation processing unit (MAC unit), the processing result of the first layer (e.g., the multiplication processing result) and a delayed version of at least a portion of the processing result can be calculated using additional coefficients to produce a second processing result. The processing result of the second layer can be calculated using additional coefficients to produce output data B(1)230.

[0150] Box 520 may be followed by box 524, which states that it “provides output data representing properly decoded encoded data, such that the output data is an estimate of decoded data based on the encoded data.” As described herein, neural network 150 or recurrent neural network 170 provides output data as output bits representing processed data corresponding to encoded input data (e.g., m encoded input bits) that have been decoded according to an encoding technique. Therefore, the neural network described herein transforms the encoded input data into decoded data (e.g., an estimate of the decoded data). Box 524 may be followed by box 528, which terminates instance method 500. In some instances, box 512 may be optional.

[0151] The boxes included in the described instance methods 400 and 500 are for illustrative purposes. In some embodiments, the boxes may be executed in a different order. In some other embodiments, various boxes may be eliminated. In still other embodiments, various boxes may be divided into additional boxes, supplemented by other boxes, or combined together into fewer boxes. Other variations of these specific boxes are contemplated, including changes in the order of the boxes, changes in the splitting or combination of the content of the boxes into other boxes, etc.

[0152] The instances described herein may refer to various components that are “coupled” or signals that are “provided to” or “received from” certain components. It should be understood that in some instances, a component directly couples one component to another, while in other instances, a component couples with an intermediate component placed between them. Similarly, a signal may be provided directly to and / or received directly from the listed components without intermediate components, but it may also be provided to and / or received from certain components via intermediate components.

Claims

1. An apparatus for data processing, comprising: a first stage of circuitry configured to receive encoded data, combine the encoded data, and evaluate at least one non-linear function using the combination of the encoded data and a delayed version of the combination of the encoded data to provide intermediate data; and at least a second stage of circuitry configured to receive the intermediate data and combine the intermediate data using a predetermined set of weights to provide decoded data, the predetermined set of weights based at least in part on an encoding technique associated with the encoded data, wherein the first and second stages of circuitry comprise a first plurality of multiply / accumulate units each configured to multiply at least one bit of the encoded data with at least one of the predetermined set of weights and sum a plurality of weighted bits of the encoded data.

2. The apparatus of claim 1, wherein the predetermined set of weights is based on training of a neural network on known encoded data and a delayed version of a combination of the known encoded data.

3. The apparatus of claim 2, wherein the neural network is configured to use the predetermined set of weights to provide an estimate of decoded data based on the received encoded data.

4. The apparatus of claim 2, further comprising a memory encoded with executable instructions to select the predetermined set of weights for the neural network according to the encoding technique associated with the encoded data.

5. The apparatus of claim 1, wherein the encoding technique comprises Reed-Solomon decoding, Bose-Chaudhuri-Hocquenghem (BCH) decoding, Low Density Parity Check (LDPC) decoding, Polar decoding, or a combination thereof.

6. The apparatus of claim 1, wherein the first stage of circuitry further comprises a first plurality of memory lookup units (MLUs) each configured to retrieve at least one intermediate data value corresponding to an output of a respective one of the first plurality of multiply / accumulate units based on the at least one non-linear function.

7. The apparatus of claim 1, wherein the first stage of circuitry further comprises a plurality of delay units configured to provide the delayed version of the combination of the encoded data based on the at least one intermediate data value provided by the first plurality of MLUs.

8. The apparatus of claim 1, further comprising a memory in communication with the first and second stages of circuitry, the memory configured to store a plurality of predetermined sets of weights, including the predetermined set of weights.

9. A method for data processing, comprising: receiving, at a computing device comprising a neural network, signaling indicative of a set of data pairs each comprising known encoded data and decoded data, wherein the known encoded data is encoded with an encoding technique; training the neural network on the set of data pairs; and providing, at the computing device, an estimate of decoded data based on received encoded data using a predetermined set of weights of the neural network. determining a set of weights for the neural network to decode data with the encoding technique based in part on the signaling, a coefficient multiplication result associated with the signaling, and a delayed version of the coefficient multiplication result associated with the signaling; receiving, from a memory of the computing device, signaling indicative of data encoded with the encoding technique; decoding the data using the neural network using the weights, wherein decoding the data using the neural network includes weighting bits of the data using at least some of the set of weights and summing selected weighted bits of the data to provide intermediate data, followed by weighting bits of the intermediate data using at least some of the weights and summing selected weighted bits of the intermediate data and selected delayed versions of the weighted bits of the intermediate data to provide the decoded data; and writing the decoded data to or reading the decoded data from the memory or storage medium of the computing device.

10. The method of claim 9, further comprising: determining a plurality of sets of weights for the neural network using additional encoded data encoded with other encoding techniques, pairs of decoded data, and delayed versions of additional coefficient multiplication results based on the additional encoded data encoded with the other encoding techniques and the pairs of decoded data, wherein the plurality of sets of weights each correspond to a different encoding technique; selecting the set of weights associated with the encoding technique; and providing the set of weights to the neural network or another neural network for decoding the data.

11. The method of claim 10, further comprising: receiving additional input data encoded with another encoding technique; selecting a selected set of weights of the plurality of sets of weights associated with the other encoding technique; providing the selected set of weights to the neural network; and decoding the additional input data using the selected set of weights.

12. The method of claim 9, wherein the encoding technique comprises Reed-Solomon decoding, Bose-Chaudhuri-Hocquenghem (BCH) decoding, Low-Density Parity-Check (LDPC) decoding, Polar decoding, or a combination thereof.

13. The method of claim 9, wherein determining the set of weights comprises selecting weights that result in a minimized value of an error function between an output of the neural network and known decoded data.

14. The method of claim 9, wherein determining the set of weights comprises determining a center vector for the neural network using a k-means clustering technique.

15. A method for data processing, comprising: mixing encoded data received at a processing unit using a plurality of coefficients and an additional plurality of coefficients, the plurality of coefficients and the additional plurality of coefficients both specific to an encoding technique associated with the encoded data, wherein mixing the encoded data comprises: calculating, at a first layer of MAC units of a plurality of multiply / accumulate processing units (MAC units), the encoded data and a delayed version of the respective output of the first layer of MAC units with the plurality of coefficients to produce a first processing result; calculating, at an additional layer of MAC units of the plurality of MAC units, the first processing result and a delayed version of at least a portion of the first processing result with the additional plurality of coefficients to produce a second processing result; and providing output data based in part on the second processing result, the output data representing the encoded data being decoded, such that the output data is an estimate of decoded data based on the encoded data, wherein the plurality of MAC units comprises a first plurality of MAC units each configured to multiply at least one bit of the encoded data with at least one of a set of predetermined weights and sum a plurality of weighted bits of the encoded data.

16. The method of claim 15, wherein mixing the encoded data received at the processing unit using the plurality of coefficients further comprises: delaying the respective output of the first layer of MAC units at respective delay units associated with the first layer of MAC units to produce the delayed version of the respective output of the first layer of MAC units.

17. The method of claim 15, wherein calculating, at the first layer of MAC units of a plurality of multiply / accumulate processing units (MAC units), the encoded data and the delayed version of the respective output of the first layer of MAC units with the plurality of coefficients comprises: multiplying the encoded data and the delayed version of the respective output of the first layer of MAC units with respective ones of the plurality of coefficients to produce the first processing result.

18. The method of claim 15, further comprising: obtaining, at the processing unit, signaling indicative of the encoded data from a coupled memory, the signaling indicative of the encoded data including an indication of utilization of the encoding technique from a touchscreen of a mobile communication device.

19. The method of claim 15, wherein the encoding technique comprises Reed-Solomon decoding, Bose-Chaudhuri-Hocquenghem (BCH) decoding, Low-Density Parity-Check (LDPC) decoding, Polar decoding, or a combination thereof.

Images