Artificial neural network system and method to execute boolean neuron in boolean artificial neural network

Abstract

An Artificial Neural Network system comprising a processing arrangement configured to execute a Boolean neuron in a layer in a Boolean Artificial Neural Network and the Boolean neuron comprises an input module configured to receive a set of inputs, a weight module configured to hold a Boolean weight for each input, each input and Boolean weight forming a pair, a preactivation module configured to determine a preactivation sum (s) based on applying a logic function to each pair of input and Boolean weight producing an intermediate result and applying a summing function on the intermediate results and an activation module to generate a neuron output. Moreover, the processing arrangement is further configured to apply a scale adjustor to the preactivation sum providing a scale-adjusted preactivation sum and generate the neuron output by applying the activation function to the scale-adjusted preactivation sum.

Description

[0001] ARTIFICIAL NEURAL NETWORK SYSTEM AND METHOD TO EXECUTE BOOLEAN NEURON IN BOOLEAN ARTIFICIAL NEURAL NETWORK TECHNICAL FIELD

[0002] The present disclosure relates generally to the field of wireless communication networks and more specifically, to an artificial neural network system to execute Boolean neuron in a layer in a Boolean artificial neural network and a method for the artificial neural network system, such as by expressive Boolean neurons.

[0003] BACKGROUND

[0004] In existing neural network architectures, artificial neurons utilize floating-point weights to process and analyze input data, requiring substantial computational resources for both training and inference operations. Moreover, such floating-point weights require complex arithmetic operations and have considerable memory requirements thereby impacting the overall efficiency of the neural network. However, due to the increasing demand for the deployment of the neural network in resource-constrained environments, such computational requirements often pose significant challenges, such as increased power consumption, low data processing speed, limited scalability on edge devices, and reduced feasibility for real-time applications.

[0005] Conventionally, certain attempts have been taken to address the computational complexity associated with floating-point weights, such as by using binarized neurons or operating directly in the Boolean domain, and the like. However, such attempts failed due to various limitations, including reduced expressivity, performance gaps as compared to full-precision networks, and continued reliance on floating-point computations during training. Thus, there exists a technical problem of how to provide an artificial neuron module structure that can utilize 1-bit weights while maintaining sufficient expressivity to capture complex patterns in the input data, without compromising computational efficiency or performance in high-precision tasks.

[0006] Therefore, in light of the foregoing discussion, there exists a need to overcome the aforementioned drawbacks associated with the conventional artificial neural network systems for executing a Boolean neuron in a layer in a Boolean artificial neural network.

[0007] SUMMARY

[0008] The present disclosure provides a to an artificial neural network system to execute Boolean neuron in a layer in a Boolean artificial neural network and a method for the artificial neural network system, such as by expressive Boolean neurons. The present disclosure provides a solution to the existing problem of how to provide an artificial neuron module structure that can utilize 1-bit weights while maintaining sufficient expressivity to capture complex patterns in the input data, without compromising computational efficiency or performance in high-precision tasks. An objective of the present disclosure is to provide a solution that overcomes at least partially the problems encountered in the prior art and provides the artificial neural network system to execute Boolean neuron in a layer in a Boolean artificial neural network and a method for the artificial neural network system, such as by expressive Boolean neurons.

[0009] One or more objectives of the present disclosure are achieved by the solutions provided in the enclosed independent claims. Advantageous implementations of the present disclosure are further defined in the dependent claims.

[0010] In one aspect, the present disclosure provides an artificial neural network system comprising a processing arrangement configured to execute a Boolean neuron in a layer in a Boolean Artificial Neural Network, wherein the Boolean neuron comprises an input module configured to receive a set (X) of inputs (x), a weight module configured to hold a Boolean weight (wB°01) for each input (x), each input (x) and Boolean weight (wB°01) forming a pair, a preactivation module configured todetermine a preactivation sum (s) based on applying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result and applying a summing function on the intermediate results, and an activation module configured to generate a neuron output (y) by applying an activation function (f) to the preactivation sum (s; y=f(s)). Moreover, the artificial neural network system is characterized in that the processing arrangement is further configured to apply a scale adjust to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) and generate the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)).

[0011] Advantageously, the Artificial Neural Network system comprising a Boolean neuron provides efficient and scalable processing capabilities by utilizing Boolean weights and logic-based operations to minimize computational complexity and memory usage. By determining a preactivation sum through a logic function applied to input-weight pairs, the system significantly reduces reliance on computationally expensive floating-point operations while maintaining accuracy. Furthermore, the inclusion of a full-precision scale adjustor enhances the expressivity of the Boolean neuron, allowing it to fine-tune outputs and effectively model complex data patterns. The combination of Boolean arithmetic and learnable scaling enables high-performance neural computations, particularly in resource-constrained environments. Additionally, the iterative learning process optimizes the scale adjustor and Boolean weights to achieve performance levels comparable to floating-point neurons. Thus, the Artificial Neural Network system effectively reduces data processing complexity while enhancing precision and adaptability for advanced deep learning tasks across diverse applications.

[0012] In another aspect, the present disclosure provides a method for an Artificial Neural Network system comprising a Boolean neuron in a layer in a Boolean Artificial Neural Network, wherein the method comprises receiving a set (X) of inputs (x), obtaining a Boolean weight (wB°01) for each input (x), each input (x) and Boolean weight (wB°01) forming a pair, determining a preactivation sum (s) based on applying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result and applying a summing function on the intermediate results, and generating a neuron output (y ) by applying an activation function (f) to the preactivation sum (s; y=f(s)). Moreover, the method is characterized in that the method further comprises applying a scale adjustor to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) and generating the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)).

[0013] The method achieves all the advantages and technical effects of the artificial neural network of the present disclosure.

[0014] It is to be appreciated that all the aforementioned implementation forms can be combined.

[0015] It has to be noted that all devices, elements, circuitry, units, and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application, as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.

[0016] Additional aspects, advantages, features, and objects of the present disclosure would be made apparent from the drawings and the detailed description of the illustrative implementations construed in conjunction with the appended claims that follow.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.

[0018] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following diagrams wherein:

[0019] FIG. 1 is a block diagram illustrating an artificial neural network system configured to execute a Boolean neuron, in accordance with an embodiment of the present disclosure;

[0020] FIG. 2 is a diagram that illustrates an architecture of an expressive Boolean neuron module, in accordance with an embodiment of the present disclosure;

[0021] FIG. 3 is a diagram that illustrates utilization of a shared scale adjustor and a shared shift adjustor by Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure;

[0022] FIG. 4 is a diagram that illustrates utilization of a shared scale adjustor for all Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure;

[0023] FIG. 5 is a diagram that illustrates utilization of a shared shift adjustor for Boolean neurons within a same layer of a neural network, in accordance with an embodiment of the present disclosure;

[0024] FIG. 6 is a diagram that illustrates utilization of independent scale adjustors and independent shift adjustors for Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure; and

[0025] FIG. 7 is a flowchart depicting a method for an Artificial Neural Network system comprising a Boolean neuron in a layer in a Boolean Artificial Neural Network, in accordance with an embodiment of the present disclosure.

[0026] In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.

[0027] DETAILED DESCRIPTION OF EMBODIMENTS

[0028] The following detailed description illustrates embodiments of the present disclosure and ways in which they can be implemented. Although some modes of carrying out the present disclosure have been disclosed, those skilled in the art would recognize that other embodiments for carrying out or practicing the present disclosure are also possible.

[0029] FIG. 1 is a block diagram illustrating an artificial neural network system configured to execute a Boolean neuron, in accordance with an embodiment of the present disclosure. With reference to FIG. 1, there is shown a block diagram 100 of an artificial Neural Network system 102 comprising a processing arrangement 104, an activation module 106, an input module 108, a weight module 110, a preactivation module 112, a memory 114, and a network interface 116.The processing arrangement 104 is configured to execute a Boolean neuron in a layer in a Boolean Artificial Neural Network. Examples of the processing arrangement 104 may include but are not limited to a central data processing device, a microprocessor, a microcontroller, a complex instruction set computing (CISC) processor, an application-specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a state machine, and other processors or control circuitry.

[0030] A memory 114 is used to store the instructions received from the processing arrangement 104. Examples of implementation of the memory 114 may include, but are not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Dynamic Random Access Memory (DRAM), Random Access Memory (RAM), Read-Only Memory (ROM), Hard Disk Drive (HDD), Flash memory, a Secure Digital (SD) card, Solid-State Drive (SSD), and / or CPU cache memory.

[0031] A network interface 116 is used to communicate with the processing arrangement 104 and the memory 114. Examples of implementation of the network interface 116 may include but are not limited to a network interface, a computer port, a network socket, a network interface controller (NIC), and any other network interface device.

[0032] In operation, the processing arrangement 104 is configured to execute a Boolean neuron in a layer in a Boolean Artificial Neural Network. The processing arrangement 104 is configured to receive input that is processed using Boolean weights and logical operations to compute pre-activation values. Moreover, the pre-activation values are then adjusted using a full-precision scale adjustor, which multiplies the pre-activation by a learnable scalar, and a shift adjustor, which adds a learnable scalar. This refinement step enhances the flexibility and performance of the Boolean neuron. Finally, the adjusted values are passed through an activation function, producing the output. The arrangement also supports shared or independent adjustors across neurons in the same layer, offering flexibility and adaptability based on the specific application.

[0033] Furthermore, the Boolean neuron includes the input module 108 configured to receive a set (X) of inputs (x). In an implementation, the input module 108 acts as an interface for feeding data into the neuron in order to ensure compatibility with various input formats and preprocessing requirements. The inputs are passed to the Boolean neuron for further processing in order to ensure that the inputs are formatted accurately and are compatible with the logical operations performed by the neuron. Moreover, if the inputs are full-precision, then, in that case, the inputs are converted or processed in a manner suitable for Boolean computations while preserving their semantic value in order to ensure a consistent flow of data through the network while optimizing the overall performance of the network.

[0034] Furthermore, the Boolean neuron includes the weight module 110 configured to hold a Boolean weight (wB°01) for each input (x), each input (x), and Boolean weight (wB°01) forming a pair. The Boolean weights are represented as 1 -bit values, either 00 or 11 , and are associated with respective inputs to control the influence of each input on the outputs of the Boolean neuron. Moreover, by using 1 -bit Boolean weights, the weight module reduces the memory footprint and energy consumption of the Boolean neuron, and the pairing of inputs and weights ensures precise control over the contribution of each input to the pre-activation value of the Boolean neuron. In an implementation, when the inputs are processed, the corresponding Boolean weights are applied using logical operations (e.g., AND, XOR, XNOR) to compute the pre-activation value of the Boolean neuron. As a result, the weight module 110 that is configured to hold the Boolean weight for each input is used to ensure that the neuron operates entirely within the Boolean domain for weights, minimizing computational complexity while retaining sufficient flexibility to capture patterns in the data.

[0035] Furthermore, the Boolean neuron includes the preactivation module 112 configured to determine a preactivation sum (s) based on applying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result and applying a summing function on the intermediate results. The preactivation module 112 is configured to process each pair of input and Boolean weight by applying a predefined logic function. For example, the preactivation module 112 is configured toapply an AND or XOR operation that combines the input and the corresponding weight to generate an intermediate Boolean result which is then aggregated using a summing function in order to compute the preactivation sum. Moreover, the sum serves as the input for subsequent adjustments (e.g., scaling and shifting) and activation functions, enabling the Boolean neuron to provide the final output. As a result, the preactivation module 112 is configured to reduce power consumption and processing time, especially for low-power devices and real-time applications. Additionally, the flexibility in selecting logic functions allows the preactivation module 112 to adapt to different tasks and datasets, ensuring simplicity, adaptability, and efficiency enhance the overall performance and scalability of the neural network.

[0036] Furthermore, the Boolean neuron includes the activation module 106 configured to generate a neuron output (y) by applying an activation function (f) to the preactivation sum (s; y=f(s)). Moreover, the Artificial Neural Network system 102 is characterized in that the processing arrangement 104 is further configured to apply a scale adjustor (a) to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) and generate the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)). The activation module 106 first takes the preactivation sum, which is computed by the preactivation module 112 and before applying the activation function, the processing arrangement 104 uses a scale adjustor and a learnable scalar to modify the preactivation sum in order to obtain a scale-adjusted value. Moreover, the adjusted value is then passed through the activation function, such as ReLU or sigmoid, to produce the neuron output in order to ensure that the output reflects both the binary efficiency of the Boolean domain and the enhanced expressivity introduced by the scaling factor.

[0037] In accordance with an embodiment, the processing arrangement 104 is further configured to apply a shift adjustor ( ) to the scale-adjusted preactivation sum (s’) providing a shift-adjusted preactivation sum (s”) and generate the neuron output (y) by applying the activation function (f) to the shift-adjusted preactivation sum (y=f(s”)). The processing arrangement 104 first computes the scale-adjusted preactivation sum, such as by using the scale adjustor, and then applies the shift adjustor to the scaled value thereby producing the shift-adjusted preactivation sum. Finally, the shift-adjusted sum is passed through an activation function, such as ReLU or sigmoid, to compute the neuron output. As a result, the processing arrangement 104 is configured to align with the requirements of the activation function that improves the expressivity and flexibility of the neuron while retaining the computational efficiency of the Boolean neuron.

[0038] In accordance with an embodiment, the scale adjustor (a) is a full precision learnable scalar. The scale adjustor is a fullprecision learnable scalar, which is used to scale the preactivation sum in order to modulate the influence of the inputs on the output of the Boolean Neuron. Moreover, the scale adjustor allows the Boolean neuron to dynamically adjust the scale of its preactivation values based on the training data, allowing it to adapt to different data distributions. Additionally, the scale adjustor multiplies the preactivation sum to produce a scale-adjusted value during the training process through gradient-based techniques to minimize errors in order to enhance the ability of the Boolean neuron to learn complex patterns while maintaining the low computational complexity and effectively bridging the performance gap between Boolean and full-precision Boolean neurons. As a result, the Boolean neuron can efficiently handle tasks requiring higher precision, such as natural language processing and computer vision, without sacrificing computational efficiency.

[0039] In accordance with an embodiment, the shift adjustor (P) is a full precision learnable scalar. The shift adjustor is a full-precision learnable scalar that enables the adjustment of the preactivation sum by adding an adjustable value. Similar to the scale adjustor, the shift adjustor is optimized during training to refine the output of the Boolean neuron in order to provide an additional layer of flexibility, allowing the Boolean neuron to align its output with complex patterns in the data. Moreover, by offsetting the preactivation sum with a learnable value, the shift adjustor overcomes the inherent limitations of 1 -bit Boolean weights, offering precise control over the response of the neuron, the shift adjustor is configured to add to the scale-adjusted preactivation sumto produce a shift-adjusted sum. As a result, the shift adjustor allows the neuron to adaptively shift the preactivation values to improve the performance of the network.

[0040] In accordance with an embodiment, the Boolean neuron comprises the scale adjustor (a). Moreover, the scale adjustor is multiplied with the preactivation sum to provide a scale-adjusted value, which is performed before the application of the activation function thereby ensuring that the Boolean neuron can modify the influence of its inputs according to the learned scalar. Additionally, the scale adjustor within the Boolean neuron is used to provide additional flexibility and expressivity in order to allow the neuron to adjust the magnitude of the preactivation sum.

[0041] In accordance with an embodiment, the processing arrangement 104 is further configured to apply the same scale adjustor (a) to other neurons of the same layer. Moreover, the application of the same scale adjustor to all neurons within the same layer ensures consistency in how the neurons scale their preactivation values, improving the coherence and efficiency of the network. By sharing a common scale adjustor, the neuron of the same layer works together effectively thereby reducing the overall number of learnable parameters and promoting efficient learning across the layer.

[0042] In accordance with an embodiment, the Boolean neuron comprises the shift adjustor (P). Moreover, the shift adjustor is used to provide the Boolean neuron with the ability to adjust the offset of its preactivation sum, which enables the processing arrangement 104 to handle complex patterns in the data. Additionally, the shift adjustor adds an extra layer of control over the preactivation values thereby improving the expressivity of the Boolean neuron.

[0043] In accordance with an embodiment, the processing arrangement 104 is further configured to apply the same shift adjustor ( ) to other neurons of the same layer. In other words, the processing arrangement 104 is configured to apply the same shift adjustor to the preactivation sums of all neurons in the same layer. Moreover, the scale-adjusted preactivation sum of each neuron is modified by the same shift scalar value before the activation function is applied, allowing the entire layer to operate with consistent offsets. As a result, the neurons in the layer can be optimized together thereby reducing the number of parameters to learn and increasing training efficiency while reducing the overall complexity of the network.

[0044] Advantageously, the Artificial Neural Network system 102 comprising a Boolean neuron provides efficient and scalable processing capabilities by utilizing Boolean weights and logic-based operations to minimize computational complexity and memory usage. By determining a preactivation sum through a logic function applied to input-weight pairs, the system significantly reduces reliance on computationally expensive floating-point operations while maintaining accuracy. Furthermore, the inclusion of a full-precision scale adjustor enhances the expressivity of the Boolean neuron, allowing it to fine-tune outputs and effectively model complex data patterns. The combination of Boolean arithmetic and learnable scaling enables high-performance neural computations, particularly in resource-constrained environments. Additionally, the iterative learning process optimizes the scale adjustor and Boolean weights to achieve performance levels comparable to floating-point neurons. Thus, the Artificial Neural Network system 102 is configured to reduce the overall processing complexity while enhancing the precision and adaptability for advanced deep learning tasks across diverse applications.

[0045] FIG. 2 is a diagram that illustrates an architecture of an expressive Boolean neuron module, in accordance with an embodiment of the present disclosure. FIG.2 is described in conjunction with elements from FIG. 1. With reference to FIG.2, there is shown a diagram 200 of the architecture of the expressive Boolean neuron module.

[0046] In an implementation, the architecture of the expressive Boolean neuron module includes a full-precision scale adjustor and a shift adjustor after the pre-activation. The Boolean neuron module includes inputs (e.g., x±,x2, ... , xM), such as, a first input 202A, a second input 202B, up to Nth input 202N and weights, such as a first weight 204A, a second weight 204B, up to nth weight 204N that are either full precision or Boolean. Moreover, the Boolean neuron module include logic gates, such asXNOR, XOR, OR, AND, and others, which perform operations on the inputs and weights. Additionally, the expressive Boolean neuron module includes a scale adjustor 210 and a shift adjustor 212 that are a full-precision scalar, which are applied after the pre-activation stage and before the activation function 208 and can be shared among neurons within the same layer of a neural network. Furthermore, the expressive Boolean neuron module is configured to compute the output 214 the expressive Boolean neuron by using the formula 206 given below:

[0047] s = ^= 1L xi,w‘!ool

[0048] As a result, the expressive Boolean neuron module is configured to be used in all applications where existing full-precision artificial neurons are employed, such as natural language processing, computer vision, and large language models.

[0049] FIG. 3 is a diagram that illustrates utilization of a shared scale adjustor and a shared shift adjustor by Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure. FIG. 3 is described in conjunction with elements from FIG. 1 and 2. With reference to FIG. 3, there is shown a diagram 300 of the utilization of the shared scale adjustor and the shared shift adjustor by Boolean neurons of same layer of the neural network.

[0050] In an implementation, the Boolean neurons within the same layer of the neural network, such as a first Boolean neuron 302A, a second Boolean neuron 302B, a third Boolean neuron 302C, a fourth Boolean neuron 302D, up to nth Boolean neuron 302N are shares a common shared scale adjustor 304 and a shared shift adjustor 306, which is a full-precision and learnable adjustor. Moreover, the shared scale adjustor 304 and the shared shift adjustor 306 are applied uniformly to the preactivation of the Boolean neurons in the layer of the neural network. As a result, the number of learnable parameters are reduced in order to simplify the training process and reduce the overall computational complexity, while maintaining sufficient flexibility to capture complex patterns in the data.

[0051] FIG. 4 is a diagram that illustrates utilization of a shared scale adjustor for all Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure. FIG. 4 is described in conjunction with elements from FIG. 1,2, and 3. With reference to FIG. 4, there is shown a diagram 400 of the utilization of the shared scale adjustor for all Boolean neurons within the same layer of the neural network.

[0052] In an implementation, the Boolean neurons in the layer, such as the first Boolean neuron 302A, the second Boolean neuron 302B, the third Boolean neuron 302C, and up to an nth Boolean neuron 302N, share a common shared scale adjustor 304, which is a full-precision and learnable scalar along with an independent shift adjustors, such as a first shift adjustor 402A, a second shift adjustor 402B, a third shift adjustor 402C, a fourth shift adjustor 402D up to nth shift adjustor 402N. Moreover, each of the Boolean neuron within the same layer utilizes the independent shift adjustors thereby allowing for an improved flexibility in adjusting individual neuron outputs.

[0053] FIG. 5 is a diagram that illustrates utilization of a shared shift adjustor for Boolean neurons within a same layer of a neural network, in accordance with an embodiment of the present disclosure. FIG. 5 is described in conjunction with elements from FIG. 1, 2, 3, and 4. With reference to FIG. 5, there is shown a diagram 500 of the utilization of the shared shift adjustor for Boolean neurons within the same layer of the neural network.

[0054] In an implementation, the Boolean neurons in the layer, such as the first Boolean neuron 302A, the second Boolean neuron 302B, the third Boolean neuron 302C, and up to an nth Boolean neuron 302N shares the common shared shift adjustor 306, which is the full precision and learnable scalar along with an independent scale adjustors, such as a first scale adjustor 502A, a second scale adjustor 502B, a third scale adjustor 502C, a fourth scale adjustor 502D up to nth scale adjustor 502N. Moreover, each of the Boolean neuron within the same layer utilizes the independent scale adjustors thereby allowing for an improvedflexibility in adjusting individual neuron outputs that reduces the number of learnable parameters, streamlining training and lowering computational costs while preserving the capacity to model complex data patterns.

[0055] FIG. 6 is a diagram that illustrates utilization of independent scale adjustors and independent shift adjustors for Boolean neurons within the same layer of a neural network, in accordance with an embodiment of the present disclosure. FIG. 6 is described in conjunction with elements from FIG. 1 ,2, 3,4, and 5. With reference to FIG. 6, there is shown a diagram 600 of the utilization of the independent scale adjustors and the independent shift adjustors for the Boolean neurons within the same layer of the neural network.

[0056] In an implementation, the Boolean neurons in the layer, such as the first Boolean neuron 302A, the second Boolean neuron 302B, the third Boolean neuron 302C, and up to an nth Boolean neuron 302N, utilize independent scale adjustors, such as the first scale adjustor 502A, the second scale adjustor 502B, the third scale adjustor 502C, up to the nth scale adjustor 502N and independent shift adjustors, such as the first shift adjustor 402A, the second shift adjustor 402B, the third shift adjustor 402C, and up to the nth shift adjustor 402N. Moreover, by providing each Boolean neuron with independent scale and shift adjustors provides an enhanced flexibility in adjusting individual neuron outputs in order to capture and complex data patterns of the neural network.

[0057] FIG. 7 is a flowchart depicting a method for an Artificial Neural Network system comprising a Boolean neuron in a layer in a Boolean Artificial Neural Network, in accordance with an embodiment of the present disclosure. With reference to FIG. 7, there is shown a flowchart of a method 700 for the Artificial Neural Network system 102 (FIG. 1 ) comprising a Boolean neuron in a layer in a Boolean Artificial Neural Network comprising steps carried out at the Artificial Neural Network system 102 of the Boolean Artificial Neural network. The method 700 includes steps 702 to 708.

[0058] At step 702, the method 700 includes receiving a set (X) of inputs (x). Furthermore, at step 704, the method 700 includes obtaining a Boolean weight (wB°01) for each input (x), each input (x) and Boolean weight (wB°01) forming a pair. At step 706, the method 700 includes determining a preactivation sum (s) based on applying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result, such as at sub-step 706A and applying a summing function on the intermediate results, such as at sub-step 706B. At step 708, the method 700 includes generating a neuron output (y) by applying an activation function (f) to the preactivation sum (s; y=f(s)) and the method 700 is characterized in that the method 700 further comprises applying a scale adjustor (a) to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) and generating the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)).

[0059] In accordance with an embodiment, the method 700 further includes applying a shift adjustor (P) to the scale-adjusted preactivation sum (s’) providing a shift-adjusted preactivation sum (s”) and generating the neuron output (y) by applying the activation function (f) to the shift-adjusted preactivation sum (y=f(s”)). In an implementation, the shift adjustor is optimized during training using gradient-based techniques, dynamically adapting to the data for improved performance. By enabling precise control over the preactivation values, the method 700 is used to enhance the flexibility and expressivity of the Boolean neuron while maintaining low computational complexity and also allows the Boolean neuron to perform on par with fullprecision neurons in tasks like natural language processing and computer vision.

[0060] In accordance with an embodiment, the method 700 further comprises learning the scale adjustor (a) and the Boolean weights by using a gradient-based training method, wherein each Boolean weight is trained in order to decide whether the Boolean weight is to be inverted or to be kept. In other words, each Boolean weight undergoes a training process to determine whether if the corresponding Boolean weight should be inverted or retained in order to minimize errors by fine-tuning both the scale adjustor and the Boolean weights. Moreover, by combining the computational efficiency of 1-bit Boolean weights with theflexibility provided by the learnable scale adjustor, the method 700 is used to provide low power consumption and an enhanced computational efficiency.

[0061] In accordance with an embodiment, the method 700 further comprises also learning the shift adjustor (P) along with the scale adjustor (a) and the Boolean weights. The shift adjustor is trained as a full-precision learnable scalar, such as by using standard gradient-based optimization techniques, which includes computation of the preactivation by using logic gates and Boolean weights, application of the scale adjustor and the shift adjustor, computation of gradients, and the like. As a result, the learning of the shift adjustor in conjunction with the scale adjustor and Boolean weights enhances the expressivity of the Boolean neurons.

[0062] In accordance with an embodiment, the method 700 further comprises learning using an iterative training process including a forward and backward propagations. The learning of the iterative training process is used to optimize the parameters of the neural network systematically. In an implementation, the forward propagation is used to compute the output of the neural network based on the current parameter values and the backward propagation is used to calculate the gradients needed to update the corresponding parameters.

[0063] The steps 702 to 708 are only illustrative, and other alternatives can also be provided where one or more steps are added, one or more steps are removed, or one or more steps are provided in a different sequence without departing from the scope of the claims herein.

[0064] There is provided a computer program comprising instructions that, when executed by a computer system, cause the computer system to implement the method 700. In an example, the instructions are implemented on the computer-readable media, which include, but are not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Random Access Memory (RAM), Read-Only Memory (ROM), Hard Disk Drive (HDD), Flash memory, a Secure Digital (SD) card, Solid-State Drive (SSD), a computer-readable storage medium, and / or CPU cache memory.

[0065] Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as "including", "comprising", "incorporating", "have", "is" used to describe and claim the present disclosure are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural. The word "exemplary" is used herein to mean "serving as an example, instance or illustration". Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or to exclude the incorporation of features from other embodiments. The word "optionally" is used herein to mean "is provided in some embodiments and not provided in other embodiments". It is appreciated that certain features of the present disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable combination or as suitable in any other described embodiment of the disclosure.

Claims

CLAIMS1. An Artificial Neural Network system (102) comprising a processing arrangement (104) configured to execute a Boolean neuron in a layer in a Boolean Artificial Neural Network, wherein the Boolean neuron comprises:an input module (108) configured to receive a set (X) of inputs (x);a weight module (110) configured to hold a Boolean weight (wB°01) for each input (x), each input (x) and Boolean weight (wB°01) forming a pair,a preactivation module (112) configured to:determine a preactivation sum (s) based onapplying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result andapplying a summing function on the intermediate results; andan activation module (106) configured to:generate a neuron output (y) by applying an activation function (f) to the preactivation sum (s; y=f(s)); wherein the Artificial Neural Network system (102) is characterized in that the processing arrangement (104) is further configured to:apply a scale adjustor (a) to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) andgenerate the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)).

2. The Artificial Neural Network system (102) according to claim 1, wherein the processing arrangement (104) is further configured to:apply a shift adjustor (P) to the scale-adjusted preactivation sum (s’) providing a shift-adjusted preactivation sum (s”) andgenerate the neuron output (y) by applying the activation function (f) to the shift-adjusted preactivation sum (y=f(s”)).

3. The Artificial Neural Network system (102) according to claim 1 or 2, wherein the scale adjustor (a) is a full precision learnable scalar.

4. The Artificial Neural Network system (102) according to any preceding claim, wherein the shift adjustor (P) is a full precision learnable scalar.

5. The Artificial Neural Network system (102) according to any preceding claim, wherein the Boolean neuron comprises the scale adjustor (a).

6. The Artificial Neural Network system (102) according to any of claims 1 to 4, wherein the processing arrangement (104) is further configured to apply a same scale adjustor (a) to other neurons of the same layer.

7. The Artificial Neural Network system (102) according to claim 5 or 6, wherein the Boolean neuron comprises the shift adjustor (P).

8. The Artificial Neural Network system (102) according to claim 5 or 6, wherein the processing arrangement (104) is further configured to apply a same shift adjustor (P) to other neurons of the same layer.

9. A method (700) for an Artificial Neural Network system (102) comprising a Boolean neuron in a layer in a Boolean Artificial Neural Network, wherein the method (700) comprises:receiving a set (X) of inputs (x);obtaining a Boolean weight (wB°01) for each input (x), each input (x) and Boolean weight (wB°01) forming a pair, determining a preactivation sum (s) basedon applying a logic function (L) to each pair of input (x) and Boolean weight (wB°01) producing an intermediate result andapplying a summing function on the intermediate results; andgenerating a neuron output (y) by applying an activation function (f) to the preactivation sum (s; y=f(s)); wherein the method (700) is characterized in that the method (700) further comprises:applying a scale adjustor (a) to the preactivation sum (s) providing a scale-adjusted preactivation sum (s’) and generating the neuron output (y) by applying the activation function (f) to the scale-adjusted preactivation sum (y=f(s’)).

10. The method (700) according to claim 9, wherein the method (700) further comprises:applying a shift adjustor (P) to the scale-adjusted preactivation sum (s’) providing a shift-adjusted preactivation sum (s”) andgenerating the neuron output (y) by applying the activation function (f) to the shift-adjusted preactivation sum (y=f(s”)).

11. The method (700) according to claim 9, wherein the method (700) further comprises learning the scale adjustor (a) and the Boolean weights by using a gradient-based training method, wherein each Boolean weight is trained in order to decide whether the Boolean weight is to be inverted or to be kept.

12. The method (700) according to claim 10 and 11, wherein the method (700) further comprises also learning the shift adjustor (P) along with the scale adjustor (a) and the Boolean weights.

13. The method (700) according to claim 11 or 12, wherein the method (700) further comprises learning using an iterative training process including a forward and backward propagations.

14. A computer program product comprising program instructions for performing the method (700) according to claim 12 or 13, when executed by one or more processors in an Artificial Neural Network system (102).

Images