System and method of training a boolean deep learning model

Abstract

A method for training a Boolean deep learning model (308, 408, 608), performed by a processor device (302, 402, 510, 602), is provided. The method involves receiving a request from a Boolean deep learning application (306, 406, 512, 606) to instantiate a multi-dimensional Boolean matrix array (414, 502) comprising binary values representing the model's weights or biases. A sequence of matrix arithmetic operations is then received, detailing tasks to be performed on this Boolean matrix array. For each operation in the sequence, the method determines whether the bit data's memory layout (310, 410, 520) needs reordering into a different bit vectorization dimension. If reordering is required, the method converts the current bit data layout accordingly. Following each conversion, the method updates the indicative bit vectorization parameters to reflect the new layout. This dynamic adjustment ensures optimal memory utilization and computational efficiency throughout the training process, improving the performance of the Boolean deep learning model.

Description

[0001] SYSTEM AND METHOD OF TRAINING A BOOLEAN DEEP LEARN ING MODEL

[0002] TECHNICAL FIELD

[0003] The disclosure relates generally to training Boolean deep learning models, and more particularly, the disclosure relates to a method of training a Boolean deep learning model. Moreover, the disclosure relates to a system of training the Boolean deep learning model.

[0004] BACKGROUND

[0005] As Artificial Intelligence, Al continues to advance rapidly, it is significantly transforming industries and consumer behavior on a global scale. However, these advancements come with an exponential increase in the investment required for training state-of-the-art deep learning models. The growing complexity of training of deep learning models encompassing computation complexity, data movement, memory consumption, energy consumption, manpower, and resources has led to a burgeoning interest in Boolean computing. This approach aims to reduce computational complexity and energy consumption.

[0006] Real- valued systems involve the use of real numbers, which may take on any value within a range. These numbers are typically stored using multiple bits in digital systems, with common formats being floating-point or fixed-point representations. The real-valued systems perform arithmetic operations such as addition, subtraction, multiplication, and division on these multi-bit values. For example, multiplying two vectors A and B in a real-valued system involves multiple multiplications and additions of the real numbers stored in each matrix element. For example, each element A[i] and B[i] is a real number stored in b bits, either in floating-point or fixed-point format and performs N multiplications A[i]xB[i] for all elements A[i] and B[i] as shown in FIG. 1A.

[0007] Boolean systems use Boolean values, which may only take on one of two possible values: true (1) or false (0). These systems are fundamental to digital logic and binary decision-making processes. Each Boolean value is typically stored using 1 bit, as only two states need to be represented. The Boolean systems perform logical operations such as AND, OR, NOT, and XOR on these single-bit values. For example, in a Boolean matrix, each element may undergo a logical operation with a corresponding element from another matrix. Each element A[i] and B[i] is a Boolean value stored in 1 bit and performs native logic operations L(A[i], B[i]) for all elements A[i] and B[i] as shown in FIG. IB.

[0008] The real-valued systems require more memory because they store complex numbers using multiple bits, whereas the Boolean systems are more memory-efficient, using just 1 bit per element / value. The real-valued systems involve arithmetic operations that are computationally more intensive due to the need for multi-bit calculations. In contrast, the Boolean systems involve native logical operations that are faster and less resource-intensive. The current hardware of the Boolean systems is designed for per-byte allocation. Hence, the challenge lies in leveraging the advantages of 1 -bit Boolean systems within the constraints of current per-byte allocation hardware.

[0009] A natural method for implementing Boolean computing within conventional computing architectures involves handling supported datatypes, such as INT8, INT16, FLT16, andFLT32, as vectors. Basic linear algebra routines are low-level programs typically designed for operations on vectors and matrices composed of byte-sized elements. The basic linear algebra routines serve as essential low-level kernels that enable platform-dependent optimization of matrix algebra operations, such as matrix multiplications. Despite variations in the size of computing words in bits, the byte remains the smallest addressable unit of memory in most dominant computer architectures. Therefore, it is crucial to adapt existing architectures to allow basic linear algebra routines to operate at optimal performance, even when handling single-bit data types. When performing an operation on a Boolean variable (i.e., true or false, 1 or 0), the entire bit vector may be processed simultaneously. It then becomes the application's responsibility to prune the data bits. Additionally, modem acceleration frameworks typically operate at a minimum resolution of one byte, which, when applied to Boolean logic algebra, may result in wasted resources or unnecessary overhead. The challenge lies in leveraging the advantages of 1 -bit Boolean systems within the current per-byte allocation hardware for efficient access to and manipulation of unitary bit elements.

[0010] When arithmetic operations are performed repeatedly on a multi-dimensional Boolean array, the dimension selected for vectorizing / packing the Boolean elements of a Boolean variable A in a byte-based format depends on the specific operation being executed. For instance, based on its role as an operand, a matrix may be processed in either row major or column major order, i.e., pack the Boolean elements in a row if the operation is row wise, and pack the Boolean elements in a column if the operation is column wise as illustrated in FIG. 2A.

[0011] In scenarios, where a Boolean matrix is repeatedly accessed across multiple operation cycles, requiring the vectorizing dimension to alternate between different orientations, this frequent switching (i.e., the usual processing loop of Pack-Unpack- Process-Pack) introduces additional / repeated pack-unpack latency and increases data movement as shown in FIG. 2B. As a result, the use of bit-level data structures in linear memory models may complicate the process, making it difficult to fully leverage spatial locality with conventional methods. Consequently, existing approaches may not directly exploit these structures to achieve optimal performance. The challenge lies in leveraging the advantages of 1-bit Boolean systems within current per-byte allocation hardware while minimizing the recurring latency and data movement complexity that arises when a multi-dimensional Boolean array is sequentially processed across alternating dimensions.

[0012] To implement bit-level mapping on byte-level addressing architectures, applications typically store bit values within supported data types such as integers. However, this approach requires the application to manage the underlying hierarchical addressing manually, which can be both error-prone and suboptimal in performance.

[0013] To leverage Basic Linear Algebra Subprograms (BLAS) features during a machine learning training process, an application may alternate the vectorized dimension of a bit value matrix between the forward and backward passes. To pass from one data representation to another, an application often necessitates reconstructing an intermediate format that allows access to and manipulation of individual elements. This process may demand significant dynamic memory allocation, leading to inefficiencies.

[0014] Therefore, there arises a need to address the aforementioned technical problems / drawbacks in training a Boolean deep learning model.

[0015] SUMMARY

[0016] It is an object of the disclosure to provide a method of training a Boolean deep learning model at a processor device. Moreover, the disclosure relates to a system of training the Boolean deep learning model at the processor device.

[0017] This object is achieved by the features of the independent claims. Further, implementation forms are apparent from the dependent claims, the description, and the figures.

[0018] The disclosure provides a method of training a Boolean deep learning model at a processor device. Moreover, the disclosure relates to a system of training the Boolean deep learning model at the processor device.

[0019] According to a first aspect, there is provided a method of training a Boolean deep learning model. The method is performed at a processor device. The method includes receiving, from a Boolean deep learning application which uses the Boolean deep learning model, a request to instantiate a multi-dimensional Boolean matrix array comprising a plurality of binary values. Each of the plurality of binary values represents a weight or bias of the deep learning model. The method includes receiving a sequence of matrix arithmetic operations to be performed on the multi-dimensional Boolean matrix array. The method includes determining, from the operation, whether bit data in a memory layout is to be updated so as to be reordered into a different bit vectorization dimension, for each matrix arithmetic operation in the received sequence. The method includes, for each matrix arithmetic operation in the received sequence, if it is determined that the memory layout is to be updated, converting a current bit data into a reordered layout in accordance with the different bit vectorization dimension. The method includes, for each matrix arithmetic operation in the received sequence, updating indicative bit vectorization parameters, after the current bit data is converted into the reordered layout.

[0020] The method employs the processor device to facilitate bit-level addressing on architectures that do not natively support bitlevel addressing. The method alternates the dimensions of bit vectorization within a matrix of bit values during matrix arithmetic operations. The method converts a bit data structure from one vectorized structure / format to another vectorized structure, while optionally maintaining multiple replicas / cached versions of the bit data structure across different vectorized formats.

[0021] The method facilitates efficient and consistent Boolean computing on existing systems (i.e., non-single bit architectures) that lack native bit-level addressing by providing an adaptable abstraction layer optimized for the hardware. This approach simplifies integration with computing acceleration and eliminates the need for complex and error-prone bit-level addressing resolution at the application level. By offering simpler, more streamlined API routines, the method eliminates application overhead associated with mapping between Boolean and memory layouts, thereby reducing the volume of data transfer between application layers.

[0022] The method manages bit vectorization, thereby preventing the application from implementing suboptimal routines for converting between vectorized dimensions, both in terms of memory and computation. This management significantly reduces the data transfer overhead between the application and the deep learning frameworks. Moreover, the method supports efficient bit vectorized dimension transitions, such as in-place conversions, with minimal caching requirements. The method enables parallel processing by handling matrix sub-blocks simultaneously or maintaining multiple replicas of data in various bit structures to minimize the computational load. Additionally, the method ensures that resulting matrices are contiguous, enhancing matrix operations through improved memory locality.

[0023] The method provides an abstraction layer that seamlessly maps Boolean layouts to memory and vice versa, specifically designed for Boolean deep learning applications. The method facilitates the transformation of multi-dimensional n-bit arrays, adapting both the processor device and the method to optimize performance on hardware platforms. The processor device and the method enable in-place matrix transformations, ensuring efficient and tailored operations for the underlying hardware.

[0024] Optionally, the processor device stores a plurality of memory layouts for the multi-dimensional Boolean matrix array. Optionally, one of the plurality of memory layouts is a row major layout and another of the plurality of memory layouts is a column major layout. Preferably, one of the matrix arithmetic operations is a matrix multiplication operation. Optionally, the memory layout includes vectorized single bit values.

[0025] Optionally, the processor device is integrated into a deep learning framework as an extension library. Optionally, the processor device stores parameters indicating the matrix dimension of consecutive bit values that are arranged in linear storage. Optionally, the parameters are bit stride values in each dimension of the matrix. According to a second aspect, there is provided a system comprising means adapted for carrying out all the steps of the above described method.

[0026] The system employs the processor device to facilitate bit-level addressing on architectures that do not natively support bit-level addressing. The system alternates the dimensions of bit vectorization within a matrix of bit values during a sequence of matrix arithmetic operations. The system converts a bit data structure from one vectorized structure / format to another vectorized structure, while optionally maintaining multiple replicas / cached versions of the bit data structure across different vectorized formats. The system enables the dynamic switching between vectorized structures based on the inference request / system load.

[0027] The system facilitates efficient and consistent Boolean computing on existing systems (i.e., non-single bit architectures) that lack native bit-level addressing by providing an adaptable abstraction layer optimized for the hardware. This approach simplifies integration with computing acceleration and eliminates the need for complex and error-prone bit-level addressing resolution at the application level. By offering simpler, more streamlined API routines, the system eliminates application overhead associated with mapping between Boolean and memory layouts, thereby reducing the volume of data transfer between application layers.

[0028] The system manages bit vectorization, thereby preventing the application from implementing suboptimal routines for converting between vectorized dimensions, both in terms of memory and computation. This management significantly reduces the data transfer overhead between the application and the deep learning frameworks. Moreover, the system supports efficient bit vectorized dimension transitions, such as in-place conversions, with minimal caching requirements. The system enables parallel processing by handling matrix sub-blocks simultaneously or maintaining multiple replicas of data in various bit structures to minimize the computational load. Additionally, the system ensures that resulting matrices are contiguous, enhancing matrix operations through improved memory locality.

[0029] The system provides an abstraction layer that seamlessly maps Boolean layouts to memory and vice versa, specifically designed for Boolean deep learning applications. The system facilitates the transformation of multi-dimensional n-bit arrays, adapting both the processor device and the system to optimize performance on hardware platforms. The processor device and the system enable in-place matrix transformations, ensuring efficient and tailored operations for the underlying hardware.

[0030] According to a third aspect, there is provided a computer program including instructions for carrying out all the steps of the above described method, when the computer program is executed on a computer system.

[0031] Therefore, in contradistinction to the existing solutions, the system employs the processor device to facilitate bit-level addressing on architectures that do not natively support bit-level addressing. The system alternates the dimensions of bit vectorization within a matrix of bit values during matrix arithmetic operations. The system converts a bit data structure from one vectorized structure / format to another vectorized structure, while optionally maintaining multiple replicas / cached versions of the bit data structure across different vectorized formats. The system facilitates efficient and consistent Boolean computing on existing systems (i.e., non-single bit architectures) that lack native bit-level addressing by providing an adaptable abstraction layer optimized for the hardware. This approach simplifies integration with computing acceleration and eliminates the need for complex and error-prone bit-level addressing resolution at the application level. By offering simpler, more streamlined API routines, the system eliminates application overhead associated with mapping between Boolean and memory layouts, thereby reducing the volume of data transfer between application layers.

[0032] These and other aspects of the disclosure will be apparent from the implementation s) described below. BRIEF DESCRIPTION OF DRAWINGS

[0033] Implementations of the disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0034] FIGS. 1A & IB (PRIOR ART) illustrate a typical coordinate-wise arithmetic operation of 2 vectors A and B in real valued systems and Boolean systems;

[0035] FIGS. 2A & 2B (PRIOR ART) illustrate a typical contextualization in a Boolean linear layer training;

[0036] FIG. 3 is a block diagram that illustrates a system of training a Boolean deep learning model in accordance with an implementation of the disclosure;

[0037] FIG. 4 illustrates a block diagram of a system including a multi-dimensional Boolean matrix array for training a Boolean deep learning model in accordance with an implementation of the disclosure;

[0038] FIGS. 5A-5D illustrate a bit vectorization performed by a system of training a Boolean deep learning model in accordance with an implementation of the disclosure;

[0039] FIG. 6 illustrates an exemplary view of a system in accordance with an implementation of the disclosure;

[0040] FIGS. 7A-7B are flow diagrams that illustrate a method of training a Boolean deep learning model in accordance with an implementation of the disclosure; and

[0041] FIG. 8 is an illustration of a computer system in which the various architectures and functionalities of the various previous implementations may be implemented.

[0042] DETAILED DESCRIPTION OF THE DRAWINGS

[0043] Implementations of the disclosure provide a method of training a Boolean deep learning model at a processor device. Moreover, the disclosure relates to a system of training the Boolean deep learning model at the processor device.

[0044] To make solutions of the disclosure more comprehensible for a person skilled in the art, the following implementations of the disclosure are described with reference to the accompanying drawings.

[0045] Terms such as "a first", "a second", "a third", and "a fourth" (if any) in the summary, claims, and foregoing accompanying drawings of the disclosure are used to distinguish between similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that the terms so used are interchangeable under appropriate circumstances, so that the implementations of the disclosure described herein are, for example, capable of being implemented in sequences other than the sequences illustrated or described herein. Furthermore, the terms "include" and "have" and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, a method, a system, a product, or a device that includes a series of steps or units, is not necessarily limited to expressly listed steps or units but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or device.

[0046] FIG. 3 is a block diagram that illustrates a system of training a Boolean deep learning model 308 in accordance with an implementation of the disclosure. The system includes a processor device 302. The processor device 302 receives and processes one or more requests from a Boolean deep learning application 306. The Boolean deep learning application 306 utilizes the Boolean deep learning model 308. The Boolean deep learning application 306 is a specific software program or application that is built on top of a Boolean deep learning framework 304 and utilizes the capabilities provided by the Boolean deep learning framework 304 to solve a particular problem. The Boolean deep learning framework 304 is a software platform or a collection of libraries and tools that provides the infrastructure necessary to build, train, and deploy deep learning models using Boolean operations. Optionally, the processor device 302 is integrated into a deep learning framework (e.g., the Boolean deep learning framework 304) as an extension library. The processor device 302 may be a bit stride processor (BSP) device.

[0047] The Boolean deep learning application 306 constructs data structures in a binary layout view. That is, the Boolean deep learning application 306 allocates data matrices using a word size as small as a single bit. This is achieved by employing a dedicated software-defined type to hold single-bit values. The software-defined bit type is then interpreted by the processor device 302 (e.g. a low-level processor device), which maps the data into a two-tier memory access system, first at a byte level, and then at a bit level.

[0048] The processor device 302 receives a request from the Boolean deep learning application 306 to instantiate / create a multidimensional Boolean matrix array. The multi-dimensional Boolean matrix array includes one or more binary values (0s and 1 s), each representing weights or biases of the Boolean deep learning model 308. Optionally, the processor device 302 internally stores parameters indicating the matrix dimension of consecutive bit values that are arranged in linear storage. Optionally, the parameters are bit stride values in each dimension of the matrix or other indicative parameters. The bit stride values refer to a specific arrangement or offset of bits within a data structure or a memory layout 310. For example, Table 1 below illustrates a two-dimensional Boolean matrix array (i.e. , matrix of Boolean values), as instantiated by the Boolean deep learning application 306, including binary values. Table 2 and Table 3 demonstrate two possible memory layouts 310 for the same Boolean matrix array within the system (e.g., a computing device). In the memory layouts 310, single-bit values may be vectorized either in a row major layout / order as shown in Table 2 or in a column major layout / order as shown in Table 3. From a byte-level perspective, Table 2 and Table 3 include different elements within their respective Boolean matrix arrays. Optionally, the processor device 302 stores the memory layouts 310 for the multi-dimensional Boolean matrix array. Optionally, one of the memory layouts 310 is a row major layout (e.g., Table 2) and another of the memory layouts 310 is a column major layout (e.g., Table 3).

[0049] Table 1 : M=16 rows by N=16 columns of a two-dimensional Boolean matrix array. Table 2: M=16 rows by N=2 columns of an 8-bit Boolean matrix array in a row major layout.

[0050] Table 3: M=2 rows by N=16 columns of an 8-bit Boolean matrix array in a column major layout. The processor device 302 offers a low-level bit value mapping to the memory layout 310, which is commonly stored in the word of size of a multiple of byte. Optionally, the memory layout 310 includes vectorized single bit values. The processor device 302 receives a sequence of matrix arithmetic operations that needs to be performed on the multi-dimensional Boolean matrix array (e.g. , Table 1 ). The sequence of matrix arithmetic operations may be predefined. The sequence of matrix arithmetic operations is typically associated with the training process of the Boolean deep learning model 308, where the deep learning model's parameters (i.e., weights and biases) are updated.

[0051] In the sequence of matrix arithmetic operations, a deep learning training epoch includes a forward pass, a backpropagation, and weight and bias updates. The deep learning training epoch refers to one complete cycle through the entire training dataset by the Boolean deep learning model 308. Preferably, one of the matrix arithmetic operations is a matrix multiplication operation.

[0052] The processor device 302 is configured to automatically arrange bit data in the memory layout 310 that is suited for the current matrix operation. For example, when performing a forward matrix multiplication on weights in a linear layer of the Boolean deep learning model 308, the processor device 302 first receives instructions indicating that the forward matrix multiplication needs to be performed on the weights. The received instruction also includes an intended vector order (e.g., a row major layout or a column major layout) to perform the arithmetic operation. For each matrix arithmetic operation in the received sequence, the processor device 302 determines if the bit data in the memory layout 310 needs to be updated or reordered into a different bit vectorization dimension (e.g., a row major layout or a column major layout). This determination is based on whether the operation requires a change in the bit vectorization dimension (i.e., the way bits are organized within the matrix).

[0053] To optimize memory spatial locality for performing the matrix arithmetic operation (e.g., vector-to-vector dot products), the processor device 302 determines if the bit data in the memory layout 310 needs to be updated. If the update is necessary, the processor device 302 converts a current bit data to a reordered / desired layout and updates the relevant bit data vectorization parameters). For each matrix arithmetic operation in the received sequence, if it is determined that the memory layout 310 update is necessary, the processor device 302 converts the current bit data into the reordered layout according to the different bit vectorization dimension. This involves reorganizing the bits in the Boolean matrix array to optimize or facilitate the specific matrix arithmetic operation being performed. The immediate subsequent forward passes may not trigger a memory layout change for the weight data, as the current data structure on the memory meets the conditions for the arithmetic matrix operation (i.e., spatial locality).

[0054] Similarly, during the backpropagation phase (i.e., a backward pass), the processor device 302 identifies the need to reorder the bit data into an alternate bit vectorization dimension (e.g., a row major layout or a column major layout). The processor device 302 converts the data structure in memory when processing the first batch of incoming gradient values. Subsequent backward passes may not necessitate further data memory conversions. That is, after converting the bit data, the processor device 302 may read indicative bit vectorization parameters (i.e., the parameters that indicate the current bit vectorization status) for each matrix arithmetic operation in the received sequence. The indicative bit vectorization parameters are essential for tracking and managing the structure of the data as the sequence of operations progresses, thereby ensuring consistent data handling.

[0055] When updating weights and biases, the processor device 302 may either directly modify the bit values in the stored data layout or apply a memory layout conversion for the next forward pass / call or inference. In the case of a convolutional layer of the Boolean deep learning model 308, the shape of the weight matrix encompasses output channels, input channels, and kernel filters (e.g., ID, 2D, 3D). The bit vectorization dimension may be performed to any dimension of the weight matrix. However, during a Convolutional Neural Network (CNN) training epoch, it may be more efficient to apply bit vectorization along the input channel dimension for the forward pass and along the output channel dimension for the backward pass. The processor device 302 may be configured to prioritize computational efficiency over memory usage or to determine a balance between the two, based on the runtime context, such as available resources or application-specific configurations. The system includes a hardware platform 312 that is an underlying physical computing infrastructure that supports the operation of software applications, operating systems, or computing tasks. The hardware platform 312 encompasses all the physical components and architecture that enable the execution of programs and processes. The hardware platform 312 includes a central processing unit (CPU) or graphics processing unit (GPU), Random Access Memory (RAM), Storage devices like hard drives (HDDs) or solid-state drives (SSDs), etc.

[0056] The system converts binary layouts to memory layouts (i.e., storage formats) and from memory layouts to binary layouts. The processor device 302 optimizes Boolean computing workflows and addresses the limitations of traditional 1-byte stride operations in the multi-dimensional Boolean matrix array by efficiently managing binary layout transformations between layouts / storage formats (for example, from binary layouts to memory layouts and memory layouts to binary layouts). The processor device 302 is specifically customized for the host hardware platform 312, such as CUDA®, OpenCL™, Neural Processing Unit, NPU, and Tensor Processing Unit), TPU. The processor device 302 also offers an abstraction layer for Boolean deep learning frameworks for streamlining Boolean computing tasks.

[0057] The system employs the processor device to facilitate bit-level addressing on architectures that do not natively support bit-level addressing. The system alternates the dimensions of bit vectorization within a matrix of bit values during matrix arithmetic operations. The system converts a bit data structure from one vectorized structure / format to another vectorized structure, while optionally maintaining multiple replicas / cached versions of the bit data structure across different vectorized formats. The system enables the dynamic switching between vectorized structures based on the inference request / system load.

[0058] The system facilitates efficient and consistent Boolean computing on existing systems (i.e., non-single bit architectures) that lack native bit-level addressing by providing an adaptable abstraction layer optimized for the hardware. This approach simplifies integration with computing acceleration and eliminates the need for complex and error-prone bit-level addressing resolution at the application level. By offering simpler, more streamlined API routines, the system eliminates application overhead associated with mapping between Boolean and memory layouts, thereby reducing the volume of data transfer between application layers.

[0059] The system manages bit vectorization, thereby preventing the application from implementing suboptimal routines for converting between vectorized dimensions, both in terms of memory and computation. This management significantly reduces the data transfer overhead between the application and the deep learning frameworks. Moreover, the system supports efficient bit vectorized dimension transitions, such as in-place conversions, with minimal caching requirements. The system enables parallel processing by handling matrix sub-blocks simultaneously or maintaining multiple replicas of data in various bit structures to minimize the computational load. Additionally, the system ensures that resulting matrices are contiguous, enhancing matrix operations through improved memory locality.

[0060] The system provides an abstraction layer that seamlessly maps Boolean layouts to memory and vice versa, specifically designed for Boolean deep learning applications. The system facilitates the transformation of multi-dimensional n-bit arrays, adapting both the processor device and the system to optimize performance on hardware platforms. The bit stride processor device and the system enable in-place matrix transformations, ensuring efficient and tailored operations for the underlying hardware.

[0061] FIG. 4 illustrates a block diagram of a system including a multi-dimensional Boolean matrix array 414 for training a Boolean deep learning model 408 in accordance with an implementation of the disclosure. The system includes a processor device 402, a Boolean deep learning framework 404, and a hardware platform 412. The Boolean deep learning framework 404 includes a Boolean deep learning application 406 and a Boolean deep learning model 408. The functionalities of the system, the Boolean deep learning framework 404, the Boolean deep learning application 406, the Boolean deep learning model 408 and the processor device 402 are as described in FIG. 3. In an embodiment, the processor device 402 is configured to store multiple copies of the multi-dimensional Boolean matrix array 414, each corresponding to a different bit vectorization dimension (e.g., a row major layout / row pack or a column major layout / column pack or a depth major layout / depth pack) in memory layouts 410, to minimize data movement during runtime. The processor device 402 is then synchronize bit data updates across these multiple copies of the multi-dimensional Boolean matrix array 414. This approach remains advantageous for computation, as updates typically occur less frequently than the combined number of forward and backward passes. The processor device 402 manages single-bit precision conversions of the multi-dimensional Boolean matrix array 414, including binarization, in-place transposition, and reshaping, without requiring the reconstruction of the entire / full-scale matrix.

[0062] FIGS. 5A-5D illustrate a bit vectorization performed by a system of training a Boolean deep learning model in accordance with an implementation of the disclosure. The system includes a processor device. The processor device receives and processes one or more requests from a Boolean deep learning application. The Boolean deep learning application initializes a Boolean deep learning model (e.g., a neural network model) with a linear layer through a Boolean deep learning framework. To set up the linear weights, the Boolean deep learning application sends a request to instantiate / create a multi-dimensional Boolean matrix array 502 (e.g., as shown in Table 1) to the processor device, which may be integrated into the deep learning framework as an extension library. The processor device may automatically choose the most suitable vectorized dimension for inference (e.g., a row major layout or a column major layout). The multi-dimensional Boolean matrix array 502 includes one or more binary values (0s and Is), each representing weights or biases of the deep learning model as shown in FIG. 5A and FIG. 5B.

[0063] During training, the bit vectorization dimension (e.g., a row major layout or a column major layout) already aligns with the spatial locality requirements for a forward pass matrix multiplication. When the backpropagation begins, and the first batch of gradient outputs is passed to the layer for a backward pass, the processor device, which manages all arithmetic operations on the bit data, checks if the current bit data vectorization satisfies the spatial locality needed for matrix multiplication. The processor device converts the data structure in memory when processing the first batch of incoming gradient values.

[0064] During conversion, the processor device switches the vectorization dimension on sub-blocks of the multi-dimensional Boolean matrix array 502 (i.e., bit data array). To maximize parallelism and utilize shared memory 504 effectively, the sub-blocks may meet certain dimensional requirements, such as capturing enough bit data elements to reconstruct a bit vector in the targeted direction. For example, with a word size of 16 bits, a sub-block may contain a multiple of 16 words to generate 16 words of 16 bits in the other vectorization dimension.

[0065] As shown in FIG. 5A, for a sub-block of the multi-dimensional Boolean matrix array 502 consisting of 16-word entries processed by 16 threads 506, each thread (e.g., 506A) reads a single word from the sub-block and copies it into the shared memory 504. Once all sub-block entries are stored in the shared memory 504, new words, in the sense of achieving the desired bit vectorization dimension, can be composed from the shared memory 504 (i.e., cached entries) as shown in FIG. 5B. In an embodiment, each thread is assigned to compose a single byte (i.e. a subword) of the new words (e.g. word may be of length 1-2-4 bytes). Each thread fetches the bit values from the shared memory using its local register to store the bits composing the subword, and finally, each 506 writes its registers back into the sub-block of the Boolean matrix array 502.

[0066] In an embodiment, the processor device may allocate a new destination buffer instead of performing an in-place conversion. In another instance, the processor device is configured to maintain a cached version of the bit data in another / different bit vectorization dimension. This configuration may be established at the beginning of the training process and the cached data can be discarded once the training is completed. During the training, the processor device performs operations to the appropriate bit data buffer address based on the intended / required bit vector order for the arithmetic matrix operation being executed. When an update occurs, the processor device simultaneously updates the bit data in both the original and cached data. In FIG. 5C, the processor device 510 abstracts bit-level access for the Boolean deep learning application 512. The processor device 510 internally organizes bits into contiguous bytes. The examples of bit-level operations that the Boolean deep learning application 512 may invoke include sparse matrix update (i.e., Bit indices to flip) operation 514, matrix diagonal bit-level operation 516, and conversion from column-packed to row-packed bit-level operation 518, without an intermediate unpacked representation.

[0067] In the matrix diagonal bit-level operation 516, the processor device 510 instantiates multiple threads 506 to read each byte of data encapsulating the requested bit values and store this data in the shared memory 504. Each thread 506 is then assigned the task of constructing a byte of data to be returned to the Boolean deep learning application 512, enabling all threads 506 to operate concurrently.

[0068] While modem computing architectures support 32, 64, or 128-bit registers, data transfer between different memory levels often incurs high latency. To mitigate this, only the requested bit values, arranged consecutively in a memory buffer, may be transferred before any data movement.

[0069] In the sparse matrix update operation 514, the processor device 510 optimizes matrix bit values updates by applying write instruction to the memory only when necessary, thereby significantly reducing the overall number of write operations compared to applying a bitmask across the entire multi-dimensional Boolean matrix array 502. The processor device 510 may invoke multiple threads 506 to read individual words from the bitmask matrix, and each thread detects non-zero bits and applies the corresponding bit updates to the bit data of the Boolean matrix array 502.

[0070] In the matrix transposition bit-level, binary matrices may not be directly transposed by non-Boolean arithmetic frameworks. The processor device may first convert the bit data structure into an appropriate dimension for transposition before the Boolean deep learning framework performs the matrix transposition byte-level.

[0071] For instance, the original bit data in the memory layout includes 16 rows and 2 columns (e.g., a row major layout) as illustrated in the below step 1. The processor device switches bit vectorization dimension to 2 rows and 16 columns (e. g . , a column maj or layout) in the memory layout as illustrated in the below step 2 by performing the operation 518. The processor device then performs matrix transposition as illustrated in the below step 3, where the rows are swapped with the columns to complete the matrix transposition.

[0072] Step 1 : Original bit data in the memory layout includes 16 rows and 2 columns.

[0073] Step 2: switch bit vectorization dimension 2 rows and 16 columns, using the operation 518.

[0074] Step 3: Matrix transposition

[0075] In FIG. 5D, which illustrates conversion from column-packed to row-packed without an intermediate unpacked representation, the bit data includes 4 bits. That is, the bit data in the memory layout 520 includes 8 rows and 2 columns (i.e., a row major layout, row, m=8 and column, n=2). The processor device switches bit vectorization dimension to 2 rows and 8 columns (i.e., a column major layout, m=2 and column, n=8) in the memory layout 520 and swaps the bit data values (i.e., swapping its rows with its columns). In an embodiment, the processor device repeats bit data transpose, updates the bit data in the memory layout 520 and swaps the bit data values for each block until the final memory layout 520 is obtained and preserving spatial locality of the Boolean deep learning array.

[0076] FIG. 6 illustrates an exemplary view of a system in accordance with an implementation of the disclosure. The system includes a processor device 602. The processor device 602 receives and processes one or more requests from a Boolean deep learning application 606. The Boolean deep learning application 606 utilizes the Boolean deep learning model 308. The Boolean deep learning model 608 is represented by multi-dimensional Boolean matrix arrays including binary values. The Boolean deep learning application 606 is a specific software program or application that is built on top of a Boolean deep learning framework 604 and utilizes the capabilities provided by the Boolean deep learning framework 604 to solve a particular problem. The Boolean deep learning framework 604 is a software platform or a collection of libraries and tools that provides the infrastructure necessary to build, train, and deploy deep learning models using Boolean operations. Optionally, the processor device 602 is integrated into the deep learning framework (e.g., the Boolean deep learning framework 604) as an extension library.

[0077] The Boolean deep learning application 606 constructs data structures in a binary layout view. That is, the Boolean deep learning application 606 allocates data matrices using a word size as small as a single bit. This is achieved by employing a dedicated software-defined type to hold single-bit values. The software-defined bit type is then interpreted by the processor device 602 (e.g. a low-level processor device), which maps the data into a two-tier memory access system, first at a byte level, and then at a bit level.

[0078] The system includes a hardware platform 610 which is an underlying physical computing infrastructure that supports the operation of software applications, operating systems, or computing tasks. The hardware platform 610 encompasses all the physical components and architecture that enable the execution of programs and processes. The hardware platform 610 includes a central processing unit (CPU) or graphics processing unit (GPU) 612, Video Random Access Memory (VRAM) 614. VRAM 614 is a type of memory used in GPUs 612 to store image data and textures that the GPU 612 needs to render graphics. The hardware platform 610 further includes Random Access Memory (RAM), Storage devices like hard drives (HDDs) or solid- state drives (SSDs), etc. The functionalities of the system are described in FIG. 3.

[0079] FIGS. 7A-7B are flow diagrams that illustrate a method of training a Boolean deep learning model in accordance with an implementation of the disclosure. The method is performed at a processor device. At a step 702, a request is received from a Boolean deep learning application which uses the Boolean deep learning model, to instantiate a multi-dimensional Boolean matrix array including one or more binary values. Each of the binary values represents a weight or bias of the deep learning model. At a step 704, a sequence of matrix arithmetic operations to be performed on the multi-dimensional Boolean matrix array is received. At a step 706, for each matrix arithmetic operation in the received sequence, the method determines, from the operation, whether bit data in a memory layout is to be updated so as to be reordered into a different bit vectorization dimension. At a step 708, for each matrix arithmetic operation in the received sequence, if it is determined that the memory layout is to be updated, a current bit data is converted into a reordered layout in accordance with the different bit vectorization dimension. At a step 710, for each matrix arithmetic operation in the received sequence, indicative bit vectorization parameters are updated after converting the bit data into the reordered layout (i.e., in accordance with the results of the converting step). The method employs the processor device to facilitate bit-level addressing on architectures that do not natively support bitlevel addressing. The method alternates the dimensions of bit vectorization within a matrix of bit values during matrix arithmetic operations. The method converts a bit data structure from one vectorized structure / format to another vectorized structure, while optionally maintaining multiple replicas / cached versions of the bit data structure across different vectorized formats.

[0080] The method facilitates efficient and consistent Boolean computing on existing systems (i.e., non-single bit architectures) that lack native bit-level addressing by providing an adaptable abstraction layer optimized for the hardware. This approach simplifies integration with computing acceleration and eliminates the need for complex and error-prone bit-level addressing resolution at the application level. By offering simpler, more streamlined API routines, the method eliminates application overhead associated with mapping between Boolean and memory layouts, thereby reducing the volume of data transfer between application layers.

[0081] The method manages bit vectorization, thereby preventing the application from implementing suboptimal routines for converting between vectorized dimensions, both in terms of memory and computation. This management significantly reduces the data transfer overhead between the application and the deep learning frameworks. Moreover, the method supports efficient bit vectorized dimension transitions, such as in-place conversions, with minimal caching requirements. The method enables parallel processing by handling matrix sub-blocks simultaneously or maintaining multiple replicas of data in various bit structures to minimize the computational load. Additionally, the method ensures that resulting matrices are contiguous, enhancing matrix operations through improved memory locality.

[0082] Optionally, the processor device stores one or more memory layouts for the multi-dimensional Boolean matrix array. Optionally, one of the memory layouts is a row major layout and another one of the memory layouts is a column major layout. Preferably, one of the matrix arithmetic operations is a matrix multiplication operation. Optionally, the memory layout includes vectorized single bit values.

[0083] Optionally, the processor device is integrated into a deep learning framework as an extension library. Optionally, the processor device stores parameters indicating the matrix dimension of consecutive bit values that are arranged in linear storage. Optionally, the parameters are bit stride values in each dimension of the matrix.

[0084] FIG. 8 is an illustration of a computer system in which the various architectures and functionalities of the various previous implementations may be implemented. As shown, the computer system 800 includes at least one processor 804 that is connected to a bus 802, wherein the computer system 800 may be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), Hyper Transport, or any other bus or point-to-point communication protocol (s). The computer system 800 also includes a memory 806.

[0085] Control logic (software) and data are stored in the memory 806 which may take a form of random-access memory (RAM). In the disclosure, a single semiconductor platform may refer to a sole unitary semiconductor-based integrated circuit or chip. It should be noted that the term single semiconductor platform may also refer to multi-chip modules with increased connectivity which simulate on-chip modules with increased connectivity which simulate on-chip operation, and make substantial improvements over utilizing a conventional central processing unit (CPU) and bus implementation. Of course, the various modules may also be situated separately or in various combinations of semiconductor platforms per the desires of the user.

[0086] The computer system 800 may also include a secondary storage 810. The secondary storage 810 includes, for example, a hard disk drive and a removable storage drive, representing a floppy disk drive, a magnetic tape drive, a compact disk drive, digital versatile disk (DVD) drive, recording device, universal serial bus (USB) flash memory. The removable storage drive at least one of reads from and writes to a removable storage unit in a well-known manner. Computer programs, or computer control logic algorithms, may be stored in at least one of the memory 806 and the secondary storage 810. Such computer programs, when executed, enable the computer system 800 to perform various functions as described in the foregoing. The memory 806, the secondary storage 810, and any other storage are possible examples of computer-readable media.

[0087] In an implementation, the architectures and functionalities depicted in the various previous figures may be implemented in the context of the processor 804, a graphics processor coupled to a communication interface 812, an integrated circuit (not shown) that is capable of at least a portion of the capabilities of both the processor 804 and a graphics processor, a chipset (namely, a group of integrated circuits designed to work and sold as a unit for performing related functions, and so forth).

[0088] Furthermore, the architectures and functionalities depicted in the various previous-described figures may be implemented in a context of a general computer system, a circuit board system, a game console system dedicated for entertainment purposes, an application-specific system. For example, the computer system 800 may take the form of a desktop computer, a laptop computer, a server, a workstation, a game console, an embedded system.

[0089] Furthermore, the computer system 800 may take the form of various other devices including, but not limited to a personal digital assistant (PDA) device, a mobile phone device, a smart phone, a television, and so forth. Additionally, although not shown, the computer system 800 may be coupled to a network (for example, a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, or the like) for communication purposes through an VO interface 808.

[0090] It should be understood that the arrangement of components illustrated in the figures described are exemplary and that other arrangement may be possible. It should also be understood that the various system components (and means) defined by the claims, described below, and illustrated in the various block diagrams represent components in some systems configured according to the subject matter disclosed herein. For example, one or more of these system components (and means) may be realized, in whole or in part, by at least some of the components illustrated in the arrangements illustrated in the described figures.

[0091] In addition, while at least one of these components are implemented at least partially as an electronic hardware component, and therefore constitutes a machine, the other components may be implemented in software that when included in an execution environment constitutes a machine, hardware, or a combination of software and hardware.

[0092] Although the disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims.

Claims

CLAIMS1. A method of training a Boolean deep learning model (308, 408, 608), comprising steps, performed at a processor device (302, 402, 510, 602), of: receiving, from a Boolean deep learning application (306, 406, 512, 606) which uses the Boolean deep learning model, a request to instantiate a multi-dimensional Boolean matrix array (414, 502) comprising a plurality of binary values, each representing a weight or bias of the Boolean deep learning model; receiving a sequence of matrix arithmetic operations to be performed on the multi-dimensional Boolean matrix array; for each matrix arithmetic operation in the received sequence, determining, from the operation, whether bit data in a memory layout (310, 410, 520) is to be updated so as to be reordered into a different bit vectorization dimension; for each matrix arithmetic operation in the received sequence, where the determining step determines that the memory layout is to be updated, converting a current bit data into a reordered layout in accordance with the different bit vectorization dimension; and for each matrix arithmetic operation in the received sequence, updating indicative bit vectorization parameters in accordance with the results of the converting step.

2. The method of claim 1, wherein the processor device stores a plurality of memory layouts for the multi-dimensional Boolean matrix array.

3. The method of claim 2, wherein one of the plurality of memory layouts is a row major layout.

4. The method of claim 3, wherein another of the plurality of memory layouts is a column major layout.

5. The method of claim 1, wherein one of the matrix arithmetic operations is a matrix multiplication operation.

6. The method of claim 1, wherein the memory layout includes vectorized single bit values.

7. The method of claim 1, wherein the processor device is integrated into a deep learning framework (304, 404, 604) as an extension library.

8. The method of claim 1, wherein the processor device stores parameters indicating the matrix dimension of consecutive bit values that are arranged in linear storage.

9. The method of claim 8, wherein the parameters are bit stride values in each dimension of the matrix.

10. A system comprising means adapted for carrying out all the steps of the method according to any preceding method claim.

11. A computer program comprising instructions for carrying out all the steps of the method according to any preceding method claim, when said computer program is executed on a computer system.

Images