A MATRIX MULTIPLICATION ALGORITHM WITH TIME COMPLEXITY OF O(n 2), EXTENDABLE FOR SOLVING SYSTEMS OF LINEAR EQUATIONS AND IMPLEMENTABLE ON PARALLEL PROCESSING ARCHITECTURES

Abstract

This invention introduces a parallel matrix multiplication algorithm achieving O(n²) time complexity through: 1. Binary Fusion Technique: Multi-layer bitwise data representation. Vectorized counting-logical operations (AND + popcount). Kronecker-based weighted reconstruction. 2. Key Advantages: Eliminates floating-point rounding errors. Reduces memory bandwidth requirements. Hardware-agnostic design (GPU / FPGA / ASIC compatible). 3. Applications: Machine learning acceleration. Cryptographic systems optimization. Large-scale scientific computing. The algorithm transforms numerical operations into parallel bitwise processes while maintaining full precision. Compared to conventional methods (e.g. Strassen), it demonstrates superior scalability for large matrices without recursive decomposition.

Description

A matrix multiplication algorithm with time complexity of O(n2), extendable for solving systems of linear equations and implementable on parallel processing architectures

[0001] The present invention pertains to:- Numerical computation,- Optimization of matrix algorithms, and- Parallel processing systems.

[0002] Core InnovationThe "Binary Fusion" algorithm features:- Matrix multiplication with O(n²) time complexity,- Extendable applications to:- Solving linear equation systems,- Matrix inversion,- Large-scale system modeling.

[0003] Implementation  Deployable as:- Software-based solutions,- Hardware implementations, or- Hybrid (software-hardware) configurations.Limitations of Prior Art & Technical Objectives

[0004] Current matrix multiplication algorithms (e.g., standard triple-loop, Strassen, Coppersmith-Winograd) exhibit significant limitations:

[0005] 1. Computational Challenges:- High time complexity (O(n³)) in classical approaches- Numerical instability in Strassen's algorithm due to round-off errors- Theoretical complexity improvements in Coppersmith-Winograd lacking practical applicability

[0006] 2. Implementation Difficulties:- Complex recursive structures requiring substantial memory overhead- Limited adaptability to parallel processing architectures (GPU / FPGA)- Implementation constraints in real-world systems

[0007] Technical Objectives of the Invention

[0008] This invention provides:- A simplified, stable matrix multiplication algorithm with O(n²) time complexity- Elimination of recursive subdivisions and associated computational overhead- Native compatibility with parallel processing architectures (GPU / FPGA)- Memory-optimized operations without numerical instability issues

[0009] Key improvements over prior art:- Direct hardware implementation capability- Avoidance of complex recursive structures- Maintained numerical precision- Scalable parallel processing support.

[0010] Key distinctions from prior art

[0011] Existing patents (e.g., US5170370A, US10534838B2, US11194886B2, US10621269B2, US10489479B1, US11334648B2, US20230289398A1, EP3861467B1 and TW202020598A) address matrix multiplication optimization partially.

[0012] None provide a direct O(n²) algorithm without recursive subdivisions.

[0013] None are fully compatible with parallel GPU / FPGA architectures.

[0014] None ensure numerical stability while optimizing memory usage.Key Technical Improvements

[0015] 1. Computational Efficiency- O(n²) time complexity for matrix multiplication- Utilizes:• Multi-layer bitwise representation• Vectorized AND + popcount operations• Kronecker weighting system2.Hardware Optimization- Native GPU / FPGA compatibility- Reduced memory requirements- Parallel processing support3.Performance Advantages- Maintained numerical precision- Scalable for large matrices- Lower resource utilization vs. Strassen / Coppersmith-Winograd.

[0016] Existing matrix multiplication algorithms (classical triple-loop, Strassen, Coppersmith-Winograd) exhibit high computational complexity (O(n³) or unstable O(n2) with numerical issues).

[0017] Recursive or hierarchical structures in prior methods create implementation difficulties, including large memory overhead and poor adaptability to parallel architectures.

[0018] Current methods lack direct compatibility with modern GPU / FPGA systems, making hardware acceleration inefficient.

[0019] Maintaining numerical precision while reducing computation time remains a significant unresolved challenge.

[0020] Overall, there is a need for an algorithm that is simpler, stable, memory-efficient, and natively parallelizable, while achieving O(n²) time complexity.

[0021] Introduces a direct O(n²) matrix multiplication algorithm eliminating recursive subdivisions.

[0022] Ensures numerical stability even for large matrices.

[0023] Fully compatible with parallel GPU / FPGA architectures, enabling hardware acceleration.

[0024] Provides memory-optimized operations, reducing overhead compared to classical algorithms.

[0025] Supports scalable parallel processing, allowing simultaneous computations across multiple cores.

[0026] Simplifies implementation without compromising accuracy or performance, addressing limitations of prior art.

[0027] Reduces computation time from O(n³) to O(n²) while maintaining accuracy.

[0028] Eliminates complex recursive operations, simplifying implementation.

[0029] Enables efficient parallelization on GPU / FPGA architectures.

[0030] Optimizes memory usage, reducing hardware requirements.

[0031] Maintains numerical stability even for very large matrices.

[0032] Provides scalable and reliable performance, suitable for both academic and industrial applications.Fig.1

[0033] shows a flowchart of the Binary Fusion matrix multiplication algorithm, illustrating the overall process from the input matrices to the reconstructed output.Fig.2

[0034] shows a schematic diagram of the complete workflow of the Binary Fusion algorithm, where each stage of the process is represented with a symbolic block and a brief description.Fig.3

[0035] shows a parallelization diagram of the Binary Fusion algorithm, in which the input matrices are represented as binary layers processed concurrently by GPU threads, with subsequent operations performed in FPGA cores using logical AND and popcount functions.

[0036] This invention introduces a novel, highly parallelizable computational method for performing matrix multiplication. Instead of relying on conventional numerical operations such as floating-point multiplication and addition, the proposed method employs a layered binary representation of input elements and executes counting–logical operations with structured weighting.

[0037] Step 1 – Initial Input Normalization

[0038] All elements of the input matrices are first mapped to a fixed binary precision. This mapping is performed by applying an appropriate offset, enabling negative or fractional numbers to be transformed into the domain of non-negative integer binary values.

[0039] To achieve this:

[0040] - The minimum numerical value in each input matrix (e.g., min(A) and min(B)) is identified separately.

[0041] - An offset is defined so that all normalized values become non-negative.- If fractional precision is required, a scaling factor (e.g. ) is applied to shift values into the integer domain.

[0042] Step 2 – Binary Layered Representation

[0043] Each normalized element of the input matrices is decomposed into a fixed-length vector of bits (e.g. 8-bit or 16-bit).

[0044] By grouping bits of the same positional value across all elements, parallel binary layers are formed.

[0045] Thus, each input matrix (e.g. A) is transformed into a unique three-dimensional structure of size , where m and n are the original matrix dimensions and L is the number of layers.

[0046] Although the algorithm implementation decomposes numbers directly in binary for computational efficiency, for explanatory purposes—especially for non-technical audiences—this can be conceptualized as a two-stage process:1. Separation of decimal-base components.

[0047] 2. Conversion of each component into binary representation.(See.)

[0048] Step 3 – Structural Organization for Parallel Processing

[0049] The binary layers of the input matrices are structurally rearranged so that vertical and horizontal alignment patterns correspond to their computational positions.Simultaneously, two independent weight vectors v and w are generated according to the ordering of the binary layers.

[0050] - These weight vectors represent the contribution of each layer to numerical reconstruction.

[0051] - The values are arranged as powers of two, from least significant to most significant, and their lengths match the number of corresponding layers.This organization provides the structural foundation for cross-combination operations with high parallelization capability. (See.)

[0052] Step 4 – Counting–Logical Multiplication of Binary Data

[0053] At this stage, counting operations are applied to the logical product of binary matrices, producing the Counting–Logical matrix (CL). This is performed vectorially and in parallel, using the logical AND operator—where only the combination of two “1” bits produces an output of 1, and all other combinations yield 0.

[0054] This simple, optimized approach dramatically accelerates computation and is fully parallelizable, with each GPU thread responsible for bitwise combinations at specific layer positions.

[0055] - Memory consumption is minimized.

[0056] - No integer or floating-point multiplications are required during execution.

[0057] -The combination of binary decomposition and counting–logical multiplication constitutes the foundation of the Binary Fusion method. (See Figures 2.)

[0058] Step 5 – Weighting and Kronecker Combination

[0059] To reconstruct the final value of each element in the output matrix, a Kronecker product structure is used:

[0060] Identity matrices of suitable sizes are Kronecker-multiplied by the weight vectors to expand them into weight matrices.

[0061] This structure enables hierarchical combination of binary layers.

[0062] If certain layers or horizontal / vertical vectors of the CL matrix are devoid of meaningful data (zeros), entire or partial weight matrix blocks can be removed—improving processing speed and conserving memory / resources.

[0063] The CL matrix itself is a block matrix formed from cross-combinations of meaningful binary layers. (See.)

[0064] Step 6 – Reconstruction of the Output Matrix

[0065] The output matrix is reconstructed by structured multiplication of the weight matrices on both sides of the CL matrix.

[0066] This is equivalent to performing a weighted linear combination of the CL matrix blocks, with weights derived from Kronecker combinations of the weight matrices.

[0067] The number of such combinations is limited to , independent of the original matrix dimensions.

[0068] The algorithm maintains a time complexity of in this stage.

[0069] Step 7 – Offset Reversal (Dependent on Step 1)

[0070] If an initial offset or fractional scaling factor was applied in Step 1, offset reversal is carried out:

[0071] - Inverse scaling factors are applied.

[0072] - The offset is removed, which may involve subtracting offset - induced matrix terms - such as products of the input matrices with unit matrices (depending on multiplication side) and products of two unit matrices.

[0073] The final result is exactly equivalent to the conventional matrix product of the original matrices—without having used floating-point operations at any stage. This eliminates rounding and numerical instability errors.

[0074] Implementation Notes

[0075] The algorithm can be implemented with ease in environments such as MATLAB, Python, and the C language family.

[0076] Its structure inherently supports deployment on parallel processing architectures such as GPUs (including CUDA) and FPGAs. (See.)Examples

[0077] Example1.In one preferred embodiment of the invention, the proposed algorithm can be applied in the field of real-time image processing to perform matrix multiplication between pixel matrices.

[0078] In this embodiment, pixel colors are represented as non-negative integers in the range of 0 to 255. Therefore, no offsetting stage for converting negative or fractional values is required, and the input values are directly transformed into layered binary representations. Each color channel of the image (such as R, G, or B) is processed as an independent matrix and decomposed into a three-dimensional structure of binary layers.

[0079] In GPU-enabled environments, the counting–logical operations between binary layers are executed in parallel by logic units based on FPGA (or equivalent logical modules). These operations consist of counting the positions where active bits (1s) in two matrices interact, thereby producing a counting matrix. Subsequently, weight vectors corresponding to layer positions are expanded into two weight matrices by applying Kronecker products of identity matrices to the vectors, which are then multiplied on both sides of the counting matrix. As a result, the output matrix representing the weighted multiplication of the counting blocks is reconstructed. This output is finally transformed back into displayable color values, replacing the original pixel values.

[0080] In this application, matrix multiplication between two images can represent various outcomes depending on the intended purpose. For example, when two images or two channels of the same scene are multiplied, the result may be a composite image reflecting the overlap of visual information. In more specific cases, when one matrix represents a filter or convolution kernel, its multiplication with the input image applies that filter, such as edge detection, blurring, or edge enhancement.

[0081] This embodiment significantly reduces the consumption of classical computational resources (multiplications and additions) while enabling real-time image processing in resource-constrained environments such as unmanned aerial vehicles (UAVs), surveillance systems, and smart cameras.

[0082] Example 2:In this example, the invention is used to accelerate neural network matrix operations:

[0083] Weight and input matrices in a neural network layer are converted into layered binary representations. Parallel bitwise counting reduces the computation needed for matrix multiplications in fully connected or convolution layers. Output matrices are reassembled into standard numerical values for activation functions.

[0084] Applications: deep learning training and inference on GPU-accelerated systems. This approach reduces the heavy reliance on floating-point multiplications, which are often a bottleneck in large-scale AI models. By using binary decomposition, the algorithm not only reduces energy consumption but also allows efficient parallelization on FPGA or GPU hardware. As a result, it enables faster training of deep networks and more efficient inference in edge devices with limited resources.

[0085] Example3:Here, the invention demonstrates performance on large-scale matrices (e.g., 10,000 × 10,000):

[0086] Decomposition and parallel bitwise counting allow the algorithm to scale efficiently without recursion overhead. Memory usage is optimized compared to classical approaches.

[0087] Applications: big data analytics, high-performance computing, and simulation models. In practice, this means that operations such as covariance analysis, finite element simulations, or large-scale optimization can be performed more rapidly and with reduced infrastructure requirements. The algorithm’s binary structure also makes it inherently more cache-friendly, leading to performance gains on multi-core CPUs and distributed computing environments. This scalability highlights its suitability for cloud-based and supercomputing platforms.

[0088] Example4:(Multi-Domain Application). The invention is applied across domains to show versatility:

[0089] Image processing, linear algebra, and neural network operations are combined in a single workflow.

[0090] Layered binary representations are reused across different matrices for different computations.

[0091] Demonstrates reduction in computational complexity and increased throughput in multi-task pipelines.

[0092] For instance, an integrated system can simultaneously handle real-time video analysis, numerical modeling, and AI inference without requiring three separate computational frameworks. By reusing the same binary decomposition structures, cross-domain workflows become both faster and more memory-efficient. This versatility suggests that the invention can act as a unifying computational backbone in heterogeneous applications, reducing development time and improving cross-compatibility between fields.

[0093] The industrial applications of the present invention include, but are not limited to, the following domains:

[0094] High-speed image processing systems, including real-time analysis for surveillance drones, smart cameras, and autonomous vehicles.

[0095] Numerical data compression and secure data processing, enabling efficient storage and retrieval of large-scale numerical datasets.

[0096] Cryptography-resistant and encrypted data reconstruction systems, providing high-performance computation while maintaining security requirements.

[0097] Real-time analytical systems, capable of processing streaming data for monitoring, diagnostics, or predictive analysis.

[0098] Numerical modeling of physical phenomena, including simulations in engineering, physics, and environmental studies.

[0099] Machine learning and artificial intelligence implementations, both in software and hardware-accelerated environments.

[0100] AI accelerators and hardware optimization systems, for improved performance in specialized computing units.

[0101] Low-resource AI applications, suitable for edge computing devices with limited computational capacity.

[0102] Potential applications in telecommunications, defense, computational medicine, and software-based simulation environments.

[0103] This invention enables significant reductions in computational overhead while maintaining high numerical accuracy, making it directly applicable across a wide range of industrial, scientific, and technological fields.

[0104] Patterson, D., & Hennessy, J. Computer Organization and Design, 5th Edition, 2013.

[0105] Cormen, T., Leiserson, C., Rivest, R., & Stein, C. Introduction to Algorithms, 4th Edition, 2022.

[0106] NVIDIA. CUDA C++ Programming Guide, 2025.

[0107] Zhang, X., et al. “Parallel Matrix Multiplication on FPGA Architectures.” IEEE Transactions on Computers, 2020.

[0108] Li, H., et al. “Bit-level Matrix Decomposition for High-performance Computing.” ACM Transactions on Mathematical Software, 2021.

[0109] US5170370A – Focuses on bit-oriented matrices; lacks precomputed tables and aligned pattern techniques.

[0110] US10534838B2 – Focuses on logical circuit selection; differs from this invention which emphasizes layered algorithmic decomposition.

[0111] US11194886B2 – Uses analog circuits; the current invention is fully digital with logic-based decision-making.

[0112] US10621269B2 – Systolic arrays and precision management; differs from the current invention’s binary-layered parallel approach.

[0113] US10489479B1 – Multi-dimensional convolution and row-wise processing; does not use binary-layered representation or logic-count multiplication.

[0114] US11334648B2 – FPGA-based acceleration for large matrices; focuses on hardware memory optimization, not layered numeric decomposition.

[0115] US20230289398A1 – GPU-based matrix multiplication algorithm; lacks binary representation and layer-weighted operations.

[0116] EP3861467B1 – Matrix architecture for machine learning; relies on compression and repetition reduction, not layer-wise numeric reconstruction.

[0117] TW202020598A – Multi-architecture matrix processing; emphasizes parallel scheduling rather than layer-based modeling.

[0118] Dongarra, J., et al. “High-Performance Matrix Multiplication: Algorithms and Architectures.” Journal of Parallel and Distributed Computing, 2019.

[0119] Volkov, V., & Demmel, J. “Benchmarking GPUs for Matrix Computations.” ACM Transactions on Mathematical Software, 2008.

[0120] Alom, M., et al. “Efficient GPU-Accelerated Matrix Multiplication.” IEEE Access, 2021.

[0121] Chen, Y., et al. “Layered Binary Matrix Techniques for Neural Networks.” Neural Computing & Applications, 2022.

Claims

A method for representing and preparing numerical matrix data for matrix multiplication in a counting–logical and parallel manner, comprising the following steps:Normalizing the input values into non-negative integers in case of decimal or negative values;Decomposing each element, in a parallel and extensive manner, into a string of binary bits (binary), forming binary layers;Forming a set of independent binary layers from bits of the same position across all elements, wherein the number of layers equals the bit-length of the largest input value for each matrix, independent of the matrix dimensions;Extracting two separate weight vectors aligned with the layers, corresponding to the positional value of the bits, arranged row-wise and column-wise for the left and right matrices, respectively;Performing counting–logical operations on the layers using the logical AND operator in a fully parallel and vectorized manner, converting numerical multiplication into a combination of active bits (1) in the layers.The method of Claim 1, wherein the layering of input data is performed in two stages, first decomposing each element into decimal-digit components and then independently converting each component into binary form, such that the combination of the resulting layers reconstructs the binary structure equivalent to direct binary decomposition, for improved interpretability and simplified layer extraction.The method of Claim 1, wherein to support decimal and negative values in the input matrices, a constant offset matrix is first added, followed by converting all values to non-negative integers, scaling by a suitable factor, such that binary layering can be performed without loss of precision or alteration of relative values.The method of Claim 1, wherein the binary layers of the left matrix are vertically arranged and the binary layers of the right matrix are horizontally arranged, and the logical AND operator is applied, in parallel and vectorized form, between corresponding layers, producing an output that counts positions of coinciding active bits (counting matrix). The overall process remains of order O(n²) since no classical multiplication or addition is performed.The method of Claim 4, wherein the weight vectors extracted from identity matrices corresponding to the rows and columns of the input matrices are expanded into weight matrices using the Kronecker product with the layer structure, and these weight matrices are multiplied from the left and right with the resulting counting matrix. The result is an exact weighted linear combination of the counting-matrix blocks without direct numerical multiplication, while the number of blocks is independent of the input matrix size, maintaining the time complexity at O(n²)The method of Claim 5, wherein layers, rows, or columns of the counting matrix containing only inactive values (0) or having no computational contribution are identified and removed before the weighted multiplication, and the corresponding rows or columns in the weight matrices are also removed in a vectorized manner, thereby reducing memory and processing requirements without loss of numerical reconstruction accuracy.The method of Claim 5, wherein the computational cost of direct multiplication of multi-digit numbers is replaced by decomposing into binary bits and performing distributed counting–logical interactions between layers, combined with the structured integration of the counting layers and weight matrices, maintaining the overall algorithm complexity at O(n²) and enabling implementation in parallel processing architectures.The method of Claim 3, wherein, if an offset matrix or scaling factor is applied to the two matrices during the initial preparation stage, an inversion process is performed at the end, including division by the scaling factors and removal of the offset-matrix contributions, reconstructing an exact numerical multiplication of the original input matrices without rounding error or numerical distortion.The method of Claim 8, wherein the offset-matrix contributions, including products of the input matrices with unit matrices on either side of the multiplication and the product of two unit matrices, are computed and stored in parallel during execution, so that the inversion process, including division by the scaling factors, does not require additional computation time, and the complexity of these complementary operations remains O(n²).A system for implementing counting–logical operations in which simultaneous counting between binary layers is executed on programmable architectures such as GPUs or FPGAs with independent logic units, wherein the bit-oriented and vectorized nature of the operations enables massive parallelism, leading to significant speed improvements and reduced resource consumption.

Images