Optimization of Sparsified Neural Network Layers for Semi-Digital Crossbar Architectures

Abstract

To provide a method for allowing previously trained and pruned networks to be optimized into crossbar architectures without losing the accuracy.SOLUTION: The method includes mapping a binary matrix to an undirected graph form, applying to the mapped binary matrix a two-way graph partition algorithm that minimizes edge cuts between partitions in the mapped binary matrix, applying a greedy algorithm recursively to find a set of row or column permutations that maximizes a transfer of non-zeros from sparse blocks to nonsparse blocks, and sparsifying or densifying the binary matrix according to the applied greedy algorithm.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure generally relates to a method and apparatus for modifying a sparse matrix so that sub-blocks become sparsifiers (or densifiers). [Background technology]

[0002] Demand for hardware accelerators for machine learning (ML) applications is increasing. A computation that dominates many such ML applications is matrix-vector multiplication. Pruning methods have been established to sparsify layers and generate a significant fraction of zero-valued elements. Sparsifying vector-matrix multiplication techniques have been developed for all-digital architectures, improving power and performance for inference tasks. For non-sparse matrices, matrix-vector multiplication can be performed very efficiently in analog fashion over a crossbar network. However, for pruned networks, such analog or semi-analog crossbar networks generally do not achieve the same power savings as all-digital designs because the overhead associated with array power supplies creates overhead that does not significantly reduce even when a significant fraction of weights are zero. To save power, zero-valued elements can be locally correlated. Summary of the Invention [Problem to be solved by the invention]

[0003] The technical problem to be solved by the present invention is to provide a method that can optimize a previously trained and pruned network into a crossbar structure without loss of accuracy.

[0004] The technical problem to be solved by the present invention is to provide a system and method for generating local sparsity that can be utilized to improve the overall performance of a Parallel Prefix Adder (PPA) in a Network on Chip (NoC) structure using a crossbar structure. [Means for solving the problem]

[0005] In an embodiment of the present disclosure, a method includes mapping a binary matrix to an undirected graph form, applying a bidirectional graph partitioning algorithm to the mapped binary matrix that minimizes edge cuts between partitions in the mapped binary matrix, recursively applying a greedy algorithm to find a set of row or column permutations that maximizes the transfer of non-zero values ​​from sparse blocks to non-sparse blocks, and sparsifying or densifying the binary matrix according to the applied greedy algorithm.

[0006] In an embodiment of the present disclosure, a system includes a memory and a processor, and the processor is configured to map a binary matrix into an undirected graph form, apply a bidirectional graph partitioning algorithm to the mapped binary matrix that minimizes edge cuts between partitions in the mapped binary matrix, recursively apply a greedy algorithm to find a set of row or column permutations that maximizes the transfer of non-zero values ​​from sparse blocks to non-sparse blocks, and sparsify or densify the binary matrix according to the applied greedy algorithm. [Brief explanation of the drawings]

[0007] Other aspects, features, and advantages of particular embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Figure 1] 1 illustrates a flow diagram for reordering a sparse matrix according to one embodiment of the present disclosure. [Figure 2] 1 illustrates a diagram of a sparse matrix according to one embodiment of the present disclosure. [Figure 3] 1 illustrates a graph of block sparsity versus number of blocks for a sparse matrix according to one embodiment of the present disclosure. [Figure 4] 1 illustrates a block diagram of an electronic device in a network environment according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The same elements are designated by the same reference numerals even though they are shown in different drawings. In the following description, specific details such as detailed configurations and components are provided merely to facilitate a general understanding of the embodiments of the present disclosure. Therefore, it will be apparent to those skilled in the art that various changes and modifications to the embodiments described herein are possible without departing from the scope of the present disclosure. In addition, descriptions of well-known functions and configurations may be omitted for clarity and conciseness. The terms described below are defined in consideration of the functions of the present disclosure and may differ depending on the user's intentions or practices. Therefore, the definitions of terms should be defined based on the contents of the entire specification.

[0009] The present disclosure may have various modifications and various embodiments, of which embodiments will be described in detail with reference to the accompanying drawings, but it should be understood that the present disclosure is not limited to the embodiments, and includes all modifications, equivalents, and alternatives within the scope of the present disclosure.

[0010] Although terms including ordinal numbers such as first, second, etc. are used to describe various elements, structural elements are not limited by the terms. These terms are used only to distinguish one element from another. For example, a first structural element could be referred to as a second structural element without departing from the scope of this disclosure. Similarly, a second structural element could be referred to as the first structural element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated items.

[0011] The terms used herein are only used to describe various embodiments of the present disclosure and are not intended to limit the present disclosure. The singular includes the plural unless the context clearly dictates otherwise. The terms "comprise" or "have" as used herein should be understood to mean a feature, number, step, operation, structural element, part, or combination thereof. They do not exclude the presence or possibility of one or more other additional features, numbers, steps, operations, structural elements, parts, or combinations thereof.

[0012] Unless otherwise defined, all terms used herein have the same meaning as would be understood by a person of ordinary skill in the art to which this disclosure pertains. Terms, such as those defined in commonly used dictionaries, should be interpreted to have the same meaning in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined herein.

[0013] An electronic device according to an embodiment of the present disclosure may be one of various types of electronic devices, including, for example, a portable communication device (e.g., a smartphone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance, but is not limited to the above.

[0014] The terms used herein are not intended to limit the present disclosure but to encompass various modifications, equivalents, or alternatives to the embodiments. In connection with the description of the accompanying drawings, like reference numerals may be used to refer to similar or related elements. The singular forms of nouns referring to items may include one or more unless the relevant context clearly dictates otherwise. As used herein, terms such as "A or B," "one or more of A and B," "one or more of A or B," "A, B, or C," "one or more of A, B, and C," and "one or more of A, B, or C" may include all possible combinations of the items listed together in one of the terms. As used herein, terms such as "first," "second," "primary," and "secondary" may be used to distinguish a component from other components, but are not intended to limit the components in other respects (e.g., importance or order). When an element (e.g., a first element) is referred to as being "coupled," "coupled," "associated," or "connected" to another element (e.g., a second element), with or without the terms "operably" or "communicatively," this indicates that the element may be coupled to the other element directly (e.g., wired), wirelessly, or via a third element.

[0015] As used herein, the term "module" may include a unit embodied in hardware, software, or firmware, and may be used interchangeably with other terms, such as "logic," "logic block," "portion," and "circuit." A module may be a single integrated component or a minimum unit or portion configured to perform one or more functions. For example, according to one embodiment, a module may be embodied in the form of an application specific integrated circuit (ASIC).

[0016] The present system and method alters a sparse matrix so that sub-blocks are sparsified (or densified). The present system and method can identify row and column permutation vectors that selectively sparsify or densify regions of the sparse matrix. For example, sparsifying one or more regions may densify other regions, and vice versa. The present system and method can sparsify one or more regions and densify one or more other regions, or densify one or more regions and sparse one or more other regions. Given a sparse matrix with a significant proportion of zero-valued elements (e.g., 80% zeros), the sparse matrix can be subdivided into sub-blocks. The present system and method finds row and column permutation vectors that increase or decrease the number of non-zero elements in some subset of the sub-blocks.

[0017] The present system and method generates local sparsity that can be leveraged to improve overall parallel prefix adder (PPA) performance in a network-on-chip (NoC) architecture using a crossbar structure. It is assumed that the matrix associated with each layer can be decomposed into blocks, and the user provides information on how the matrix is ​​decomposed and how the target subsets should be constructed. The matrix can be decomposed into square blocks of uniform size (provided by the user), and the number of blocks to be sparsified is also provided by the user.

[0018] FIG. 1 illustrates a flow diagram 100 for modifying a sparse matrix according to one embodiment of the present disclosure. At 102, the system maps a binary matrix to an undirected graph form and applies a bidirectional graph partitioning algorithm that minimizes edge cuts between partitions. Step 102 may be a preprocessing step that sparsifies one half of the binary matrix and densifies the other half. The system may binarize the binary matrix and perform a min-cut algorithm assuming bidirectional partitioning. The min-cut algorithm may be a hypergraph min-cut algorithm. Those skilled in the art will recognize available min-cut algorithms (e.g., from the METIS product family). The system may store row and column permutation vectors corresponding to the min-cut algorithm. k may be equal to 2.

[0019] At 104, the system recursively applies a greedy algorithm to find a set of row or column permutations that maximizes the transfer of non-zero values ​​from sparse blocks to non-sparse blocks.

[0020] At 104a, given the number N of sub-blocks to be sparsified, the system selects N sub-blocks from the output of step 102. The system may select the sub-blocks with the fewest non-zero elements as the candidate subset. The candidate subset of blocks may not be uniform in size.

[0021] At 104b, the system may assign a ranking to pairwise row permutations according to the change in the number of non-zero elements in the target subset. The system may rank row permutations based on smallest negative change to largest negative change. Permutations in the list that result in a non-negative change or that contain rows in other permutations with more negative change may be removed from the list.

[0022] At 104c, the system may assign a ranking to pairwise column permutations according to the change in the number of non-zero elements in the target subset. Permutations are ranked from mostly negative to mostly positive. Permutations in the list that result in a non-negative change or that contain columns in other permutations with more negative changes may be removed from the list.

[0023] The rankings in 104b and 104c can include additional terms in the expansion of all possible permutations beyond the pairwise permutations in 102 steps. This can include both row and column permutations (e.g., the space for all simultaneous pairwise row and pairwise column permutations). Also, for a matrix with M rows (columns), the number of row (column) permutations is M!, which is intractable for exhaustive search. However, by restricting the search space to only row (column) pairs, the total number of permutations becomes (M-1)(M-2), making it easier to exhaustively search larger matrices with thousands or tens of thousands of rows (columns). Large-scale high-performance computing (HPC) platforms can expand the search space to include possible permutations (e.g., three-way, four-way, or simultaneous row / column sets), which can lead to better results.

[0024] At 104d, the system can construct a permutation vector from the list (row list or column list) by adding non-zero total changes from both the row and column lists, where the total changes are approximately negative changes, and can apply this permutation vector to each of the previous vectors (row vector or column vector) so as to reconstruct a sparsified matrix from the original.

[0025] In 104e, the system may repeat steps 104a through 104d until the two lists generated in step 104d are empty (meaning no further improvement is possible). The target subset of blocks may change with each iteration. The subset with the fewest non-zero counts may be selected. The system may use the actual values ​​of the matrix instead of binarizing it in 102. If the actual values ​​are negative, then magnitude must be used instead of amplitude when calculating the changes in 104b and 104c.

[0026] Alternatively, in steps 104a through 104c, the system can maximize the number of non-zero values ​​in the subset of the target sub-block. The ranking is performed from most positive to least positive, and the permutation that results in a net positive change in the number of non-zero values ​​in the subset of the target block is retained.

[0027] Figure 2 shows a diagram of a sparse matrix according to one embodiment of the present disclosure. Figure 3 shows a graph of block sparsity versus block number for a sparse matrix according to one embodiment of the present disclosure.

[0028] As an illustration of the method described in FIG. 1, in FIG. 2, a random 128×128 matrix 202 is provided with 80% sparsity, and graph 302 shows block sparsity versus number of blocks. After applying steps 104a-104d of FIG. 1, matrix 202 may be sparsified into matrix 204 having 32 blocks. Graph 304 shows block sparsity versus number of blocks in matrix 204. By repeating steps 104a-104d (e.g., step 104e), matrix 204 may be further sparsified into matrix 206 having 16 blocks. Graph 306 shows block sparsity versus number of blocks in matrix 206. By further repeating steps 104a-104d, matrix 206 may be further sparsified into matrix 208 having 8 blocks. Graph 308 shows block sparsity versus number of blocks in matrix 208.

[0029] 4 is a block diagram of an electronic device 401 in a network environment 400 according to one embodiment of the present disclosure. As shown in FIG. 4 , the electronic device 401 in the network environment 400 may communicate with an electronic device 402 via a first network 498 (e.g., a short-range wireless communication network) and with an electronic device 404 or a server 408 via a second network 499 (e.g., a long-range wireless communication network). The electronic device 401 may communicate with the electronic device 404 via the server 408. The electronic device 401 may include a processor 420, a memory 430, an input device 450, an acoustic output device 455, a display device 460, an audio module 470, a sensor module 476, a haptic module 479, a camera module 480, a power management module 488, a battery 489, a communication module 490, a subscriber identification module (SIM) 496, or an antenna module 497. In some embodiments of the present disclosure, at least one of the components (e.g., display device 460 or camera module 480) may be omitted from electronic device 401, or one or more other components may be added to electronic device 401. In some embodiments of the present disclosure, some of the components may be embodied as a single integrated circuit (IC). For example, sensor module 476 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) may be embedded in display device 460 (e.g., a display).

[0030] The processor 420 may, for example, execute software (e.g., program 440) to control at least one other component (e.g., a hardware or software component) of the electronic device 401 coupled to the processor 420 and perform various data processing or calculations. As at least a part of the data processing or calculations, the processor 420 loads instructions or data received from other components (e.g., the sensor module 476 or the communication module 490) into the volatile memory 432, processes the instructions or data stored in the volatile memory 432, and stores the resulting data in the non-volatile memory 434. The processor 420 may include a main processor 421 (e.g., a central processing unit (CPU) or application processor (AP)) and an auxiliary processor 410 (e.g., a graphics processing unit (GPU)), image signal processor (ISP), sensor hub processor, or communication processor (CP)) that can operate together with or independently of the main processor 421. Additionally or alternatively, the auxiliary processor 410 may be configured to consume less power or perform specific functions than the main processor 421. The auxiliary processor 410 may be implemented separately from the main processor 421 or may be implemented as part of it.

[0031] The auxiliary processor 410 may control at least a portion of the functions or states associated with at least one of the components of the electronic device 401 (e.g., the display device 460, the sensor module 476, the communication module 490), either on behalf of the main processor 421 while the main processor 421 is in an inactive (e.g., sleep) state, or together with the main processor 421 while the main processor 421 is in an active state (e.g., running an application). According to one embodiment of the present disclosure, the auxiliary processor 410 (e.g., the ISP or CP) may be embodied as part of another component (e.g., the camera module 480, the communication module 490) functionally associated with the auxiliary processor 410.

[0032] The memory 430 may store a variety of data used by at least one component (e.g., the processor 420 or the sensor module 476) of the electronic device 401. The variety of data may include, for example, input data or output data for software (e.g., the program 440) and its associated commands. The memory 430 may include a volatile memory 432 or a non-volatile memory 434.

[0033] The programs 440 may be stored in the memory 430 as software and may include, for example, an operating system (OS) 442 , middleware 444 or applications 446 .

[0034] The input device 450 may receive instructions or data from outside the electronic device 401 (e.g., a user) for use by other components of the electronic device 401 (e.g., the processor 420). The input device 450 may include, for example, a microphone, a mouse, or a keyboard.

[0035] The audio output device 455 may output an audio signal external to the electronic device 401. The audio output device 455 may include, for example, a speaker or a receiver. The speaker may be used for general purposes such as multimedia playback or recording, and the receiver may be used for receiving incoming phone calls. According to one embodiment of the present disclosure, the receiver may be embodied separately from the speaker or as part of the speaker.

[0036] The display device 460 may provide visual information to an external (e.g., user) of the electronic device 401. The display device 460 may include, for example, a display, a hologram device, or a projector, and control circuitry capable of controlling the corresponding one of the display, the hologram device, or the projector. According to an embodiment of the present disclosure, the display device 460 may include touch circuitry configured to sense a touch or a sensor circuit (e.g., a pressure sensor) configured to measure the strength of a force generated by a touch.

[0037] The audio module 470 may convert sound into electrical signals and vice versa. According to one embodiment of the present disclosure, the audio module 470 acquires sound via the input device 450 and outputs sound via the sound output device 455 or headphones of an external electronic device 402 that is directly (e.g., wired) or wirelessly coupled to the electronic device 401.

[0038] The sensor module 476 may sense an operating state (e.g., power supply or temperature) of the electronic device 401 or an environmental state (e.g., a user's state) outside the electronic device 401 and generate an electrical signal or data value corresponding to the sensed state. The sensor module 476 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface 477 may support one or more specified protocols used for the electronic device 401 to couple directly (e.g., wired) or wirelessly with the external electronic device 402. According to an embodiment of the present disclosure, the interface 477 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital (SD) card interface, or an audio interface.

[0040] The connection terminal 478 may include a connector by which the electronic device 401 may be physically coupled to the external electronic device 402. According to one embodiment of the present disclosure, the connection terminal 478 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] Haptic module 479 may convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that can be perceived by the user via touch or kinesthetic sensations. According to one embodiment of the present disclosure, haptic module 479 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0042] The camera module 480 may capture still or video images. According to one embodiment of the present disclosure, the camera module 480 may include one or more lenses, an image sensor, an ISP, or a flash.

[0043] The power management module 488 may manage the power supplied to the electronic device 401. The power management module 488 may be embodied, for example, as at least part of a power management integrated circuit (PMIC).

[0044] The battery 489 may provide power to at least one component of the electronic device 401. According to one embodiment of the present disclosure, the battery 489 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module 490 may support the establishment of a direct (e.g., wired) or wireless communication channel between the electronic device 401 and an external electronic device (e.g., the electronic device 402, the electronic device 404, or the server 408) and may perform communication via the established communication channel. The communication module 490 may operate independently of the processor 420 (e.g., an AP) and may include one or more communication processors supporting the direct (e.g., wired) or wireless communication. According to one embodiment of the present disclosure, the communication module 490 may include a wireless communication module 492 (e.g., a cellular communication module, a short-range wireless communication module, or a Global Navigation Satellite System (GNSS) communication module) or a wired communication module 494 (e.g., a local area network (LAN) communication module, or a power line communication (PLC) module). The corresponding one of such communication modules may be connected to a first network 498 (e.g., Bluetooth TM The wireless communication module 492 may communicate with the external electronic communication device via a first network 498 or a second network 499 (e.g., a short-range communication network such as Wireless Fidelity (Wi-Fi) Direct or Infrared Data Association (IrDA) standards) or a second network 499 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be embodied as a single component (e.g., a single IC) or as multiple components separated from each other (e.g., multiple ICs). The wireless communication module 492 may identify and authenticate the electronic device 401 in a communication network such as a first network 498 or a second network 499 using subscriber information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the subscriber identification module 496.

[0046] The antenna module 497 may transmit and receive signals or power to and from outside the electronic device 401 (eg, the external electronic device).

[0047] According to one embodiment of the present disclosure, the antenna module 497 may include one or more antennas, from which, for example, the communication module 490 (e.g., the wireless communication module 492) may select at least one antenna suitable for a communication method used in a communication network, such as the first network 498 or the second network 499. Then, signals or power may be transmitted and received between the communication module 490 and the external electronic device via the selected at least one antenna.

[0048] At least some of the above-described components may be interconnected to communicate signals (e.g., commands or data) via a peripheral-to-peripheral communication scheme (e.g., a bus, a general-purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

[0049] According to one embodiment of the present disclosure, commands or data may be transmitted and received between the electronic device 401 and the external electronic device 404 via a server 408 connected to a second network 499. Each of the electronic devices 402 and 404 may be the same type of device as the electronic device 401 or other types of devices. All or part of the operations performed by the electronic device 401 may be performed by one or more of the external electronic devices 402, 404, and 408. For example, if the electronic device 401 must automatically perform a function or service, or if the electronic device 401 must perform a function or service instead of or in addition to performing a function or service in response to a request from a user or another device, the electronic device 401 may request one or more external electronic devices to perform at least part of the function or service. The one or more external electronic devices that receive the request may perform at least part of the requested function or service, or an additional function or service related to the request, and may forward the execution results to the electronic device 401. The electronic device 401 may provide the results, with or without additional processing, as part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

[0050] An embodiment of the present disclosure may be embodied as software (e.g., program 440) including one or more instructions stored in a machine-readable (e.g., electronic device 401) storage medium (e.g., internal memory 436 or external memory 438). For example, a processor in electronic device 401 may call at least one of the one or more commands stored in the storage medium and execute it with or without the use of one or more other components under the control of the processor. Thus, a machine may operate to perform at least one function in accordance with the at least one called instruction. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" indicates that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), but this term does not distinguish between where data is permanently stored on the storage medium and where data is temporarily stored on the storage medium.

[0051] According to one embodiment of the present disclosure, the method of the present disclosure may be provided in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or may be available on an application store (e.g., the Play Store). TM ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily generated or temporarily stored on a machine-readable storage medium, such as the memory of a manufacturer server, an application store server, or an intermediary server.

[0052] According to one embodiment of the present disclosure, each of the components (e.g., modules or programs) described above may comprise a single entity or multiple entities. One or more of the components described above may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be combined into a single component. In this case, the combined component may perform one or more functions of each of the multiple components in the same or similar manner as performed by corresponding components among the multiple components before the combination. Operations performed by a module, program, or other component may be performed sequentially, in parallel, iteratively, or empirically, or one or more operations may be performed in a different order, omitted, or one or more other operations may be added.

[0053] Although specific embodiments of the present disclosure have been described in the detailed description of the present disclosure, the present disclosure can be modified in various forms without departing from the scope of the present disclosure. Therefore, the scope of the present disclosure should not be determined solely by the described embodiments, but by the appended claims and their equivalents. [Explanation of symbols]

[0054] 401 Electronic equipment 420 processor 430 memory

Claims

1. A processor-executed method comprising: Mapping a binary matrix into an undirected graph form, applying a bidirectional graph partitioning algorithm to the mapped binary matrix that minimizes edge cuts between partitions in the mapped binary matrix; recursively applying a greedy algorithm to find a set of row or column permutations that maximizes the transfer of non-zero values ​​from the sparse block to the non-sparse block; The method comprises sparsifying or densifying the binary matrix according to the applied greedy algorithm.

2. The method of claim 1 , wherein the bidirectional graph partitioning algorithm comprises a min-cut algorithm.

3. The method of claim 2 , wherein the min-cut algorithm is a hypergraph min-cut algorithm.

4. The method of claim 1 , wherein applying the greedy algorithm further comprises selecting the N sub-blocks with the fewest number of non-zero elements as the candidate subset.

5. The method of claim 4 , further comprising ranking the permutations of rows in pairs according to changes in the number of non-zero elements in the target subset.

6. The method of claim 5 , wherein the reordering of rows in pairs is ranked based on smallest negative change to largest negative change.

7. The method of claim 5 , further comprising: ranking the permutations of columns in pairs according to the change in the number of non-zero elements within the target subset.

8. The method of claim 7 , wherein the pairwise reordering of columns is ranked based on smallest negative change to largest negative change.

9. 8. The method of claim 7, further comprising adding non-zero total changes from both the row list and the column list to construct a permutation vector.

10. 10. The method of claim 9, further comprising repeating the row permutation ranking, the column permutation ranking, and adding the total changes to construct the permutation vector until the row list and column list are empty.

11. Memory and a processor, The processor maps a binary matrix into an undirected graph form; applying a bidirectional graph partitioning algorithm to the mapped binary matrix that minimizes edge cuts between partitions in the mapped binary matrix; recursively applying a greedy algorithm to find a set of row or column permutations that maximizes the transfer of non-zero values ​​from the sparse block to the non-sparse block; A system configured to perform sparsifying or densifying the binary matrix according to the applied greedy algorithm.

12. The system of claim 11 , wherein the bidirectional graph partitioning algorithm comprises a min-cut algorithm.

13. The system of claim 12 , wherein the min-cut algorithm is a hypergraph min-cut algorithm.

14. The system of claim 11 , wherein applying the greedy algorithm further comprises selecting the N sub-blocks with the fewest number of non-zero elements as the candidate subset.

15. 15. The system of claim 14, wherein the processor further comprises ranking the permutation of rows in pairs according to changes in the number of non-zero elements in the target subset.

16. 16. The system of claim 15, wherein the reordering of rows in pairs is ranked based on smallest negative change to largest negative change.

17. 16. The system of claim 15, wherein the processor further comprises ranking the permutations of columns in pairs according to changes in the number of non-zero elements within the target subset.

18. 18. The system of claim 17, wherein the pairwise reordering of columns is ranked based on smallest negative change to largest negative change.

19. 20. The system of claim 17, wherein the processor further comprises adding non-zero total changes from both the row list and the column list to construct a permutation vector.

20. 20. The system of claim 19, further comprising the processor repeating the row permutation ranking, the column permutation ranking, and adding the total changes to construct the permutation vector until the row list and column list are empty.

Images