Data scheduling method based on reconfigurable computing array and neural network accelerator

Abstract

This invention provides a data scheduling method and neural network accelerator based on a reconfigurable computing array. The method includes: receiving a raw data stream and performing valid data detection on the raw data stream to generate a sparse mask sequence corresponding to the raw data stream. Valid data detection is used to remove invalid data from the raw data stream. Invalid data includes invalid padding data and / or zero values ​​or non-important data below a set threshold. Data scheduling is performed on the raw data stream based on the sparse mask sequence to obtain a valid data sequence in the raw data stream. The processing path of the valid data sequence on the reconfigurable computing array is determined based on the sparse mask sequence, and the valid data sequence is processed based on the processing path. This invention can eliminate computational power consumption for invalid data, reduce storage access, achieve zero-pause loading of irregular data streams, reduce inference latency, meet the high frame rate requirements of scenarios such as autonomous driving perception and real-time voice on the edge, and reduce power consumption.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data scheduling method based on a reconfigurable computing array and a neural network accelerator. Background Technology

[0002] When performing neural network inference at the edge, the systolic array is currently the most mainstream hardware architecture, in which the neural network can be a Transformer, a Bidirectional Encoder Representations from Transformers (BERT), or a Graph Neural Network (GNN).

[0003] For input sequences with variable lengths (such as variable-length sentences in natural language processing) or sparse feature maps (such as point cloud data), traditional systolic arrays typically employ a "zero-padding" strategy. This involves padding all input data to a maximum fixed length before feeding it into a regular computational array for inference. Furthermore, traditional systolic arrays use a fixed "nearest neighbor interconnect" method, meaning data can only be passed step-by-step between physically adjacent processing elements (PEs).

[0004] The introduction of the "zero-padding" strategy and the "proximity interconnection" transmission method have led to shortcomings in traditional pulse arrays, such as wasted power due to invalid computation, low efficiency in sparse data processing, and lack of flexibility, which restrict the energy efficiency and real-time performance of edge chips. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a data scheduling method based on a reconfigurable computing array and a neural network accelerator.

[0006] This invention provides a data scheduling method based on a reconfigurable computing array, comprising: Receive the raw data stream and perform valid data detection on the raw data stream to generate a sparse mask sequence corresponding to the raw data stream. The valid data detection is used to remove invalid data in the raw data stream. The invalid data is invalid padding data and / or zero value or non-important data below a set threshold that exists in the data itself. Data scheduling is performed on the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream; Based on the sparse mask sequence, the effective data sequence is determined in the processing path of the reconfigurable computing array, and the effective data sequence is processed based on the processing path.

[0007] According to the present invention, a data scheduling method based on a reconfigurable computing array, wherein determining the processing path of the effective data sequence in the reconfigurable computing array based on the sparse mask sequence and processing the effective data sequence based on the processing path includes: For each reconfigurable processing unit in the reconfigurable computing array, the working mode of the reconfigurable processing unit is determined according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence. When the working mode is intensive computing mode, the computing unit of the reconfigurable processing unit processes the valid data corresponding to the reconfigurable processing unit in the valid data sequence and passes it to the next level of reconfigurable processing unit. When the operating mode is sparse pass-through mode, the effective data sequence is transmitted to the next level reconfigurable processing unit through the bypass pass-through channel of the reconfigurable processing unit.

[0008] According to a data scheduling method based on a reconfigurable computing array provided by the present invention, the step of determining the working mode of the reconfigurable processing unit based on the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence includes: The sparse mask corresponding to the current clock cycle in the sparse mask sequence is determined as the sparse mask corresponding to the reconfigurable processing unit. When the sparse mask corresponding to the reconfigurable processing unit is a first value, the working mode is determined to be the dense computing mode. When the sparse mask corresponding to the reconfigurable processing unit is the second value, the working mode is determined to be the sparse pass-through mode, where the first value is different from the second value.

[0009] According to a data scheduling method based on a reconfigurable computing array provided by the present invention, the step of performing effective data detection on the original data stream and generating a sparse mask sequence corresponding to the original data stream includes: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data fails the valid data detection, the sparse mask corresponding to the original data is determined to be the second value; By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

[0010] According to a data scheduling method based on a reconfigurable computing array provided by the present invention, the step of scheduling the original data stream based on the sparse mask sequence to obtain an effective data sequence in the original data stream includes: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

[0011] The present invention also provides a neural network accelerator, comprising: Interconnected topological sparse sensing units, elastic data schedulers, and reconfigurable computing arrays; The topology sparse sensing unit is used to receive the original data stream, perform effective data detection on the original data stream, and generate a sparse mask sequence corresponding to the original data stream. The effective data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero value or non-important data below a set threshold that exists in the data itself. The elastic data scheduler is used to schedule the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream. The reconfigurable computing array is used to determine the processing path of the effective data sequence in the reconfigurable computing array based on the sparse mask sequence and to process the effective data sequence based on the processing path.

[0012] A neural network accelerator according to the present invention further includes: On-chip compact buffer; The elastic data scheduler and the reconfigurable computing array are connected via the on-chip compact buffer; The on-chip compact buffer is used to receive the valid data sequence determined by the elastic data scheduler and to compactly store the valid data sequence. The reconfigurable computing array is specifically used to obtain the valid data sequence from the on-chip compact buffer.

[0013] According to a neural network accelerator provided by the present invention, the reconfigurable computing array includes a plurality of reconfigurable processing units; The reconfigurable processing unit includes a dynamic routing logic unit, a computing unit, and a bypass transparent transmission channel; The dynamic routing logic unit includes a first input terminal, a second input terminal, and a control terminal; The first input terminal is connected to the output terminal of the computing unit in the previous reconfigurable processing unit, and the second input terminal is connected to the output terminal of the bypass transparent transmission channel in the previous reconfigurable processing unit. The control terminal is used to obtain the effective data sequence from the first input terminal or the second input terminal, and determine the working mode of the reconfigurable processing unit according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence; when the working mode is dense computing mode, the effective data sequence is transmitted to the computing unit; when the working mode is sparse pass-through mode, the effective data sequence is transmitted to the bypass pass-through channel. The processing unit is used to process the valid data corresponding to the reconfigurable processing unit in the valid data sequence and pass it to the next level reconfigurable processing unit. The bypass transparent transmission channel is used to transmit the valid data sequence to the next-level reconfigurable processing unit.

[0014] According to a neural network accelerator provided by the present invention, the topological sparse sensing unit is specifically used for: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data does not contain valid data, the sparse mask corresponding to the original data is determined to be the second value. By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

[0015] According to a neural network accelerator provided by the present invention, the elastic data scheduler is specifically used for: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data scheduling method based on a reconfigurable computing array as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data scheduling method based on a reconfigurable computing array as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data scheduling method based on a reconfigurable computing array as described above.

[0019] This invention provides a data scheduling method and neural network accelerator based on a reconfigurable computing array. The method involves receiving a raw data stream and performing valid data detection on the stream to generate a sparse mask sequence. The valid data detection removes invalid data from the raw data stream, including invalid padding data and / or data with zero values ​​or values ​​below a set threshold. Data scheduling is then performed on the raw data stream based on the sparse mask sequence to obtain a valid data sequence. Finally, the processing path of the valid data sequence on the reconfigurable computing array is determined based on the sparse mask sequence, and the valid data sequence is processed based on this path. This invention eliminates computational power consumption associated with invalid data, significantly reduces storage access, achieves zero-pause loading of irregular data streams, significantly reduces inference latency, and meets the stringent high frame rate requirements of scenarios such as autonomous driving perception and real-time voice input on the edge, while also significantly reducing power consumption. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1This is one of the flowcharts illustrating the data scheduling method based on a reconfigurable computing array provided by the present invention.

[0022] Figure 2 This is the second flowchart of the data scheduling method based on reconfigurable computing array provided by the present invention.

[0023] Figure 3 This is a schematic diagram of the reconfigurable processing unit provided by the present invention.

[0024] Figure 4 This is the third flowchart of the data scheduling method based on reconfigurable computing array provided by the present invention.

[0025] Figure 5 This is one of the structural schematic diagrams of the neural network accelerator provided by the present invention.

[0026] Figure 6 This is the second schematic diagram of the neural network accelerator provided by the present invention.

[0027] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] First, a brief description of the relevant content involved in this invention will be given.

[0030] Existing technologies suffer from the following significant drawbacks, which limit the energy efficiency and real-time performance of edge chips: (1) Invalid computation leads to power waste: The "zero-padding" strategy introduces a large number of invalid zero-value computations. In long-tailed variable-length sequence tasks, the proportion of invalid computation may exceed 50%, resulting in serious dynamic power waste.

[0031] (2) Low efficiency in sparse data processing: In attention mechanisms or sparse graph computation, the effective data is spatially discontinuous. Fixed interconnected pulsating arrays cannot skip invalid nodes in the middle, resulting in a large number of "stalls" in the pipeline and low hardware utilization.

[0032] (3) Lack of flexibility: Existing architectures are difficult to simultaneously balance the high throughput of dense computation (such as Convolutional Neural Networks (CNN)) and the irregular access of sparse computation (such as Transformer), usually requiring two different sets of hardware, which increases chip area.

[0033] To address the aforementioned technical problems, this invention provides a data scheduling method and neural network accelerator based on a reconfigurable computing array. Through the reconfigurable computing array and elastic scheduling mechanism, it achieves "zero-pause" processing of irregular streaming data (data scheduling of raw data streams).

[0034] The following is combined Figures 1 to 7 The present invention describes a data scheduling method and neural network accelerator based on a reconfigurable computing array.

[0035] Figure 1 This is one of the flowcharts illustrating the data scheduling method based on a reconfigurable computing array provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Receive the original data stream and perform valid data detection on the original data stream to generate a sparse mask sequence corresponding to the original data stream. The valid data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero values ​​or non-important data below a set threshold that exist in the data itself.

[0036] The execution subject of the data scheduling method based on reconfigurable computing array provided by the present invention can be a neural network accelerator, an electronic device, or any other object that can execute the data scheduling method based on reconfigurable computing array.

[0037] Specifically, the raw data stream is a data stream composed of raw data of a set length (fixed length). The raw data can be dense data, with a data length of the set length, meaning data that does not require padding; or it can be sparse data after padding (such as data with variable sequence length or sparse feature maps), where the data length before padding is less than the set length, and the data length after padding is equal to the set length. In the case of sparse data after padding, invalid data includes the data used for padding (invalid padding data) and non-essential data with zero values ​​or values ​​below a set threshold. In the case of dense data, invalid data only includes the data used for padding (invalid padding data).

[0038] Valid data detection refers to detecting valid data in the original data stream, i.e., removing invalid data from the original data stream. The sparse mask sequence is formed by arranging the sparse masks corresponding to each piece of original data in the original data stream according to the order of the original data in the original data stream.

[0039] In practical applications, a Topological Sparse Perception Unit (TSPU) can be used to receive the original data stream containing padding, i.e., the original data stream after padding. The TSPU is deployed at the data input front end. Furthermore, the TSPU senses the topological features of the original data stream in real time (valid data detection) and generates a sparse mask sequence indicating validity.

[0040] Step 102: Perform data scheduling on the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream.

[0041] Specifically, a valid data sequence refers to the valid data in the original data stream arranged in the order of their corresponding original data.

[0042] In practical applications, the Elastic Scheduler can be used to schedule the raw data stream. The Elastic Scheduler is connected to the output of the Topology Sparse Aware Unit to obtain the sparse mask sequence and the raw data stream from the Topology Sparse Aware Unit.

[0043] Furthermore, the elastic data scheduler controls the write address pointer based on the sparse mask sequence, performs online data compression, removes invalid data from the original data stream, and retains only the valid data, thereby obtaining a valid data sequence.

[0044] Step 103: Based on the sparse mask sequence, determine the processing path of the effective data sequence in the reconfigurable computing array and process the effective data sequence based on the processing path.

[0045] Specifically, a reconfigurable computing array can contain N There are N reconfigurable processing units. The value of N can be flexibly configured according to the computing power requirements of the application scenario, chip area budget, and on-chip storage bandwidth. For example, in low-power edge scenarios, N can be 16 or 32; in high-performance server scenarios, N can be 64, 128, or larger.

[0046] In practical applications, a reconfigurable computing array can be used to process valid data sequences. The reconfigurable computing array is connected to the output of the topology sparse sensing unit and the output of the elastic data scheduler to obtain sparse mask sequences from the topology sparse sensing unit and valid data sequences from the elastic data scheduler.

[0047] Furthermore, the reconfigurable computing array dynamically reconstructs the computing processing path based on the sparse mask sequence to complete the inference task based on effective data.

[0048] This invention can eliminate computational power consumption for invalid data, significantly reduce storage access, and achieve zero-pause loading of irregular data streams, significantly reducing inference latency. It can meet the stringent requirements for high frame rates in scenarios such as autonomous driving perception and real-time voice on the edge, while significantly reducing power consumption.

[0049] Optionally, before determining the effective data sequence based on the sparse mask sequence and processing the effective data sequence based on the processing path of the reconfigurable computing array, the method further includes: The valid data sequence is compactly stored in an on-chip compact buffer; The step of determining the effective data sequence in the processing path of the reconfigurable computing array based on the sparse mask sequence and processing the effective data sequence based on the processing path includes: The valid data sequence is read from the on-chip compact buffer, and based on the sparse mask sequence, the processing path of the valid data sequence in the reconfigurable computing array is determined, and the valid data sequence is processed based on the processing path.

[0050] Specifically, on-chip compact buffers include, but are not limited to, static random-access memory (SRAM), register file, embedded dynamic random-access memory (DRAM), or first-in-first-out queue (FIFO).

[0051] In practical applications, data storage can be performed using an on-chip compact buffer. The input of the on-chip compact buffer is connected to the output of the elastic data scheduler, and the output of the on-chip compact buffer is connected to the reconfigurable computing array. In other words, the elastic data scheduler and the reconfigurable computing array are connected through the on-chip compact buffer.

[0052] The on-chip compact buffer is used to compactly store consecutive valid data (a valid data sequence) compressed by the scheduler, thereby minimizing the use of on-chip storage resources.

[0053] See Figure 2 , Figure 2 This is the second flowchart illustrating the data scheduling method based on a reconfigurable computing array provided by this invention: First, the original data stream is input to the topology sparse sensing unit. The topology sparse sensing unit performs valid data detection on the original data stream and generates a sparse mask sequence. Then, the topology sparse sensing unit passes the sparse mask sequence and the original data stream to the elastic data scheduler. The elastic data scheduler controls the write pointer based on the sparse mask sequence to remove invalid data (in the original data stream), obtaining a valid data stream (compressed data), and writes the compressed data to the on-chip compact buffer. In addition, the topology sparse sensing unit also synchronously transmits the sparse mask sequence to the reconfigurable computing array. Simultaneously, the reconfigurable computing array obtains a compact data stream (valid data stream) from the on-chip compact buffer and controls the mode switching of the reconfigurable processing units in the reconfigurable computing array based on the sparse mask sequence to complete the inference task of the valid data stream.

[0054] In this embodiment of the invention, the memory wall problem of variable-length sequences can be solved by elastic data scheduling and compact storage: before the data enters the array, an elastic scheduler is introduced to remove the zero-padding part in the original data according to the perceived topological characteristics, and remaps and compresses the non-contiguous valid data (valid data sequence) into the on-chip buffer for compact storage, which fundamentally reduces the number of read and write operations of the on-chip compact buffer and the on-chip bandwidth occupation.

[0055] Optionally, determining the processing path of the effective data sequence in the reconfigurable computing array based on the sparse mask sequence and processing the effective data sequence based on the processing path includes: For each reconfigurable processing unit in the reconfigurable computing array, the working mode of the reconfigurable processing unit is determined according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence. When the working mode is intensive computing mode, the computing unit of the reconfigurable processing unit processes the valid data corresponding to the reconfigurable processing unit in the valid data sequence and passes it to the next level of reconfigurable processing unit. When the operating mode is sparse pass-through mode, the effective data sequence is transmitted to the next level reconfigurable processing unit through the bypass pass-through channel of the reconfigurable processing unit.

[0056] Specifically, a reconfigurable computing array can contain N There are N reconfigurable processing units. Dense mode refers to the mode that processes ordinary input data without padding, while sparse mode refers to the mode that processes data with padding (sparse and variable length).

[0057] See Figure 3 , Figure 3 This is a schematic diagram of the reconfigurable processing unit provided by the present invention: The reconfigurable processing unit (PE) contains at least one dynamic routing logic unit, at least one computing unit, and at least one bypass lane.

[0058] The arithmetic unit includes at least one standard multiplier and at least one accumulator (a combination of an adder and an accumulator register for loop-back accumulation) for performing multiply-accumulate operations based on resident weights. Exemplarily, the arithmetic unit includes a combination of a multiplier or an accumulator, such as a multiply-accumulate (MAC) unit.

[0059] The core component of the dynamic routing logic unit is the input multiplexer (MUX). The input multiplexer includes a first input terminal, a second input terminal, and a control terminal.

[0060] The first input terminal is connected to the output terminal of the computation unit in the previous stage of the reconfigurable processing unit (for regular dense pulsating flow). If the reconfigurable processing unit is a first-stage reconfigurable processing unit (the first reconfigurable processing unit in the reconfigurable computing array), the first input terminal of the reconfigurable processing unit is connected to the output port of the on-chip compact buffer.

[0061] The second input terminal is connected to the output terminal (BypassOut) of the bypass pass-through channel in the previous stage reconfigurable processing unit, and is used to receive fast pass-through data from upstream, forming a chain-like bypass structure. If the reconfigurable processing unit is the last stage reconfigurable processing unit (the last reconfigurable processing unit in the reconfigurable computing array), the second input terminal of this reconfigurable processing unit is the output terminal of the reconfigurable computing array.

[0062] The control terminal is connected to a sparse mask signal, i.e., a sparse mask sequence, that is, synchronized with the data stream.

[0063] The bypass pass-through channel includes a bypass register and a through wire ( Figure 3 (Example not provided).

[0064] The control terminal is used to obtain the valid data sequence transmitted by the previous-level reconfigurable processing unit through the first or second input terminal. When the sparse mask signal indicates that the current reconfigurable processing unit does not need to perform calculations (or the data needs to be transmitted downstream), i.e., the operating mode is sparse pass-through mode, the input valid data sequence does not enter the multiplier, but is directly written to the bypass register and transmitted to the second input terminal of the next-level reconfigurable processing unit through the MUX switching path. When the sparse mask signal indicates that the current reconfigurable processing unit needs to perform calculations, i.e., the operating mode is intensive calculation mode, the data needs to be written to the arithmetic unit for processing through the MUX switching path and then transmitted to the first input terminal of the next-level reconfigurable processing unit.

[0065] In addition to horizontal chain interconnects, the bypass pass-through channel connections between reconfigurable computing arrays can also be configured as diagonal interconnects, tree interconnects, or dynamic routing based on a simple Network-On-Chip (NoC) to support more complex sparse hop patterns.

[0066] The arithmetic unit can also be replaced with a vector multiplier that supports Single Instruction Multiple Data (SIMD), meaning that a reconfigurable processing unit contains multiple sets of multiply-accumulate units that share the same mask control signal.

[0067] In this embodiment of the invention, the working mode is determined by sparse masking, and the MUX switching path is performed based on the working mode for processing or direct skipping. This mechanism enables valid data to skip invalid nodes in the middle, achieving low-latency "wormhole" transmission, rather than the impossible global broadcast.

[0068] Optionally, determining the operating mode of the reconfigurable processing unit based on the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence includes: The sparse mask corresponding to the current clock cycle in the sparse mask sequence is determined as the sparse mask corresponding to the reconfigurable processing unit. When the sparse mask corresponding to the reconfigurable processing unit is a first value, the working mode is determined to be the dense computing mode. When the sparse mask corresponding to the reconfigurable processing unit is the second value, the working mode is determined to be the sparse pass-through mode, where the first value is different from the second value.

[0069] Specifically, the first value is 1, and the second value is 0.

[0070] In practical applications, the reconfigurable computing array synchronously reads the data stream (effective data sequence) in the on-chip compact buffer and the sparse mask sequence output by the topological sparse sensing unit.

[0071] See Figure 3 Each reconfigurable processing unit (PE) can reconstruct the sparse mask (signal) corresponding to the processing unit based on the mask bit corresponding to the current clock cycle, and automatically switch the operating mode. Intensive computing mode (Mask=1): The multiplexer of the reconfigurable processing unit opens the path of the arithmetic unit, and valid data enters the arithmetic unit for calculation. For example, valid data enters the multiplier and performs multiplication and accumulation operations with the resident weights.

[0072] Sparse pass-through mode (Mask=0): The path of the bypass pass-through channel of the multiplexer of the reconfigurable processing unit is bypassed by the arithmetic unit, and the effective data flow is directly written to the bypass register and transmitted to the next level of reconfigurable processing unit.

[0073] In this embodiment of the invention, by using the specific value of the sparse mask, the compact data stream (effective data sequence) can be rapidly flowed to the target computing node (the reconfigurable processing unit that performs the operation) in a "chain jump" manner in the reconfigurable computing array, thus achieving zero-pause processing of sparse data.

[0074] Optionally, the step of performing effective data detection on the original data stream and generating a sparse mask sequence corresponding to the original data stream includes: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data fails the valid data detection, the sparse mask corresponding to the original data is determined to be the second value; By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

[0075] In practical applications, when data is input, the topological sparse sensing unit receives the original data stream including zero-padding. The comparator array inside the topological sparse sensing unit performs zero-value detection or validity judgment on the original data stream based on a preset threshold.

[0076] For example, the zero-value detection process is as follows: if the original data is zero, determine that the original data passes the valid data detection; if the original data is not zero, determine that the original data fails the valid data detection.

[0077] For example, the validity detection process based on a preset threshold is as follows: if the original data is greater than or equal to the preset threshold, the original data is determined to pass the valid data detection; if the original data is less than the preset threshold, the original data is determined to fail the valid data detection.

[0078] The topological sparse sensing unit generates a sparse mask sequence in real time based on the detection results. Mask=1 indicates valid data, and Mask=0 indicates invalid data (such as zero padding or background noise). This allows for the rapid and accurate filtering of valid data from the original data stream.

[0079] Furthermore, sparse mask sequences can be generated based on metadata, meaning the mask signal can be generated without real-time hardware detection. Specifically, the compiler pre-analyzes the original data stream to generate metadata, which is then embedded into the instruction stream or data header. The topology sparsity sensing unit is only responsible for parsing this metadata, thereby generating the sparse mask sequence.

[0080] The sparse mask sequence can also be directly used as the control input of the next level by the output generated by the previous level network layer (dynamic sparsity transfer).

[0081] Optionally, the step of performing data scheduling on the original data stream based on the sparse mask sequence to obtain a valid data sequence in the original data stream includes: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

[0082] Specifically, the first value is 1, and the second value is 0.

[0083] In practical applications, the elastic data scheduler reads the sparse mask (the sparse mask corresponding to the currently reconfigurable processing unit in the sparse mask sequence) and controls the behavior of the write address pointer.

[0084] When the sparse mask Mask=0, the elastic data scheduler freezes the write pointer and discards the current invalid data; when the sparse mask Mask=1, the elastic data scheduler drives the write pointer to increment and writes the valid data into the on-chip compact buffer.

[0085] It should be noted that compression granularity, i.e., data removal and compression, can be based not only on a single data point (scalar), but also on block or vector granularity to accommodate quantized data of different precisions, such as INT4 or INT8.

[0086] In this embodiment of the invention, invalid data in the original data stream is extracted by sparse masking. The non-continuous valid features in the original data stream are remapped and compressed into continuous compact data blocks, thereby realizing the continuity of storage space and eliminating the occupation of storage resources by invalid data.

[0087] The following is combined Figure 4 The present invention will further explain the data scheduling method based on reconfigurable computing arrays provided by the present invention.

[0088] join Figure 4 , Figure 4 This is the third flowchart of the data scheduling method based on reconfigurable computing array provided by the present invention: The data scheduling method based on reconfigurable computing array includes a topology perception phase, an elastic scheduling and compact storage phase, and a dynamic reconfigurable computing phase.

[0089] In the topology sensing phase, the original data stream containing invalid data is first input. The topology sparse sensing unit performs zero-value detection / threshold judgment (validity detection based on a preset threshold) on each original data in the original data stream and generates a sparse mask sequence, which is then input to the elastic data scheduler.

[0090] During the elastic scheduling and compact storage phase, the elastic data scheduler performs data scheduling by determining whether the sparse mask of the current raw data satisfies "Mask==1", i.e., whether the current raw data is valid. If so, the current raw data is compressed, mapped, and written to the on-chip compact buffer for continuous valid storage. If not, the write pointer is frozen, the data (the current raw data) is discarded, and the topology sparse sensing unit processes the next data (the next raw data in the raw data stream after the current raw data).

[0091] During the dynamic reconfiguration computation phase: the reconfigurable computing array obtains a compact data stream (effective data sequence) from the on-chip compact buffer and a sparse mask sequence from the topology sparse sensing unit, i.e., reads the compact data and synchronization mask. Further, for each reconfigurable processing unit in the reconfigurable computing array, it determines whether the sparse mask corresponding to the current reconfigurable processing unit satisfies "Mask==1" to perform mode switching: if yes, multiply-accumulate calculations are performed through the arithmetic unit of the current reconfigurable processing unit; otherwise, the multiply-accumulate calculations are skipped through the bypass pass-through channel of the current reconfigurable processing unit and transmitted to the next level of reconfigurable processing unit. This process continues until the last level of reconfigurable processing unit, at which point the final computation result is output.

[0092] The data scheduling method based on a reconfigurable computing array provided in this invention solves the problems of computational sparsity and pauses through a mask-based hardware dynamic reconfiguration mechanism. Dynamic routing logic is introduced within the reconfigurable processing unit, using a "sparse mask" transmitted synchronously with the data as a control signal to control the multiplexer within the reconfigurable processing unit in real time. When the sparse mask indicates invalid data, the reconfigurable processing unit automatically switches to "bypass pass-through mode," where data does not enter the arithmetic unit but flows directly to the next level through a bypass register. This allows sparse data to flow through the array rapidly, like "passing through a wormhole," achieving zero-pause computation.

[0093] The data scheduling method based on reconfigurable computing arrays provided in this invention also solves the memory wall problem of variable-length sequences based on elastic data scheduling and compact storage: before data enters the array, an elastic scheduler is introduced to remove the zero-padding part in the original data according to the perceived topological characteristics, and remaps and compresses the non-contiguous valid data to the on-chip buffer, which fundamentally reduces the number of read and write operations of the on-chip compact buffer and the on-chip bandwidth occupation.

[0094] In summary, the data scheduling method based on a reconfigurable computing array provided in this invention has the following advantages: Breaking through the memory wall limitation to achieve ultimate energy efficiency: Zero-padding compression is performed before data is written to the buffer, directly reducing the storage footprint and read / write frequency of the on-chip compact buffer; combined with the sparse jump mechanism of the computing array, not only is logical flipping (computational power consumption) for invalid data completely eliminated, but storage access power consumption is also significantly reduced. When processing highly sparse tasks (such as 3D point clouds or large language models), the system-level energy efficiency ratio is significantly improved compared to traditional architectures.

[0095] Enhancing effective bandwidth and achieving "zero-pause" real-time computing: The elastic scheduling mechanism reassembles previously discontinuous effective data into a continuous stream, indirectly improving the utilization rate of effective data bandwidth. Combined with the single-cycle bypass pass-through capability of the dual-mode array, it solves the pipeline bubble problem caused by variable-length sequences, achieving "zero-pause" loading of irregular data streams. This significantly reduces inference latency, meeting the stringent high frame rate requirements of scenarios such as autonomous driving perception and real-time voice processing on the edge.

[0096] The architecture offers versatility and low-cost deployment, enabling multiple modalities to be implemented with a single hardware setup: the architecture does not rely on a specific sparse format, but rather performs adaptive reconstruction based on bitmasks. This means that the same chip design can run dense CNNs at full speed, as well as efficiently accelerate Transformers, GNNs, and point cloud networks, without requiring separate silicon area stacking for sparse computation, greatly reducing the hardware cost and software adaptation complexity of the chip.

[0097] The neural network accelerator provided by the present invention is described below. The neural network accelerator described below can be referred to in correspondence with the data scheduling method based on reconfigurable computing array described above.

[0098] Figure 5 This is one of the structural schematic diagrams of the neural network accelerator provided by the present invention, such as... Figure 5 As shown, the neural network accelerator includes: The interconnected topological sparse sensing unit 501, elastic data scheduler 502, and reconfigurable computing array 503; The topology sparse sensing unit is used to receive the original data stream, perform effective data detection on the original data stream, and generate a sparse mask sequence corresponding to the original data stream. The effective data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero value or non-important data below a set threshold that exists in the data itself. The elastic data scheduler is used to schedule the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream. The reconfigurable computing array is used to determine the processing path of the effective data sequence in the reconfigurable computing array based on the sparse mask sequence and to process the effective data sequence based on the processing path.

[0099] In practical applications, a topology sparse sensing unit can be used to receive the original data stream containing invalid padding, i.e., the original data stream after invalid padding. The topology sparse sensing unit is deployed at the data input front end. Furthermore, the topology sparse sensing unit senses the topological features of the original data stream in real time (valid data detection) and generates a sparse mask sequence indicating validity.

[0100] The raw data stream can be scheduled using an elastic data scheduler. The elastic data scheduler is connected to the output of the topology sparse sensing unit to obtain the sparse mask sequence and the raw data stream from the topology sparse sensing unit. Further, the elastic data scheduler controls the write address pointer based on the sparse mask sequence to perform online data compression, removing invalid data from the raw data stream and retaining only the valid data, thus obtaining a valid data sequence.

[0101] The valid data sequence can be processed using a reconfigurable computing array. The reconfigurable computing array is connected to the output of the topological sparse sensing unit and the output of the elastic data scheduler to obtain a sparse mask sequence from the topological sparse sensing unit and a valid data sequence from the elastic data scheduler. Furthermore, the reconfigurable computing array dynamically reconstructs the computational processing path based on the sparse mask sequence to complete the inference task based on the valid data.

[0102] This invention can eliminate computational power consumption for invalid data, significantly reduce storage access, and achieve zero-pause loading of irregular data streams, significantly reducing inference latency. It can meet the stringent requirements for high frame rates in scenarios such as autonomous driving perception and real-time voice on the edge, while significantly reducing power consumption.

[0103] Optionally, in Figure 5 Based on the above, see Figure 6 , Figure 6 This is a second schematic diagram of the neural network accelerator provided by the present invention. The neural network accelerator further includes: On-chip compact buffer 601; The elastic data scheduler 502 and the reconfigurable computing array 503 are connected via the on-chip compact buffer 601; The on-chip compact buffer 601 is used to receive the valid data sequence determined by the elastic data scheduler and to compactly store the valid data sequence. The reconfigurable computing array 503 is specifically used to obtain the valid data sequence from the on-chip compact buffer.

[0104] In practical applications, data storage can be performed using an on-chip compact buffer. The input of the on-chip compact buffer is connected to the output of the elastic data scheduler, and the output of the on-chip compact buffer is connected to the reconfigurable computing array. In other words, the elastic data scheduler and the reconfigurable computing array are connected through the on-chip compact buffer.

[0105] The on-chip compact buffer is used to compactly store consecutive valid data (a valid data sequence) compressed by the scheduler, thereby minimizing the use of on-chip storage resources.

[0106] See Figure 2First, the original data stream is input to the topology sparse sensing unit (TSU). The TSU performs valid data detection on the original data stream and generates a sparse mask sequence. Then, the TSU passes the sparse mask sequence and the original data stream to the elastic data scheduler. The elastic data scheduler controls the write pointer based on the sparse mask sequence to remove invalid data from the original data stream, obtaining a valid data stream (compressed data), and writes the compressed data to the on-chip compact buffer. Furthermore, the TSU also synchronously transmits the sparse mask sequence to the reconfigurable computing array (RCA). Simultaneously, the RCA obtains the compact data stream (valid data stream) from the on-chip compact buffer and controls the mode switching of the reconfigurable processing units in the RCA based on the sparse mask sequence to complete the inference task on the valid data stream.

[0107] In this embodiment of the invention, the memory wall problem of variable-length sequences can be solved by elastic data scheduling and compact storage: before the data enters the array, an elastic scheduler is introduced to remove the zero-padding part in the original data according to the perceived topological characteristics, and remaps and compresses the non-contiguous valid data (valid data sequence) into the on-chip buffer for compact storage, which fundamentally reduces the number of read and write operations of the on-chip compact buffer and the on-chip bandwidth occupation.

[0108] Optionally, the reconfigurable computing array 503 includes a plurality of reconfigurable processing units; The reconfigurable processing unit includes a dynamic routing logic unit, a computing unit, and a bypass transparent transmission channel; The dynamic routing logic unit includes a first input terminal, a second input terminal, and a control terminal; The first input terminal is connected to the output terminal of the computing unit in the previous reconfigurable processing unit, and the second input terminal is connected to the output terminal of the bypass transparent transmission channel in the previous reconfigurable processing unit. The control terminal is used to obtain the effective data sequence from the first input terminal or the second input terminal, and determine the working mode of the reconfigurable processing unit according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence; when the working mode is dense computing mode, the effective data sequence is transmitted to the computing unit; when the working mode is sparse pass-through mode, the effective data sequence is transmitted to the bypass pass-through channel. The processing unit is used to process the valid data corresponding to the reconfigurable processing unit in the valid data sequence and pass it to the next level reconfigurable processing unit. The bypass transparent transmission channel is used to transmit the valid data sequence to the next-level reconfigurable processing unit.

[0109] See Figure 3The reconfigurable processing unit contains at least one dynamic routing logic unit, at least one computing unit, and at least one bypass transparent transmission channel.

[0110] The arithmetic unit includes at least one standard multiplier and at least one accumulator (a combination of an adder and an accumulator register for loop-back accumulation) for performing multiply-accumulate operations based on resident weights. Exemplarily, the arithmetic unit includes a combination of a multiplier or an accumulator, such as a multiply-accumulate arithmetic unit.

[0111] The core component of the dynamic routing logic unit is the input multiplexer. The input multiplexer includes a first input terminal, a second input terminal, and a control terminal.

[0112] The first input terminal is connected to the output terminal of the computation unit in the previous stage of the reconfigurable processing unit (for regular dense pulsating flow). If the reconfigurable processing unit is a first-stage reconfigurable processing unit (the first reconfigurable processing unit in the reconfigurable computing array), the first input terminal of the reconfigurable processing unit is connected to the output port of the on-chip compact buffer.

[0113] The second input terminal is connected to the output terminal of the bypass pass-through channel in the previous stage reconfigurable processing unit, and is used to receive fast pass-through data from upstream, forming a chain bypass structure. If the reconfigurable processing unit is the last stage reconfigurable processing unit (the last reconfigurable processing unit in the reconfigurable computing array), the second input terminal of the reconfigurable processing unit is the output terminal of the reconfigurable computing array.

[0114] The control terminal is connected to a sparse mask signal synchronized with the data stream, i.e., a sparse mask sequence.

[0115] The bypass pass-through channel includes a bypass register and a through wire.

[0116] The control terminal is used to obtain the valid data sequence transmitted by the previous-level reconfigurable processing unit through the first or second input terminal. When the sparse mask signal indicates that the current reconfigurable processing unit does not need to perform calculations (or the data needs to be transmitted downstream), i.e., the operating mode is sparse pass-through mode, the input valid data sequence does not enter the multiplier, but is directly written to the bypass register and transmitted to the next-level reconfigurable processing unit through the MUX switching path. When the sparse mask signal indicates that the current reconfigurable processing unit needs to perform calculations, i.e., the operating mode is intensive calculation mode, the data needs to be written to the arithmetic unit for processing through the MUX switching path before being transmitted to the next-level reconfigurable processing unit.

[0117] In addition to horizontal chain interconnects, the bypass pass-through channel connections between reconfigurable computing arrays can also be configured as diagonal interconnects, tree interconnects, or dynamic routing based on simple on-chip networks to support more complex sparse hop patterns.

[0118] The arithmetic unit can also be replaced with a vector multiplier that supports single instruction multiple data, meaning that a reconfigurable processing unit contains multiple sets of multiply-accumulate units that share the same mask control signal.

[0119] In this embodiment of the invention, the working mode is determined by sparse masking, and the MUX switching path is performed based on the working mode for processing or direct skipping. This mechanism enables valid data to skip invalid nodes in the middle, achieving low-latency "wormhole" transmission, rather than the impossible global broadcast.

[0120] Optionally, the topological sparse sensing unit 501 is specifically used for: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data does not contain valid data, the sparse mask corresponding to the original data is determined to be the second value. By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

[0121] In practical applications, when data is input, the topological sparse sensing unit receives the original data stream including zero-padding. The comparator array inside the topological sparse sensing unit performs zero-value detection or validity judgment on the original data stream based on a preset threshold.

[0122] The topological sparse sensing unit generates a sparse mask sequence in real time based on the detection results. Mask=1 indicates valid data, and Mask=0 indicates invalid data (such as zero padding or background noise). This allows for the rapid and accurate filtering of valid data from the original data stream.

[0123] Optionally, the elastic data scheduler 502 is specifically used for: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

[0124] In practical applications, the elastic data scheduler reads the sparse mask (the sparse mask corresponding to the currently reconfigurable processing unit in the sparse mask sequence) and controls the behavior of the write address pointer.

[0125] When the sparse mask Mask=0, the elastic data scheduler freezes the write pointer and discards the current invalid data; when the sparse mask Mask=1, the elastic data scheduler drives the write pointer to increment and writes the valid data into the on-chip compact buffer.

[0126] It should be noted that compression granularity, i.e., data removal and compression, can be based not only on a single data point (scalar), but also on block or vector granularity to accommodate quantized data of different precisions, such as INT4 or INT8.

[0127] In this embodiment of the invention, invalid data in the original data stream is extracted by sparse masking. The non-continuous valid features in the original data stream are remapped and compressed into continuous compact data blocks, thereby realizing the continuity of storage space and eliminating the occupation of storage resources by invalid data.

[0128] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can invoke logical instructions in the memory 730 to execute a data scheduling method based on a reconfigurable computing array. This method includes: receiving an original data stream and performing valid data detection on the original data stream to generate a sparse mask sequence corresponding to the original data stream. The valid data detection is used to remove invalid data from the original data stream, where invalid data includes invalid padding data and / or zero values ​​or non-critical data below a set threshold. Based on the sparse mask sequence, data scheduling is performed on the original data stream to obtain a valid data sequence in the original data stream. Based on the sparse mask sequence, the processing path of the valid data sequence in the reconfigurable computing array is determined, and the valid data sequence is processed based on the processing path.

[0129] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data scheduling method based on a reconfigurable computing array provided by the above methods. The method includes: receiving an original data stream and performing valid data detection on the original data stream to generate a sparse mask sequence corresponding to the original data stream. The valid data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero values ​​or non-important data below a set threshold that exist in the data itself; performing data scheduling on the original data stream based on the sparse mask sequence to obtain a valid data sequence in the original data stream; and determining the processing path of the valid data sequence on the reconfigurable computing array based on the sparse mask sequence and processing the valid data sequence based on the processing path.

[0131] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the data scheduling method based on a reconfigurable computing array provided by the above methods. The method includes: receiving an original data stream and performing valid data detection on the original data stream to generate a sparse mask sequence corresponding to the original data stream. The valid data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero values ​​or non-important data below a set threshold that exist in the data itself; performing data scheduling on the original data stream based on the sparse mask sequence to obtain a valid data sequence in the original data stream; and determining the processing path of the valid data sequence in the reconfigurable computing array based on the sparse mask sequence and processing the valid data sequence based on the processing path.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data scheduling method based on a reconfigurable computing array, characterized in that, include: Receive the raw data stream and perform valid data detection on the raw data stream to generate a sparse mask sequence corresponding to the raw data stream. The valid data detection is used to remove invalid data in the raw data stream. The invalid data is invalid padding data and / or zero value or non-important data below a set threshold that exists in the data itself. Based on the sparse mask sequence, the original data stream is scheduled to obtain the effective data sequence in the original data stream; Based on the sparse mask sequence, the processing path of the effective data sequence in the reconfigurable computing array is determined, and the effective data sequence is processed based on the processing path.

2. The data scheduling method based on a reconfigurable computing array according to claim 1, characterized in that, The step of determining the effective data sequence in the processing path of the reconfigurable computing array based on the sparse mask sequence and processing the effective data sequence based on the processing path includes: For each reconfigurable processing unit in the reconfigurable computing array, the working mode of the reconfigurable processing unit is determined according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence. When the working mode is intensive computing mode, the computing unit of the reconfigurable processing unit processes the valid data corresponding to the reconfigurable processing unit in the valid data sequence and passes it to the next level of reconfigurable processing unit. When the operating mode is sparse pass-through mode, the effective data sequence is transmitted to the next level reconfigurable processing unit through the bypass pass-through channel of the reconfigurable processing unit.

3. The data scheduling method based on a reconfigurable computing array according to claim 2, characterized in that, The step of determining the operating mode of the reconfigurable processing unit based on the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence includes: The sparse mask corresponding to the current clock cycle in the sparse mask sequence is determined as the sparse mask corresponding to the reconfigurable processing unit. When the sparse mask corresponding to the reconfigurable processing unit is a first value, the working mode is determined to be the dense computing mode. When the sparse mask corresponding to the reconfigurable processing unit is the second value, the working mode is determined to be the sparse pass-through mode, where the first value is different from the second value.

4. The data scheduling method based on a reconfigurable computing array according to any one of claims 1-3, characterized in that, The step of performing effective data detection on the original data stream and generating a sparse mask sequence corresponding to the original data stream includes: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data fails the valid data detection, the sparse mask corresponding to the original data is determined to be the second value; By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

5. The data scheduling method based on a reconfigurable computing array according to any one of claims 1-3, characterized in that, The step of scheduling the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream includes: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

6. A neural network accelerator, characterized in that, include: Interconnected topological sparse sensing units, elastic data schedulers, and reconfigurable computing arrays; The topology sparse sensing unit is used to receive the original data stream, perform effective data detection on the original data stream, and generate a sparse mask sequence corresponding to the original data stream. The effective data detection is used to remove invalid data in the original data stream. The invalid data is invalid padding data and / or zero value or non-important data below a set threshold that exists in the data itself. The elastic data scheduler is used to schedule the original data stream based on the sparse mask sequence to obtain the effective data sequence in the original data stream. The reconfigurable computing array is used to determine the processing path of the effective data sequence in the reconfigurable computing array based on the sparse mask sequence and to process the effective data sequence based on the processing path.

7. The neural network accelerator according to claim 6, characterized in that, Also includes: On-chip compact buffer; The elastic data scheduler and the reconfigurable computing array are connected via the on-chip compact buffer; The on-chip compact buffer is used to receive the valid data sequence determined by the elastic data scheduler and to compactly store the valid data sequence. The reconfigurable computing array is specifically used to obtain the valid data sequence from the on-chip compact buffer.

8. The neural network accelerator according to claim 6 or 7, characterized in that, The reconfigurable computing array includes multiple reconfigurable processing units; The reconfigurable processing unit includes a dynamic routing logic unit, a computing unit, and a bypass transparent transmission channel; The dynamic routing logic unit includes a first input terminal, a second input terminal, and a control terminal; The first input terminal is connected to the output terminal of the computing unit in the previous reconfigurable processing unit, and the second input terminal is connected to the output terminal of the bypass transparent transmission channel in the previous reconfigurable processing unit. The control terminal is used to obtain the effective data sequence from the first input terminal or the second input terminal, and determine the working mode of the reconfigurable processing unit according to the sparse mask corresponding to the reconfigurable processing unit in the sparse mask sequence; when the working mode is a dense computing mode, the effective data sequence is transmitted to the computing unit. When the working mode is sparse transparent transmission mode, the effective data sequence is transmitted to the bypass transparent transmission channel. The processing unit is used to process the valid data corresponding to the reconfigurable processing unit in the valid data sequence and pass it to the next level reconfigurable processing unit. The bypass transparent transmission channel is used to transmit the valid data sequence to the next-level reconfigurable processing unit.

9. The neural network accelerator according to claim 6 or 7, characterized in that, The topological sparse sensing unit is specifically used for: For each piece of raw data in the raw data stream, valid data detection is performed on the raw data. The valid data detection is either zero value detection or validity detection based on a preset threshold. If the original data passes the valid data detection, the sparse mask corresponding to the original data is determined to be a first value; If the original data does not contain valid data, the sparse mask corresponding to the original data is determined to be the second value. By traversing each of the original data, a sparse mask sequence corresponding to the original data stream is obtained, wherein the sparse mask sequence includes the sparse mask corresponding to each of the original data.

10. The neural network accelerator according to claim 6 or 7, characterized in that, The elastic data scheduler is specifically used for: For each piece of original data in the original data stream, read the sparse mask corresponding to the original data in the sparse mask sequence; If the sparse mask corresponding to the original data is a first value, the original data is determined to be the valid data. If the sparse mask corresponding to the original data is the second value, the original data is determined to be invalid data and discarded. By traversing each of the original data, each of the valid data in the original data stream is obtained, and each of the valid data constitutes the valid data sequence.

Images