Data processing device and method

Abstract

The invention discloses a data processing device and method, electronic equipment and a medium. The device comprises a control module which is configured to respond to a data processing instruction of a central processing unit and generate control information according to the data processing instruction; the operation module is configured to respond to the received control information, read to-be-processed data from the data storage unit according to the control information, perform operation on the to-be-processed data and write an operation result into the data storage unit; and the first queue module is configured to distribute the control information generated by the control module to the operation modules and return the processing result of each operation module on the control information to the control module.

Description

Technical Field

[0001] The present invention relates to electronic digital data processing, and in particular to a data processing device, method, electronic equipment and medium. Background Art

[0002] In modern computing systems, fields such as high-performance computing, artificial intelligence, big data processing, and network function virtualization are developing rapidly. However, the performance bottleneck of the central processing unit (CPU) is unable to meet the growing computing needs. At the same time, with the rise of dedicated hardware and the popularization of heterogeneous computing, it has become a trend to transfer computing-intensive tasks from the CPU to dedicated hardware or external devices to reduce the CPU burden.

[0003] Erasure Coding (EC) is a data protection technology widely used in cloud computing and distributed storage. Compared to traditional replication and RAID (Redundant Arrays of Independent Disks) mechanisms, it can achieve higher data reliability with less data redundancy, offering advantages such as high space utilization, flexible configuration, and high reliability. In actual deployments, while new EC algorithms are emerging, such as LRC (Locally Repairable Codes) and CLAY codes, compared to traditional Reed-Solomon Codes (RS Codes), they can effectively reduce network and disk I / O overhead during data recovery. However, due to the complexity of the encoding scheme, the primary computing challenge in EC deployment lies in the extensive matrix calculations required during the encoding and decoding processes.

[0004] EC computing is a computationally and bandwidth-intensive workload. The existing implementation solutions are generally divided into the following categories:

[0005] -Pure software: Flexible programming, but lower performance, such as the popular EC library Jerasure;

[0006] - Instruction set acceleration: Assembly libraries based on the processor's SIMD instruction set, such as Intel's ISA-L, and the erasure code encoding method based on vector instructions described in patent CN112000509B. Compared with pure software implementations, this method has higher performance, but EC load still takes up a large number of CPU cycles;

[0007] -Hardware acceleration: Some data processing units (DPUs) support a limited number of algorithms, such as only RAID 0 / 5, and lack the flexibility to offload various EC algorithms. Summary of the Invention

[0008] Based on the above situation, the main purpose of the present invention is to provide a data processing device, method, chip, electronic device and medium, which parses and executes data processing instructions through an added control module and configures an operation module to perform matrix calculations, thereby separating the control plane and data plane of data processing, thereby accelerating data calculation processing.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] A first aspect of the present invention provides a data processing device,

[0011] The device comprises:

[0012] a control module configured to respond to a data processing instruction of the central processing unit and generate control information according to the data processing instruction;

[0013] an operation module configured to respond to the received control information, read the data to be processed from the data storage unit according to the control information, perform an operation on the data to be processed, and write the operation result to the data storage unit;

[0014] The first queue module is configured to distribute the control information generated by the control module to the operation module, and return the processing result of each operation module on the control information to the control module.

[0015] Preferably, the control module includes:

[0016] The embedded processor is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

[0017] Preferably, the operation module includes:

[0018] an operation module controller, configured to receive the control information from the first queue module and return a processing result of the control information to the first queue module;

[0019] an access controller configured to perform data transmission with the data storage unit to complete data reading and writing;

[0020] a computing unit configured to concurrently process matrix calculations;

[0021] The second queue module is configured to receive the calculation instructions generated by the operation module controller based on the control information and the data to be processed acquired by the access controller based on the control information, and distribute the calculation instructions and the data to be processed to the computing unit.

[0022] Preferably, the calculation unit includes:

[0023] An input buffer configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner;

[0024] a coding matrix cache configured to include a preset coding matrix;

[0025] a plurality of multiplication modules, each of the multiplication modules being configured to perform a multiplication operation on the original data block and a corresponding coefficient in the encoding matrix according to a processing granularity;

[0026] a storage module configured to store the result of the multiplication operation;

[0027] The output cache is configured to output calculation results of the calculation unit on the plurality of original data blocks to the access controller.

[0028] Preferably, the calculation unit further includes:

[0029] a computing unit controller, configured to distribute the bytes of each of the original data blocks to the corresponding multiplication modules in parallel according to a preset data bit width;

[0030] The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

[0031] Preferably, the multiplication module is a Galois Field Multiplier (GFM).

[0032] Preferably, the plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

[0033] Preferably, the first queue module includes:

[0034] a request queue configured to cache control information generated by the control module;

[0035] The response queue is configured to cache the processing result of the control information.

[0036] A second aspect of the present invention provides a data processing method,

[0037] The method comprises the following steps:

[0038] The control module is configured to respond to data processing instructions from the central processing unit and generate control information according to the data processing instructions;

[0039] The operation module is configured to respond to the received control information, read the data to be processed from the data storage unit according to the control information, perform operation on the data to be processed and write the operation result to the data storage unit;

[0040] The first queue module is configured to distribute the control information generated by the control module to the operation modules, and return the processing result of the control information by each operation module to the control module.

[0041] Preferably, the method further comprises the steps of:

[0042] The embedded processor provided in the control module is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

[0043] Preferably, the operation module includes an operation module controller, an access controller, a calculation unit and a second queue module;

[0044] The operation module controller is configured to receive the control information from the first queue module and return a processing result of the control information to the first queue module;

[0045] The access controller is configured to perform data transmission with the data storage unit to complete data reading and writing;

[0046] The computing unit is configured to be capable of concurrently processing matrix calculations;

[0047] The second queue module is configured to receive the calculation instructions generated by the operation module controller based on the control information and the data to be processed acquired by the access controller based on the control information, and distribute the calculation instructions and the data to be processed to the computing unit.

[0048] Preferably, the computing unit includes an input buffer, a coding matrix buffer, a plurality of multiplication modules, a storage module and an output buffer;

[0049] The input buffer is configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner;

[0050] The encoding matrix cache is configured to include a preset encoding matrix;

[0051] Each of the multiplication modules is configured to perform a multiplication operation on the original data block and the corresponding coefficient in the encoding matrix according to a processing granularity;

[0052] The storage module is configured to store the result of the multiplication operation;

[0053] The output buffer is configured to output calculation results of the calculation unit on the plurality of original data blocks to the access controller.

[0054] Preferably, the calculation unit is further configured to distribute the bytes of each original data block to the corresponding multiplication modules in parallel according to a preset data bit width;

[0055] The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

[0056] Preferably, the multiplication module is a Galois Field Multiplier (GFM).

[0057] Preferably, the plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

[0058] Preferably, the first queue module includes a request queue and a response queue;

[0059] The request queue is configured to cache control information generated by the control module;

[0060] The response queue is configured to cache the processing result of the control information.

[0061] A third aspect of the present invention provides a chip, which includes the device described in the first aspect.

[0062] A fourth aspect of the present invention provides a chip, comprising:

[0063] programmable processing devices; and

[0064] A memory communicatively connected to the programmable processing device; wherein the memory stores a computer program executable by the programmable processing device, and the computer program is executed by the programmable processing device so that the programmable processing device can execute the method described in the second aspect above.

[0065] A fifth aspect of the present invention provides an erasure code encoding system, the system comprising the apparatus described in the first aspect.

[0066] A sixth aspect of the present invention provides a decoding system for erasure codes, the system comprising the apparatus described in the first aspect.

[0067] A seventh aspect of the present invention provides an electronic device comprising: a central processing unit; a data storage unit; and the apparatus as described in the first aspect above.

[0068] An eighth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is configured to be run to implement the method described in the second aspect above.

[0069] Compared with the prior art, the present invention has obvious advantages and beneficial effects, and has at least the following advantages:

[0070] The data processing device of the present invention achieves numerical control separation of data computation through a control module and a computing module. This device balances high flexibility and high performance in data computation. The control module is responsible for control-plane processing to address hardware flexibility, while the computing module is responsible for data-plane processing to address high performance. By configuring multiple computing modules, high hardware concurrency and flexibility are achieved, supporting data streaming, dynamic scalability of hardware operators, and configurable computing parameters. The data processing device of the present invention effectively offloads CPU workloads at a large granularity while achieving high processing speeds.

[0071] The data processing method of the present invention can not only meet the high flexibility requirements of data computing, but also accelerate data computing and achieve high-performance computing by separating the data plane and the control plane of data computing.

[0072] The chip, electronic device, and computer-readable storage medium of the present invention, by adopting the above-mentioned device or method, enable data computing and processing to have both high flexibility and high performance, effectively reduce the CPU burden, and meet the requirements of high-performance data computing.

[0073] The erasure code encoding and decoding systems of the present invention, through an architecture configured with a control module and an operation module, can adapt to traditional RS codes of different sizes, as well as emerging erasure code algorithms such as LRC / CLAY. Based on asynchronous streaming processing, they effectively achieve large-granularity offloading of CPU burden while achieving high-performance computing processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A schematic structural diagram of a preferred embodiment of a data processing device of the present invention;

[0075] Figure 2 Schematic diagram of the structure of a preferred embodiment of the computing module of the present invention;

[0076] Figure 3 A schematic structural diagram of a preferred embodiment of the computing unit of the present invention;

[0077] Figure 4 Schematic diagram of a preferred embodiment of the multiplication module of the present invention;

[0078] Figure 5 It is a structural schematic diagram of another preferred embodiment of the data processing device of the present invention;

[0079] Figure 6 Schematic diagram of data processing of the RS erasure code of the present invention;

[0080] Figure 7 Schematic diagram of data processing of the RS erasure code of the present invention;

[0081] Figure 8 Schematic diagram of data processing of LRC erasure code of the present invention;

[0082] Figure 9 The figure is a flow chart of a preferred embodiment of the data processing method of the present invention.

[0083] Description of reference numerals:

[0084] 1 central processing unit, 2 control module, 3 first queue module, 31 request queue, 32 response queue, 4 operation module, 41 operation module controller, 42 access controller, 43 calculation unit, 431 input cache, 432 encoding matrix cache, 433 multiplication module, 434 register module, 435 output cache, 436 calculation unit controller, 44 second queue module, 5 data storage unit. DETAILED DESCRIPTION

[0085] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of the method, device, electronic device and computer-readable storage medium proposed in accordance with the present invention, together with the accompanying drawings and preferred embodiments, as well as their specific implementation methods, methods, steps, features and effects.

[0086] See also Figure 1 , a data processing device, the device comprising:

[0087] a control module 2 configured to respond to data processing instructions from the central processing unit 1 and generate control information according to the data processing instructions;

[0088] The operation module 4 is configured to respond to the received control information, read the data to be processed from the data storage unit 5 according to the control information, perform operations on the data to be processed and write the operation results to the data storage unit 5;

[0089] The first queue module 3 is configured to distribute the control information generated by the control module 2 to the operation modules 4 , and return the processing result of the control information by each operation module 4 to the control module 2 .

[0090] There may be multiple computing modules to meet the concurrent computing requirements of large amounts of data. The data storage unit may be a computer memory, such as a dynamic random access memory (DRAM).

[0091] Specifically, the control module receives coarse-grained data calculation instructions sent from the CPU, such as EC instructions. The control module parses and executes the EC instructions. When the execution reaches the part that requires a large amount of data access and calculation, the control information generated based on the instruction is sent to a calculation module that is currently in an idle state through the first queue module for processing. The calculation module reads the data to be processed from the DRAM according to the data address in the control information. The calculation module performs corresponding operations on the data to be processed and writes the operation results back to the DRAM. After processing the instruction, the calculation module returns the response status to the control module through the first queue module. After completing the EC protocol processing, the control module returns the final processing status to the CPU.

[0092] Thus, by achieving numerical control separation of data computation through the control module and the computing module, the device can balance high flexibility and high performance in data computation. The control module is responsible for control-plane processing to address hardware flexibility, while the computing module is responsible for data-plane processing to address high performance in data computation. By configuring several computing modules, high hardware concurrency and flexibility are achieved, supporting data streaming processing, dynamic scalability of hardware operators, and configurable computing parameters. The data processing device of the present invention can effectively offload CPU burdens at a large granularity while achieving high-performance processing speeds.

[0093] As an optional embodiment, the control module includes:

[0094] The embedded processor is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

[0095] Specifically, the control module can be implemented based on an embedded processor, which is used to solve flexibility issues, such as processing parameter (such as codec matrix) preparation, granularity segmentation, data splicing, algorithm flow control, etc.

[0096] The embedded processor can asynchronously receive coarse-grained data computation instructions, such as EC instructions, sent from the CPU. The embedded processor parses and executes the EC instructions. When executing the instructions that require large amounts of data access and computation, the embedded processor generates corresponding control information based on the EC instructions. This control information includes algorithm parameters, data access addresses, and processing completion status. The embedded processor distributes this control information to the corresponding computing module via the first queue module, which then performs high-speed data computation.

[0097] Therefore, the embedded processor responds to the data processing instructions from the CPU in an asynchronous communication manner. The asynchronous communication mechanism enables the CPU to continue processing other tasks after sending the data processing instructions without waiting for the data processing device to complete the processing, thereby increasing the parallel processing capability between the CPU and the data processing device, and helping to improve the overall throughput of the system data processing.

[0098] See also Figure 2 As an optional embodiment, the operation module 4 includes:

[0099] The operation module controller 41 is configured to receive the control information from the first queue module 3 and return the processing result of the control information to the first queue module 3;

[0100] an access controller 42 configured to perform data transmission with the data storage unit 5 to complete data reading and writing;

[0101] a computing unit 43 configured to concurrently process matrix calculations;

[0102] The second queue module 44 is configured to receive the calculation instructions generated by the operation module controller 41 based on the control information and the data to be processed acquired by the access controller 42 based on the control information, and distribute the calculation instructions and the data to be processed to the calculation unit 43.

[0103] Specifically, the computing module is the main component of streaming high-speed data processing and is used to solve high-performance problems.

[0104] Among them, the operation module controller receives control information from the first queue module, and generates corresponding calculation instructions after processing the control information (for example, address resolution / translation, instructions for large-block data processing, sub-instructions for dividing into several small-block data processing, etc.), and controls the access controller to obtain the corresponding data to be processed from the data storage unit based on the data address of the control information, and sends the calculation instructions and data to be processed to the second queue module. After completing the processing of the control information, the operation module controller returns the processing result to the first queue module, and the first queue module returns it to the control module. In addition, the operation module controller can also manage the hardware resources within the operation module, such as the allocation and recovery of resources in the second queue module.

[0105] The access controller is used to implement high-speed data access, and may be, for example, a DMA controller to perform data transmission on the DRAM.

[0106] There can be multiple computing units for accelerating matrix calculations to meet the concurrent computing requirements of large amounts of data.

[0107] The second queue module includes a preparation queue of calculation instructions to be processed by the calculation unit and data to be processed.

[0108] Therefore, the high-performance problem of data calculation is solved through the operation module, high concurrency is achieved based on hardware, and multiple computing units can complete streaming high-speed data processing.

[0109] See also Figure 3 As an optional embodiment, the calculation unit 43 includes:

[0110] An input buffer 431 is configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner;

[0111] A coding matrix cache 432 is configured to include a preset coding matrix (Coding Matrix);

[0112] a plurality of multiplication modules 433, each of the multiplication modules 433 being configured to perform a multiplication operation on the original data block and a corresponding coefficient in the encoding matrix according to a processing granularity;

[0113] A storage module 434 is configured to temporarily store the intermediate accumulated result of the multiplication operation;

[0114] The output buffer 435 is configured to output the calculation results of the calculation unit 43 on the plurality of original data blocks to the access controller 42 .

[0115] Specifically, the input buffer receives computation instructions and data to be processed from the second queue module, where the data to be processed includes a number of original data blocks. Data from each original data block can enter the input buffer in a time-sharing manner based on the processing granularity, participating in the computation at different clock cycles. It should be noted that the processing granularity here can be 4 bits, 8 bits, 16 bits, or a larger number of bits.

[0116] The encoding matrix cache is used to temporarily store the encoding matrix required for processing large blocks of data, so as to perform calculation processing on the data of each processing granularity of the original data block.

[0117] There may be multiple multiplication modules for parallel processing of matrix multiplication operations on multiple bytes of the original data block. Each multiplication module may implement multiplication operations based on a single byte (8 bits) or multiple bytes, or 4-bit multiplication operations, depending on the processing granularity, which is not limited here.

[0118] The storage module is used to store the multiplication results of each multiplication module for each original data block based on the processing granularity, accumulate the multiplication results of several original data blocks received by the calculation unit based on the processing granularity, and send the accumulated calculation results to the output buffer.

[0119] Therefore, by configuring several multiplication modules in the computing unit to solve the performance problem of data calculation, high concurrency is achieved based on hardware, and streaming high-speed data processing can be completed. At the same time, the parameters of the multiplication module (such as polynomials, data bit width, etc.) are configurable, which helps to achieve the flexibility of dynamic scaling and dynamic reconfiguration of hardware operators when processing different instructions.

[0120] See also Figure 3 As an optional embodiment, the calculation unit 43 further includes:

[0121] a computing unit controller 436 configured to distribute the bytes of each of the original data blocks to the corresponding multiplication modules in parallel according to a preset data bit width;

[0122] The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

[0123] The computing unit controller is further configured to distribute the encoding matrix preset in the encoding matrix cache to each multiplication module for data calculation.

[0124] Specifically, the access controller sequentially retrieves several byte blocks of a raw data block from the data storage unit based on the system's preset data bit width. The specific size can be determined based on the efficiency of the hardware accessing the data storage unit, for example, 256 bytes or larger. For example, in EC processing, the minimum granularity at which the DMA controller can retrieve data from the DRAM is Z words, where each word can be one byte or multiple bytes. When a raw data block arrives at the computation unit, p words can be read in each computation cycle based on the number of parallel multiplication modules p. After the p multiplication modules complete the parallel matrix calculations, the results are stored in the register module. After Z / p computation cycles, the computation processing of the raw data block is completed.

[0125] As an optional embodiment, the multiplication module is a Galois Field Multiplier (GFM).

[0126] As an optional embodiment, the plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

[0127] For example, see Figure 3In the dotted box, each multiplication module is set to work in w = 8 mode, where w is the processing granularity (its specific value can be set according to the hardware architecture). That is, each multiplication module can process a single byte (8-bit) multiplication operation, so each clock cycle can process 1 Byte of data at line speed. When the operation module needs to work in w = 16 mode, that is, each word is 2 bytes, the 8-bit multiplication module (such as the 8-bit GFM) can be configured to implement 16-bit finite field multiplication operation. An example is shown below:

[0128] When w = 16, assuming that x and y are both 16-bit words, one multiplication requires four 8-bit finite field multiplication operations. Therefore, see Figure 4 , its performance is 1 / 2 line speed.

[0129] x=x H 2 8 +x L

[0130] y=y H 2 8 +y L

[0131] Among them, x H 、y H The high-order bytes of x and y are respectively, L 、y L They are the low-order bytes of x and y respectively.

[0132] but,

[0133] xy=z22 16 +z12 8 +z0

[0134] in,

[0135] z2=x H y H

[0136] z0=x L y L

[0137] z1=x H y L +x L y H

[0138] Similarly, when w=32, four 16b finite field multiplication operations can be performed, and accordingly, its performance is 1 / 4 of the line speed.

[0139] Therefore, by combining and configuring multiple multiplication modules, flexible data bit width processing can be supported, and different data processing requirements can be met without making changes to the hardware settings.

[0140] See also Figure 5 As a further improvement of the above embodiment, the first queue module 3 includes:

[0141] a request queue 31 configured to cache control information generated by the control module;

[0142] The response queue 32 is configured to cache the processing result of the control information.

[0143] Specifically, the first queue module distributes the control information in the request queue to the corresponding operation module according to the busy / idle status of the operation module, and returns the processing result of the operation module on the control information in the response queue to the control module.

[0144] The data processing process of the computing unit is illustrated by the encoding process of the RS erasure code. Assume that k = 6 and m = 3, where k is the number of original data blocks and m is the number of check blocks encoded by EC. The multiplication module is a single 8-bit (i.e., w = 8) path GFM, and the encoding matrix is ​​m*k w-bit elements, which is used for multiplication with the original data block.

[0145] See also Figure 6 The first byte D_i,0 (0≤i≤5) of each original data block participating in EC encoding enters the input buffer of the computation unit in a time-sharing manner and is assigned to one of the GFMs. In different clock cycles, D_i,0 is multiplied by different coding coefficients in the encoding matrix, generating partial products that are accumulated and stored in a register module. The register module in EC encoding is often called a parity memory. For example, in the first clock cycle, the first byte D_0,0 of the original data block D_0 is multiplied by C_0,0, C_1,0, and C_2,0 in the encoding matrix, respectively, and stored in P_0,0, P_1,0, and P_2,0. In the second clock cycle, the first byte D_1,0 of the original data block D_1 is multiplied by C_0,0, C_1,0, and C_2,0 in the encoding matrix and accumulated with P_0,0, P_1,0, and P_2,0. This continues for k = 6 accumulations, ultimately generating the encoded data P_0_0, P_1,0, and P_2,0, which are then sent from the register module to the output buffer. It should be noted that during this streaming process, the original data can be discarded, retaining only the accumulated values.

[0146] See also Figure 7The DMA controller reads the original data block from the DRAM, with a minimum granularity of Z words. It should be noted that the order in which the original data blocks D_0, D_1, ..., D_5 are read is not restricted. Assuming that data D_0 arrives first at the computation unit, the parallel calculation of the partial sum of the parity data block (D_0, i*C_j, 0) can begin. Data D_0 is read from the input buffer. During each computational round, a total of p words (p is the number of parallel multiplication modules) enter the computation unit. After the parallel GF multiplication, a total of m*p parity words are generated and stored in the register module as the temporary calculation result (partial sum). For example, in the first computational round, the dashed box above the original data block in the figure represents the p words of the original data block D_0. The m*p parity words generated after the parallel multiplication are stored in the dashed box above the coded data block. In the second computation cycle, the dashed box below the original data block in the figure contains the p words of the corresponding cycle of the original data block D_0. The m*p parity check data generated after parallel multiplication are stored in the dashed box below the encoded data block. After a total of Z / p computation cycles, all D_0 data is processed.

[0147] After processing data D_0, any of the original data blocks D_1 through D_5 can be processed next (for example, based on the order in which the original data blocks arrived at the computational unit), and the sums accumulated based on the previous partial sums. Before processing the last data stripe, D_last, the register module must store the partial sums of all parity data, meaning the parity memory size is m*Z words. When processing the last original data block, as words enter and are processed in each computation cycle, the final m*p parity words are output in this computation cycle and written back to DRAM via the DMA controller.

[0148] Therefore, when the D_i data processing is completed, that is, after the calculation of the partial sum is completed and saved, the space occupied by the input buffer can be released for reuse by other data, thereby improving hardware utilization.

[0149] The computational unit of the present invention provides flexible processing and dynamic scalability. Within a single computational unit, p parallel GFMs can be deployed to determine the unit's data processing throughput. Multiple words within the same data block (e.g., D_0,i) can be processed simultaneously, generating corresponding check block data (e.g., P_0,i, P_1,i, and P_2,i). Given fixed hardware resources (i.e., the number of GFMs), the computational unit can dynamically adapt its concurrency, p, based on the dynamically changing parameter m carried in different data processing requests, and the data throughput will also change dynamically.

[0150] See also Figure 8 The data processing process of the computing unit is illustrated using the encoding process of LRC erasure code. The control module receives the LRC calculation instruction sent from the CPU, which contains the algorithm parameters A(6,2,2) and the encoding matrix. The algorithm parameters A(6,2,2) indicate that the original data is divided into 6 parts, divided into 2 groups, and a local coding block is generated within each group, followed by two global coding blocks.

[0151] For example, a calculation instruction contains the data to be processed x_i, y_i (i = 0, 1, 2) and the memory address of the data to be written back p_i. After the instruction is parsed, the LRC algorithm process begins to execute, which includes three parts of calculation:

[0152] 1. Calculate p_x based on x_i data and write it back to memory;

[0153] 2. Calculate p_y based on y_i data and write it back to memory;

[0154] 3. Calculate p_0 and p_1 based on x_i and y_i data and write them back to memory.

[0155] Given the algorithm's independent nature, the control module can dispatch these three computations to the three computational modules for parallel, asynchronous processing. Once the control module receives responses from all three computational modules confirming successful computation, the LRC instruction is considered complete. At this point, the control module returns the final offload response to the CPU.

[0156] See also Figure 9 , a data processing method, the method comprising the following steps:

[0157] S100, the control module is configured to respond to a data processing instruction of the central processing unit and generate control information according to the data processing instruction;

[0158] S200, the operation module is configured to respond to the received control information, read the data to be processed from the data storage unit according to the control information, perform operation on the data to be processed and write the operation result to the data storage unit;

[0159] S300, the first queue module is configured to distribute the control information generated by the control module to the operation module, and return the processing result of each operation module on the control information to the control module.

[0160] Through the above steps, the data plane and control plane of data computing are separated, which can not only meet the high flexibility requirements of data computing, but also accelerate data computing and achieve high-performance computing.

[0161] As an optional embodiment, the method further includes the following steps:

[0162] The method further comprises the steps of:

[0163] The embedded processor provided in the control module is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

[0164] Through the above steps, the embedded processor responds to the data processing instructions from the CPU in an asynchronous communication manner. The asynchronous communication mechanism enables the CPU to continue processing other tasks after sending the data processing instructions without waiting for the data processing device to complete processing, thereby increasing the parallel processing capability between the CPU and the data processing device, and helping to improve the overall throughput of the system data processing.

[0165] As an optional embodiment, the operation module includes an operation module controller, an access controller, a calculation unit and a second queue module;

[0166] The operation module controller is configured to receive the control information from the first queue module and return a processing result of the control information to the first queue module;

[0167] The access controller is configured to perform data transmission with the data storage unit to complete data reading and writing;

[0168] The computing unit is configured to be capable of concurrently processing matrix calculations;

[0169] The second queue module is configured to receive the calculation instructions generated by the operation module controller based on the control information and the data to be processed acquired by the access controller based on the control information, and distribute the calculation instructions and the data to be processed to the computing unit.

[0170] Through the above steps, the high-performance problem of data calculation is solved through the operation module, high concurrency is achieved based on hardware, and multiple computing units can complete streaming high-speed data processing.

[0171] As an optional embodiment, the computing unit includes an input buffer, a coding matrix buffer, a plurality of multiplication modules, a storage module and an output buffer;

[0172] The input buffer is configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner;

[0173] The encoding matrix cache is configured to include a preset encoding matrix;

[0174] Each of the multiplication modules is configured to perform a multiplication operation on the original data block and the corresponding coefficient in the encoding matrix according to a processing granularity;

[0175] The storage module is configured to store the result of the multiplication operation;

[0176] The output buffer is configured to output calculation results of the calculation unit on the plurality of original data blocks to the access controller.

[0177] Through the above steps, several multiplication modules are configured in the computing unit to solve the performance problem of data calculation, achieve high concurrency based on hardware, and complete streaming high-speed data processing. At the same time, the parameters of the multiplication module (such as polynomial, data bit width, etc.) are configurable, which helps to achieve the flexibility of dynamic scaling and dynamic reconfiguration of hardware operators when processing different instructions.

[0178] As an optional embodiment, the computing unit is further configured to distribute the bytes of each of the original data blocks to the corresponding multiplication modules in parallel according to a preset data bit width;

[0179] The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

[0180] As an optional embodiment, the multiplication module is a Galois Field Multiplier (GFM).

[0181] As an optional embodiment, the plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

[0182] By combining and configuring multiple multiplication modules through the above steps, flexible data bit width processing can be supported, and different data processing requirements can be met without changing the hardware settings.

[0183] As an optional embodiment, the first queue module includes a request queue and a response queue;

[0184] The request queue is configured to cache control information generated by the control module;

[0185] The response queue is configured to cache the processing result of the control information.

[0186] The present invention further provides a chip, which includes the data processing device described in the above embodiment.

[0187] Therefore, by adopting the above-mentioned device, data calculation and processing can achieve both high flexibility and high performance, effectively reducing the CPU burden and meeting high-performance data calculation requirements.

[0188] The present invention also provides a chip, comprising: a programmable processing device; and a memory communicatively connected to the programmable processing device; wherein the memory stores a computer program executable by the programmable processing device, and the computer program is executed by the programmable processing device so that the programmable processing device can execute the data processing method described in the above embodiment.

[0189] Therefore, by adopting the above method, data calculation and processing can achieve both high flexibility and high performance, effectively reducing the CPU burden and meeting high-performance data calculation requirements.

[0190] The present invention also provides an erasure code encoding system, which includes the data processing device described in the above embodiment.

[0191] Therefore, by configuring an architecture with a control module and an operation module, it can adapt to traditional RS codes of different sizes, as well as emerging erasure code algorithms such as LRC / CLAY. Based on asynchronous streaming processing, it can effectively achieve large-granularity offloading of CPU burden while achieving high-performance computing processing speed.

[0192] The present invention also provides an erasure code decoding system, which includes the data processing device described in the above embodiment.

[0193] Therefore, by configuring an architecture with a control module and an operation module, it can adapt to traditional RS codes of different sizes, as well as emerging erasure code algorithms such as LRC / CLAY. Based on asynchronous streaming processing, it can effectively achieve large-granularity offloading of CPU burden while achieving high-performance computing processing speed.

[0194] The present invention further provides an electronic device comprising: a central processing unit; a data storage unit; and the data processing device described in the above embodiment.

[0195] Therefore, by adopting the above-mentioned device, data calculation and processing can achieve both high flexibility and high performance, effectively reducing the CPU burden and meeting high-performance data calculation requirements.

[0196] The present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is configured to be run to implement the data processing method as described in the above embodiment.

[0197] Therefore, by adopting the above method, data calculation and processing can achieve both high flexibility and high performance, effectively reducing the CPU burden and meeting high-performance data calculation requirements.

[0198] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the present profession can make slight changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A data processing device, characterized in that: The device comprises: a control module configured to respond to a data processing instruction of the central processing unit and generate control information according to the data processing instruction; an operation module configured to respond to the received control information, read the data to be processed from the data storage unit according to the control information, perform an operation on the data to be processed, and write the operation result to the data storage unit; The first queue module is configured to distribute the control information generated by the control module to the operation module, and return the processing result of each operation module on the control information to the control module.

2. The device according to claim 1, wherein The control module includes: The embedded processor is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

3. The device according to claim 1, wherein The operation module includes: an operation module controller, configured to receive the control information from the first queue module and return a processing result of the control information to the first queue module; an access controller configured to perform data transmission with the data storage unit to complete data reading and writing; a computing unit configured to concurrently process matrix calculations; The second queue module is configured to receive the calculation instructions generated by the operation module controller based on the control information and the data to be processed acquired by the access controller based on the control information, and distribute the calculation instructions and the data to be processed to the computing unit.

4. The device according to claim 3, characterized in that The calculation unit includes: An input buffer configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner; a coding matrix cache configured to include a preset coding matrix; a plurality of multiplication modules, each of the multiplication modules being configured to perform a multiplication operation on the original data block and a corresponding coefficient in the encoding matrix according to a processing granularity; a storage module configured to store the result of the multiplication operation; The output cache is configured to output calculation results of the calculation unit on the plurality of original data blocks to the access controller.

5. The device according to claim 4, characterized in that The calculation unit further includes: a computing unit controller, configured to distribute the bytes of each of the original data blocks to the corresponding multiplication modules in parallel according to a preset data bit width; The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

6. The device according to claim 4, characterized in that The multiplication module is a Galois Field Multiplier (GFM).

7. The device according to any one of claims 4 to 6, characterized in that The plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

8. The device according to any one of claims 1 to 7, characterized in that The first queue module includes: a request queue configured to cache control information generated by the control module; The response queue is configured to cache the processing result of the control information.

9. A data processing method, characterized in that: The method comprises the following steps: The control module is configured to respond to data processing instructions from the central processing unit and generate control information according to the data processing instructions; The operation module is configured to respond to the received control information, read the data to be processed from the data storage unit according to the control information, perform operation on the data to be processed and write the operation result to the data storage unit; The first queue module is configured to distribute the control information generated by the control module to the operation modules, and return the processing result of the control information by each operation module to the control module.

10. The method according to claim 9, wherein The method further comprises the steps of: The embedded processor provided in the control module is configured to asynchronously receive the data processing instructions, parse and execute the data processing instructions, and generate corresponding control information according to the calculation requirements in the data processing instructions.

11. The method according to claim 9, wherein The operation module includes an operation module controller, an access controller, a calculation unit and a second queue module; The operation module controller is configured to receive the control information from the first queue module and return a processing result of the control information to the first queue module; The access controller is configured to perform data transmission with the data storage unit to complete data reading and writing; The computing unit is configured to be capable of concurrently processing matrix calculations; The second queue module is configured to receive the calculation instructions generated by the operation module controller based on the control information and the data to be processed acquired by the access controller based on the control information, and distribute the calculation instructions and the data to be processed to the computing unit.

12. The method according to claim 11, wherein The computing unit includes an input buffer, a coding matrix buffer, a plurality of multiplication modules, a storage module and an output buffer; The input buffer is configured to receive a plurality of original data blocks of the data to be processed in a time-sharing manner; The encoding matrix cache is configured to include a preset encoding matrix; Each of the multiplication modules is configured to perform a multiplication operation on the original data block and the corresponding coefficient in the encoding matrix according to a processing granularity; The storage module is configured to store the result of the multiplication operation; The output buffer is configured to output calculation results of the calculation unit on the plurality of original data blocks to the access controller.

13. The method according to claim 12, wherein: The computing unit is further configured to distribute the bytes of each original data block to the corresponding multiplication modules in parallel according to a preset data bit width; The register module is further configured to accumulate the multiplication results of each original data block based on the preset data bit width, and output the accumulated results to the output buffer.

14. The method according to claim 12, wherein: The multiplication module is a Galois Field Multiplier (GFM).

15. The method according to claims 12 to 14, characterized in that The plurality of multiplication modules are further configured to perform multi-byte multiplication operations based on the processing granularity.

16. The method according to any one of claims 9 to 15, characterized in that The first queue module includes a request queue and a response queue; The request queue is configured to cache control information generated by the control module; The response queue is configured to cache the processing result of the control information.

17. A chip, characterized in that: The chip comprises the device according to any one of claims 1 to 8.

18. A chip, characterized in that: The chip comprises: programmable processing devices; and A memory communicatively connected to the programmable processing device; wherein the memory stores a computer program executable by the programmable processing device, the computer program being executed by the programmable processing device so as to enable the programmable processing device to perform the method according to any one of claims 9 to 16.

19. A coding system for erasure codes, characterized in that: The system comprises the apparatus according to any one of claims 1 to 8.

20. A decoding system for erasure codes, characterized in that: The system comprises the apparatus according to any one of claims 1 to 8.

21. An electronic device, characterized in that: include: CPU; Data storage unit; as well as The device according to any one of claims 1 to 8.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is configured to be run to implement the method according to any one of claims 9 to 16.

Images