A coding device and system

Abstract

The application discloses a kind of encoding and decoding device and system in computer technology field.The application is with parallel and time division multiplexing principle to call each RAID algorithm executor in encoding and decoding device to carry out encoding calculation or decoding calculation, and the encoding and decoding device is suitable for various types of RAID algorithm, high flexibility, widely used, can improve data throughput and hardware resource utilization, to improve the execution efficiency of RAID algorithm.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an encoding / decoding device and system. Background Technology

[0002] Currently, most RAID (Redundant Arrays of Independent Disks) algorithms are executed in software. The efficiency of software execution is limited by the performance of the host CPU (Central Processing Unit). Furthermore, the algorithm execution process may compete for resources with other services on the host, resulting in high CPU overhead and slowing down the execution efficiency of the RAID algorithm.

[0003] Therefore, how to improve the execution efficiency of RAID algorithms is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an encoding / decoding device and system to improve the execution efficiency of RAID algorithms. The specific solution is as follows:

[0005] In a first aspect, the present invention provides an encoding and decoding device, comprising: a first processing core, a first cache connected to the first processing core, a plurality of RAID algorithm executors connected to the first cache, a second cache connected to each RAID algorithm executor, and a second processing core connected to the second cache;

[0006] The first processing core is used to: acquire the target task and the data to be processed in the target task; the target task is used to perform encoding or decoding calculations for at least one RAID algorithm;

[0007] Accordingly, the first processing core is used to: divide the data to be processed according to the time-division multiplexing principle and the type of RAID algorithm implemented by the target task to obtain multiple sub-data;

[0008] The first cache is used to: store the plurality of sub-data;

[0009] Each RAID algorithm executor is used to: read at least one sub-data that matches the RAID algorithm engine in itself from the first cache, and use the RAID algorithm engine to perform encoding or decoding calculations on the read sub-data;

[0010] The second cache is used to store the calculation results output by each RAID algorithm executor;

[0011] The second processing core is used to: generate an address access command based on the calculation results stored in the second cache, and send a command generated message to the host to which the codec device belongs, so that the host can obtain the address access command based on the command generated message, and directly read the calculation results from the second cache according to the address access command; the address access command is used to access a storage address in the second cache.

[0012] Optionally, the first processing core includes:

[0013] The task parsing module is used to: parse the target task to obtain the parsing result; the parsing result includes: the type of RAID algorithm implemented by the target task, the number of encodings calculated, the number of decodings calculated, and the valid data bit information, parity bit information, address list pointer, and parameter list pointer associated with the data to be processed;

[0014] The task processing module is used to divide the data to be processed when it is determined that the parsing result is correct.

[0015] Optionally, the task processing module includes:

[0016] The time-division multiplexing control unit is used to: determine the time-division multiplexing allocation rules based on the parsing results and the operating status of each RAID algorithm executor;

[0017] The address information reading unit is used to: read the storage address of the data to be processed in the host;

[0018] The data parameter reading unit is used to: determine the row and column size of the data matrix to be read according to the time-division multiplexing allocation rule, read the data from the storage address of the data to be processed on the host, and write the read data into the first cache.

[0019] Optionally, the task processing module is further configured to: discard the target task and send a task error message to the host to which the codec device belongs when it is determined that the parsing result is incorrect.

[0020] Optionally, the first cache includes: a plurality of input cache units;

[0021] Accordingly, each input cache unit is used to store at least one sub-data that matches the same RAID algorithm engine.

[0022] Optionally, each RAID algorithm executor includes: at least one RAID algorithm engine;

[0023] Accordingly, any RAID algorithm engine is used to perform encoding or decoding calculations using RAID1, RAID2, RAID3, RAID5, RAID6, or RAIDTP algorithms.

[0024] Optionally, each RAID algorithm executor is also configured to: distribute at least one sub-data read to different RAID algorithm engines within itself according to the time-division multiplexing principle, so that different RAID algorithm engines within the same RAID algorithm executor operate according to the time-division multiplexing principle.

[0025] Optionally, the second cache includes: a plurality of output buffer units;

[0026] Accordingly, each output buffer unit is used to store the calculation results output by the same RAID algorithm executor.

[0027] Optionally, the second processing core includes:

[0028] The command generation module is used to: generate address access commands based on the calculation results stored in the second cache;

[0029] The triggering module is used to send a command generated message to the host to which the codec device belongs, so that the host can obtain the address access command according to the command generated message and directly read the calculation result from the second cache according to the address access command.

[0030] Optionally, the command generation module is specifically used to: when a new calculation result is detected to be written to the second cache, generate a corresponding address access command based on the storage address of the new calculation result in the second cache, and record the address access command in the circular command table.

[0031] Optionally, the triggering module is specifically used to: when an unprocessed address access command is detected in the ring command table, send a command generated message to the host to which the codec device belongs.

[0032] Optionally, the triggering module is specifically used to: detect the valid flag information corresponding to each address access command in the ring command table, and determine whether there are any unprocessed address access commands in the ring command table based on the valid flag information.

[0033] Optionally, the second processing core is further configured to: perform row and column reorganization on the calculation results output by different RAID algorithm executors in the second cache according to the target task, so as to integrate the calculation results.

[0034] Optionally, each RAID algorithm executor includes: an enabling device;

[0035] Accordingly, the enabling device is used to enable or disable the corresponding RAID algorithm executor.

[0036] Optionally, the first processing core is further configured to: send a ready message to the host to which the encoding / decoding device belongs, so that the host sends the target task and the data to be processed to the first processing core according to the ready message.

[0037] Optionally, the first processing core further includes: a configuration register;

[0038] Accordingly, the configuration register is used to: configure the number of encoding operations and the number of decoding operations in the encoding calculation in response to configuration operations from external input, so as to adjust the type of RAID algorithm supported by the encoding / decoding device.

[0039] In a second aspect, the present invention provides an encoding / decoding system, comprising: a host and the encoding / decoding device described in any of the preceding claims.

[0040] Optionally, there are multiple encoding / decoding devices; the host is communicatively connected to each encoding / decoding device.

[0041] Optionally, the host communicates with each codec device via Remote Direct Memory Access (RDMA) and / or Direct Memory Access (DMA).

[0042] Optionally, the host is connected to each codec device via an expansion chip.

[0043] As can be seen from the above scheme, the present invention provides an encoding and decoding device, including: a first processing core, a first cache connected to the first processing core, a plurality of RAID algorithm executors connected to the first cache, a second cache connected to each RAID algorithm executor, and a second processing core connected to the second cache; wherein, the first processing core is used to: acquire a target task and the data to be processed of the target task; the target task is used to implement encoding or decoding calculations of at least one RAID algorithm; correspondingly, the first processing core is used to: divide the data to be processed according to the time-division multiplexing principle and the type of RAID algorithm implemented by the target task to obtain a plurality of sub-data; the first cache is used to: store the plurality of sub-data. Data; each RAID algorithm executor is used to: read at least one sub-data that matches its own RAID algorithm engine from the first cache, and use the RAID algorithm engine to perform encoding or decoding calculations on the read sub-data; the second cache is used to: store the calculation results output by each RAID algorithm executor; the second processing core is used to: generate an address access command based on the calculation results stored in the second cache, and send a command generated message to the host to which the encoding / decoding device belongs, so that the host can obtain the address access command based on the command generated message, and directly read the calculation results from the second cache according to the address access command; the address access command is used to access a segment of storage address in the second cache.

[0044] As can be seen, the present invention provides a hardware device specifically for RAID encoding and decoding, which has the following advantages: it can call the RAID algorithm executor to perform encoding or decoding calculations in parallel and time-division multiplexing, and is applicable to various types of RAID algorithms. It is highly flexible, widely used, and can improve data throughput and hardware resource utilization, thereby improving the execution efficiency of RAID algorithms.

[0045] Correspondingly, the encoding and decoding system provided by the present invention also has the above-mentioned technical effects. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of an encoding / decoding device disclosed in this invention;

[0048] Figure 2This is a schematic diagram of another encoding / decoding device disclosed in this invention;

[0049] Figure 3 This is a schematic diagram of a time-division multiplexing process disclosed in this invention;

[0050] Figure 4 This is a schematic diagram illustrating the communication between a command processor, an encoding / decoding device, and a host disclosed in this invention.

[0051] Figure 5 This invention discloses a circular command representation.

[0052] Figure 6 This is a schematic diagram of an encoding / decoding system disclosed in this invention;

[0053] Figure 7 This is a structural diagram of the encoding / decoding device 1 provided by the present invention;

[0054] Figure 8 This is a structural diagram of the encoding / decoding device 2 provided by the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Currently, most RAID algorithms are executed in software. Software execution efficiency is limited by the host CPU performance, and the algorithm execution process may also compete for resources with other services on the host, leading to high CPU overhead and slowing down the RAID algorithm's execution efficiency. Therefore, this invention provides an encoding and decoding scheme that is applicable to various types of RAID algorithms, offering high flexibility and wide applicability. It can improve data throughput and hardware resource utilization, thereby enhancing the execution efficiency of RAID algorithms.

[0057] See Figure 1 As shown in the figure, an embodiment of the present invention discloses an encoding and decoding device, including: a first processing core, a first cache connected to the first processing core, a plurality of RAID algorithm executors connected to the first cache, a second cache connected to each RAID algorithm executor, and a second processing core connected to the second cache.

[0058] The first processing core is used to: acquire the target task and the data to be processed for the target task; the target task is used to implement the encoding or decoding calculation of at least one RAID algorithm.

[0059] Accordingly, the first processing core is used to divide the data to be processed into multiple sub-data according to the time-division multiplexing principle and the type of RAID algorithm implemented by the target task.

[0060] The first cache is used to store multiple sub-data items.

[0061] Each RAID algorithm executor is used to: read at least one sub-data that matches the RAID algorithm engine in itself from the first cache, and use the RAID algorithm engine to perform encoding or decoding calculations on the read sub-data.

[0062] The second cache is used to store the calculation results output by each RAID algorithm executor.

[0063] The second processing core is used to: generate an address access command based on the calculation results stored in the second cache, and send a command generated message to the host to which the encoding / decoding device belongs, so that the host can obtain the address access command based on the command generated message, and directly read the calculation results from the second cache according to the address access command; the address access command is used to access a storage address in the second cache.

[0064] In this embodiment, each RAID algorithm executor can run in parallel or alternately according to the time-division multiplexing principle. For example, when there are multiple target tasks, each RAID algorithm executor can process the input data of different target tasks. At the same time, different RAID algorithm engines in the same RAID algorithm executor can also run in parallel or alternately according to the time-division multiplexing principle.

[0065] In one implementation, the first processing core includes: a task parsing module for: parsing the target task to obtain parsing results; the parsing results include: the type of RAID algorithm implemented by the target task, the number of encoding operations, the number of decoding operations, and the valid data bit information, parity bit information, address list pointer, and parameter list pointer associated with the data to be processed; and a task processing module for: dividing the data to be processed when the parsing results are determined to be correct.

[0066] In one implementation, the task processing module includes: a time-division multiplexing control unit for determining time-division multiplexing allocation rules based on the parsing results and the operating status of each RAID algorithm executor; an address information reading unit for reading the storage address of the data to be processed on the host; and a data parameter reading unit for determining the row and column size of the data matrix to be read according to the time-division multiplexing allocation rules, reading the data from the storage address of the data to be processed on the host, and writing the read data into a first cache.

[0067] In one implementation, the task processing module is further configured to: discard the target task and send a task error message to the host to which the codec device belongs when it is determined that the parsing result is incorrect.

[0068] In one implementation, the first cache includes: a plurality of input cache units; correspondingly, each input cache unit is used to: store at least one sub-data that matches the same RAID algorithm engine.

[0069] In one implementation, each RAID algorithm executor includes: at least one RAID algorithm engine; correspondingly, any RAID algorithm engine is used to: perform encoding or decoding calculations using RAID1, RAID2, RAID3, RAID5, RAID6, or RAIDTP algorithms.

[0070] RAID stands for Redundant Disk Array. A disk array combines multiple independent disks to create a large-capacity disk group. Using RAID storage technology can increase storage capacity, improve system I / O response capabilities, and enhance data reliability through distributed data storage, parallel access, and information redundancy. RAID primarily utilizes data striping, mirroring, and data parity techniques to achieve high performance, reliability, high fault tolerance, and scalability. Many RAID algorithms have been developed based on the strategies and architectures that employ or combine these technologies, with RAID 5, RAID 6, and RAID TP being the most widely used. TP (Triple Parity) indicates triple parity checking.

[0071] In RAID, data is distributed across multiple disks in blocks. RAID 5 stores data and its corresponding parity information across the disks that make up the RAID 5 array. When one disk in a RAID 5 array fails, the remaining data and the corresponding parity information are used to recover the damaged data. RAID 6 adds a parity disk to RAID 5, meaning it can recover data lost from two disks. RAID TP adds another parity disk to RAID 6, meaning it can recover data lost from three disks.

[0072] In one example, the data distribution across disks for a RAID5 stripe is shown in Table 1. In Table 1, D1, D2...Dn represent n data items from different disks included in a stripe, and P represents the parity data included in the stripe, satisfying the following operational relationship: K1*D1⊕K2*D2⊕K3*D3⊕…⊕Kn*Dn⊕P=0; * represents Galois multiplication, K1, K2, K3...Kn are Galois multiplication parameters; ⊕ is the XOR operator.

[0073] Table 1

[0074]

[0075] Accordingly, the encoding formula for RAID5 is: The decoding formula is: It does not include the Dx item. If any data in a strip of RAID 5 is lost, it can be recovered from the other data.

[0076] In one example, the data distribution across disks for a RAID6 stripe is shown in Table 2. In Table 2, D1, D2…Dn represent n data items from different disks within a stripe, and P and Q represent parity data within the stripe, satisfying the following operational relationship: Based on the principle of solving a system of two linear equations, RAID6 can recover data lost from a maximum of two disks.

[0077] Table 2

[0078]

[0079] In one example, the data distribution across disks for a stripe corresponding to RAIDTP is shown in Table 3. In Table 3, D1, D2…Dn represent n data items from different disks within a stripe, and P, Q, and R represent parity data included in the stripe, satisfying the following operational relationships: Based on the principle of solving a system of three linear equations, RAIDTP can recover data lost from a maximum of three disks.

[0080] Table 3

[0081]

[0082] It is evident that different RAID algorithms can calculate different amounts of parity data, therefore different RAID algorithms require different RAID algorithm engines for encoding or decoding.

[0083] Since there are three modes for writing to disk:

[0084] Mode 1: Full stripe write. New data is written to fill the entire stripe. In this mode, the checksum information for the data to be written can be calculated beforehand, and the entire stripe can be written directly to disk. Since there is no old data, the checksum can be calculated directly.

[0085] Mode 2: Read-Modify-Write. Read the old data (D) to be modified and the original check data, and calculate the check value together with the newly written data (D′).

[0086] Mode 3: Reconstructive Write. Read the data (D) that does not need to be modified from this stripe, calculate the checksum together with the data (D′) to be newly written, and write it to disk.

[0087] Different write modes require different engine circuits. Modes two and three also require PPL calculation.

[0088] Assuming a RAID 5 stripe is constructed as shown in Table 4, where D1 to D7 are data blocks, if data updates occur on D2 and D3, updating them to D2' and D3' respectively, it's necessary to consider the possibility of disk failure (disk removal or damage) of unmodified data (D1, D4, D5, D6, and D7 in this example) during the writing of D2' and D3'. RAID 5 needs to ensure that any disk containing unmodified data can be recovered after this failure. To protect unmodified data, PPL (Checksum of Unmodified Data) is introduced. In this example,

[0089] Table 4

[0090] D1 D2 D3 D4 D5 D6 D7 P

[0091] The following describes the calculation methods for checksums P′ and PPL under three modes: In mode one, all data on all disks will be modified. The checksum is calculated using all the new data, resulting in: D1′~Dn′ are new data. Mode 2 is used for scenarios where relatively little disk data is modified (less than half of the stripe); for example, in a stripe of n data D1~Dn, only D2 and D3 are modified, then the old data in that stripe satisfies: According to the principle of XOR operation, we can obtain: Therefore, after the old data D2 and D3 are replaced by the new data D2′ and D3′, the new formula for calculating the check P′ is: In Mode 2, the PPL calculation formula is as follows: Mode 3 is suitable for scenarios where a large amount of disk data is modified (more than half of the stripe); for example, if D2 and D3 are modified in the n data points D1 to Dn of a stripe, then the new parity P′ calculation formula is: Where P′ is the newly calculated verification, which can be omitted, the PPL calculation formula under Mode 3 is as follows:

[0092] It should be noted that the RAID algorithm engine provided in this embodiment is used to calculate the various calculation formulas in the aforementioned examples. Therefore, it is evident that RAID algorithm engines need to be diverse to satisfy the encoding and decoding requirements of different RAID algorithms in different modes.

[0093] In one implementation, each RAID algorithm executor is further configured to: distribute at least one sub-data read to different RAID algorithm engines within itself according to the time-division multiplexing principle, so that different RAID algorithm engines within the same RAID algorithm executor operate according to the time-division multiplexing principle.

[0094] In one implementation, the second cache includes: a plurality of output cache units; correspondingly, each output cache unit is used to: store the calculation results output by the same RAID algorithm executor.

[0095] In one implementation, the second processing core includes: a command generation module for generating an address access command based on the calculation results stored in the second cache; and a triggering module for sending a command generation message to the host to which the encoding / decoding device belongs, so that the host obtains the address access command based on the command generation message and directly reads the calculation result from the second cache according to the address access command. Specifically, the command generation module is used to: generate a corresponding address access command based on the storage address of the new calculation result in the second cache when a new calculation result is detected written to the second cache, and record the address access command in a circular command table. Specifically, the triggering module is used to: send a command generation message to the host to which the encoding / decoding device belongs when an unprocessed address access command is detected in the circular command table.

[0096] In one implementation, the triggering module is specifically used to: detect the valid flag information corresponding to each address access command in the ring command table, and determine whether there are any unprocessed address access commands in the ring command table based on the valid flag information.

[0097] In one implementation, the second processing core is further configured to: reorganize the calculation results output by different RAID algorithm executors in the second cache according to the target task, so as to integrate the calculation results.

[0098] In one implementation, each RAID algorithm executor includes an enabling device; correspondingly, the enabling device is used to enable or disable the respective RAID algorithm executor.

[0099] In one implementation, the first processing core is further configured to: send a ready message to the host to which the encoding / decoding device belongs, so that the host sends the target task and the data to be processed to the first processing core according to the ready message.

[0100] In one implementation, the first processing core further includes a configuration register; correspondingly, the configuration register is used to configure the number of encoding operations and the number of decoding operations in response to configuration operations from external input, so as to adjust the type of RAID algorithm supported by the encoding / decoding device.

[0101] The encoding / decoding device provided in this embodiment is capable of task allocation and is a high-performance, high-concurrency RAID hardware system. Therefore, this embodiment provides a dedicated hardware device for RAID encoding and decoding. This device can invoke the RAID algorithm executor for encoding or decoding calculations using parallel and time-division multiplexing principles. It is applicable to various types of RAID algorithms, offering high flexibility and wide applicability. It can improve data throughput and hardware resource utilization, and enhance the execution efficiency of RAID algorithms.

[0102] Please see Figure 2 This invention provides another encoding / decoding device that enables time-division multiplexing of hardware from multiple dimensions and supports multi-channel concurrency. Simultaneously, time-division multiplexing is supported both between multiple channels and within a single channel. (Channel refers to...) Figure 2 A data computing unit (i.e.) Figure 1 (RAID algorithm executor in the system).

[0103] against Figure 2 The device shown can be configured with two modes: Mode 1 is the encoding process (which can also calculate PPL), and Mode 2 is the decoding process. The encoding process is used to calculate parity data or PPL. Users can configure: the number of data blocks (Dx), the number of parity blocks (Dy), and a flag indicating the location of the parity block on the disk. For example, with 16 disks, when the flag is 16 bits, it can represent a maximum of 16 disks. Users can freely configure the position of the parity bit on the disk according to their needs; in practice, the number of disks can be arbitrarily expanded. The decoding process is used to recover missing data. Users can configure: the number of data blocks (Dx), the total number of parity blocks (Dy), and a flag indicating the location of the missing data block on the disk. Again, with 16 disks, when the flag is 16 bits, it can represent a maximum of 16 disks. Users can freely configure the location of the missing data block on the disk according to their needs. The maximum number of missing data blocks is 1 for RAID 5, 2 for RAID 6, and 3 for RAID TP.

[0104] Data reading and address reading are accomplished through external auxiliary hardware, such as DMA.

[0105] if Figure 2The device shown is configured with 6 inputs and 6 outputs, and includes 8 data processing units. During operation, these 8 data processing units are time-division multiplexed, and each processing engine within a single data processing unit also time-division multiplexes to accelerate the data encoding and decoding process. It also supports simultaneous encoding or decoding of multiple channels.

[0106] The encoding / decoding device provided in this embodiment can simultaneously calculate checksums, encoded values, or PPL values ​​on multiple channels. To maximize system execution speed for different tasks, time-division multiplexing between channels and within channels is used to accelerate system performance at both the large-granularity and small-granularity levels of data. Different tasks do not need to wait for each other, maximizing the utilization of idle time between tasks.

[0107] Among these methods, time-division multiplexing between channels with large granularity of data processing is employed. For example, when task A is being executed, some idle channels can be utilized to execute task B through time-division multiplexing. This method can significantly improve the system's execution speed in complex task scenarios. By dynamically detecting idle channel resources and task resource requirements in real time at a large granularity, and rationally allocating tasks to idle channels for processing, this approach offers significantly greater flexibility and performance improvements compared to the previous task-channel binding method.

[0108] Fine-grained time-division multiplexing within a single channel is an example of this. When the workload is large and all channels are running at full capacity, different computing engines within a single channel can operate using a time-division multiplexing mechanism to maximize the utilization of idle time within that channel and improve the overall system performance. This fine-grained reuse of computing engines within a single channel avoids wasted time intervals. For instance, when processing a two-dimensional matrix and a one-dimensional matrix, processing the two-dimensional matrix first and then the one-dimensional matrix requires waiting. However, by using time-division multiplexing, the two-dimensional matrix is ​​split into multiple one-dimensional matrices, and the split one-dimensional matrices are processed in parallel with the original one-dimensional matrix, thus minimizing the overall time of both tasks and improving system efficiency.

[0109] like Figure 2 As shown, the device's hardware structure is divided into a control plane and a data plane. The control plane mainly handles the scheduling of system addresses, control commands, matrix allocation, time-division multiplexing granularity control, and command generation. The data plane mainly processes data input, output, calculation, and integration.

[0110] The task parsing module parses the CB command information and task information dispatched by the NVME protocol, obtaining information such as task type, number of data to be processed, number of data to be decoded, number of parity bits, address list pointer, parameter list pointer address, and valid data bits. It parses the NVME protocol and determines the correctness of the information; if incorrect, the task is discarded and an error report is submitted. If correct, the task processing module organizes the parsed information to obtain the instructions for each module, configuring the corresponding instructions for each module into the following units: data parameter matrix reading unit (data parameter reading unit), output data controller, time-division multiplexing matrix control unit (i.e., time-division multiplexing control unit), input data dispatcher, data address information reading unit (i.e., address information reading unit), and processing result summarization unit.

[0111] The task processing module includes: a data parameter matrix reading unit, an output data controller, a time-division multiplexing matrix control unit, an input data dispatcher, a data address information reading unit, and a processing result aggregation unit. The processing result aggregation unit can respond to the task processing progress externally, while the task calculation results are sent to the outside via a command processor.

[0112] Data parameter matrix reading unit: Based on the mode issued by the system, it reads the corresponding matrix parameters and then adjusts the row and column sizes of the scheduling matrix variably according to time-division multiplexing control. The data parameter matrix reading unit and the data address information reading unit, based on mode requirements and the amount of data to be calculated, read the parameter matrix and data address list into their internal address buffer and parameter matrix buffer, respectively, for consumption by lower-level modules.

[0113] The time-division multiplexing matrix control unit generates an encoding or decoding matrix for a single processing operation based on data transmission throughput requirements. This matrix can be flexibly configured. It calculates time-division multiplexing task allocation rules based on system task type (encoding, decoding, number of encoding / decoding operations, number of input data, and channel idle status, etc.). For example, it determines how much data each task should process, and rationally allocates corresponding channels for encoding based on channel idle time and the number of input data for different types of tasks. After receiving the configuration, the input data dispatcher determines whether it is an encoding or decoding task, further assesses the usage of each data computation channel, and schedules the corresponding encoding / decoding matrix. Data is read from external sources into a multi-channel data input buffer (i.e., multiple input buffer units). The output data controller, according to the number of encoding / decoding operations, sends the address order and rules of the solved data to the data plane for use.

[0114] Input Data Distributor: Based on instructions from the task processing module, it reads external data into the data input buffer. According to the instructions from the time-division multiplexing matrix control unit and the channel execution information from data feedback, it generates time-division multiplexing control instructions and matrix distribution rules. Based on the number of matrix rows and columns issued by the time-division multiplexing matrix control unit, the task execution status of lower-level modules (how many data buffers are occupied, how many data computation units are used, and how many are idle, etc.), and the task type from the task parsing module, it reads data and sends it to the matching data input buffer. For example: For the first 3+1 encoding task, if both the data computation unit and the data input buffer are idle, the data is stored in data buffers 1, 2, and 3, and the time-division multiplexing input data distributor distributes the data to data computation units 1, 2, and 3 for processing. For the second 3+1 decoding task, the data is sent to data input buffers 4, 5, and 6, and the time-division multiplexing input data distribution control unit sends the data to data computation units 4, 5, and 6 for processing. The time-division multiplexing input data distribution module distributes data arbitrarily to data processing units based on their availability, thereby improving system efficiency. Using this time-division multiplexing mechanism, data processing pipelines within a single RAID algorithm task achieve zero-wait operation. Data processing pipelines between tasks also achieve zero-wait operation.

[0115] Output data dispatcher: Based on the instructions from the task processing module, outputs the processed data and generates time-division multiplexing integrated output rule instructions.

[0116] Data input cache (i.e., input cache unit): Data caching function to improve system efficiency.

[0117] Data output buffer (i.e., output buffer unit): Data caching function to improve system efficiency.

[0118] The time-division multiplexing input data distribution control unit can be built into the first processing core: according to the instructions and task mode of the upper-level module, it sends data to the lower-level data computing unit. It also adjusts the direction of data distribution in real time based on the execution status of the computing unit to ensure that as much data as possible is processed per unit time.

[0119] The data computation unit performs Galois field multiplication and XOR operations on the input matrix to achieve simultaneous computation of the solution data and PPL. In one example, the data computation unit has two computation engines, each capable of performing mixed computations of the solution data and PPL data. Therefore, when task 1 is executed by computation engine 1, and the other computation engine is idle, the computation unit will automatically identify and assign the new task to the idle computation engine, achieving time-division multiplexing of different computation engines and thus improving the overall system execution efficiency. The data computation unit performs corresponding matrix calculations based on the configured data volume, matrix size, granularity, and task type. During computation, it dynamically adjusts the execution process of different tasks, using time-division multiplexing to save task execution time within the computation unit and enabling simultaneous encoding and decoding of any RAID algorithm task.

[0120] The output data integration unit can be built into the second processing core. According to the distribution order, number of verifications, number of decoders, block size information and time division multiplexing rules, the generated data is reordered and output to the time division multiplexing output data allocation unit in the second processing core in sequence.

[0121] The time-division multiplexing output data allocation unit distributes the data from the upper level into the data output buffer according to the data size and the number of data to be checked or recovered, and notifies the host that the task is complete. It can be seen that this device dynamically schedules its own hardware resources to achieve three-dimensional hierarchical time-division multiplexing, supporting simultaneous bidirectional encoding and decoding processing. Based on instructions from the upper-level module and the dynamic real-time execution status, this unit reorganizes some out-of-order computation data and distributes it into the data output buffer according to task type and task execution order.

[0122] If a single data computation unit comprises two computation engines, executing three tasks—4 (number of data blocks) + 2 (number of parity blocks) encoding, 4 (number of data blocks) + 1 (number of parity blocks) encoding, and 3 (number of data blocks) + 1 (number of parity blocks) decoding—using time-division multiplexing, then this data computation unit can complete these three tasks in two cycles. Please see [link / reference]. Figure 3 In cycle T1, two computing engines simultaneously process different data from the 4+2 encoding task; in cycle T2, computing engine 1 processes the 4+1 encoding task, and computing engine 2 processes the 3+1 decoding task. It can be seen that using multi-engine concurrency and multi-dimensional time-division multiplexing can improve resource utilization and thus increase data throughput.

[0123] Furthermore, this embodiment considers that the encoding / decoding device is located far from the host. If the encoding / decoding device directly sends the calculation results to the host, it will not only occupy the bus for a long time, but also have a large delay for each intermittent transmission, which will drag down the overall system performance. To this end, the present invention proposes a ring-shaped continuous command list. After the host reads and parses the command, it directly initiates data transfer to the encoding / decoding device through an independent path, reducing the number of interactions between the host and the encoding / decoding device and improving system performance.

[0124] Please see Figure 4 Encoding and decoding devices can be external or internal, such as... Figure 4 The command processor shown is the second processing core. Figure 4 The command processor shown has the following functions: whenever a calculation result is stored in the output buffer, it sends a data command to request the buffer, so that data transfer commands (i.e., address access commands) with the host are generated and cached for the host to read. The circular command table formed by the interactive commands is as follows: Figure 5 As shown, each command corresponds to an address segment and a valid flag bit, where 0 indicates invalid and 1 indicates valid. After the module assembles the command, it notifies the host that the command is ready. Upon receiving the command, the host reads it and maintains the read command in its own system. After obtaining the relevant data through a command, the host invalidates the command in its own system. The host also maintains a circular command table. Theoretically, as long as the commands are continuous, the encoding / decoding device only needs to notify the host once to achieve the transmission of all subsequent calculation results. This reduces the number of interactions with the host and improves the overall system performance.

[0125] like Figure 5 As shown, during initialization, the currently valid fields of newly generated data transfer commands are filled with 1, and the table bottom flag is inverted. The address in the ringing instruction command is filled with the first address of the list to link the command table into a ring. It can be seen that the entire command table consists of multiple data transfer commands and a ringing instruction command. The ringing instruction command indicates that the command has reached the bottom and commands need to be read from the table head.

[0126] As can be seen, this embodiment proposes a multi-dimensional parallel time-division multiplexing strategy from the perspective of RAID algorithm task scheduling and processing. It can flexibly configure the encoding and decoding matrix granularity, improve data throughput and resource utilization, and has easy scalability, high efficiency and good compatibility, which can meet different RAID algorithm tasks.

[0127] The following describes an encoding / decoding system provided by an embodiment of the present invention. The encoding / decoding system described below can be referred to in conjunction with other embodiments described herein.

[0128] See Figure 6As shown, an embodiment of the present invention discloses an encoding and decoding system, including: a host and at least one encoding and decoding device as described in any of the preceding embodiments.

[0129] In one implementation, there are multiple codec devices; the host is communicatively connected to each codec device.

[0130] In one implementation, the host communicates with each codec device via remote direct data access technology and / or direct memory access technology.

[0131] In one implementation, the host is connected to each codec device via an expansion chip.

[0132] In this embodiment, the encoding / decoding device includes: a first processing core, a first cache connected to the first processing core, a plurality of RAID algorithm executors connected to the first cache, a second cache connected to each RAID algorithm executor, and a second processing core connected to the second cache.

[0133] The first processing core is used to: acquire the target task and the data to be processed for the target task; the target task is used to implement the encoding or decoding calculation of at least one RAID algorithm.

[0134] Accordingly, the first processing core is used to divide the data to be processed into multiple sub-data according to the time-division multiplexing principle and the type of RAID algorithm implemented by the target task;

[0135] The first cache is used to store multiple sub-data items.

[0136] Each RAID algorithm executor is used to: read at least one sub-data that matches the RAID algorithm engine in itself from the first cache, and use the RAID algorithm engine to perform encoding or decoding calculations on the read sub-data.

[0137] The second cache is used to store the calculation results output by each RAID algorithm executor.

[0138] The second processing core is used to: generate an address access command based on the calculation results stored in the second cache, and send a command generated message to the host to which the encoding / decoding device belongs, so that the host can obtain the address access command based on the command generated message, and directly read the calculation results from the second cache according to the address access command; the address access command is used to access a storage address in the second cache.

[0139] In one implementation, the first processing core includes: a task parsing module for: parsing the target task to obtain parsing results; the parsing results include: the type of RAID algorithm implemented by the target task, the number of encoding operations, the number of decoding operations, and the valid data bit information, parity bit information, address list pointer, and parameter list pointer associated with the data to be processed; and a task processing module for: dividing the data to be processed when the parsing results are determined to be correct.

[0140] In one implementation, the task processing module includes: a time-division multiplexing control unit for determining time-division multiplexing allocation rules based on the parsing results and the operating status of each RAID algorithm executor; an address information reading unit for reading the storage address of the data to be processed on the host; and a data parameter reading unit for determining the row and column size of the data matrix to be read according to the time-division multiplexing allocation rules, reading the data from the storage address of the data to be processed on the host, and writing the read data into a first cache.

[0141] In one implementation, the task processing module is further configured to: discard the target task and send a task error message to the host to which the codec device belongs when it is determined that the parsing result is incorrect.

[0142] In one implementation, the first cache includes: a plurality of input cache units; correspondingly, each input cache unit is used to: store at least one sub-data that matches the same RAID algorithm engine.

[0143] In one implementation, each RAID algorithm executor includes: at least one RAID algorithm engine; correspondingly, any RAID algorithm engine is used to: perform encoding or decoding calculations using RAID1, RAID2, RAID3, RAID5, RAID6, or RAIDTP algorithms.

[0144] In one implementation, each RAID algorithm executor is further configured to: distribute at least one sub-data read to different RAID algorithm engines within itself according to the time-division multiplexing principle, so that different RAID algorithm engines within the same RAID algorithm executor operate according to the time-division multiplexing principle.

[0145] In one implementation, the second cache includes: a plurality of output cache units; correspondingly, each output cache unit is used to: store the calculation results output by the same RAID algorithm executor.

[0146] In one embodiment, the second processing core includes: a command generation module for generating an address access command based on the calculation results stored in the second cache; and a triggering module for sending a command generated message to the host to which the encoding / decoding device belongs, so that the host obtains the address access command based on the command generated message and directly reads the calculation results from the second cache according to the address access command.

[0147] In one implementation, the command generation module is specifically used to: when a new calculation result is detected being written to the second cache, generate a corresponding address access command based on the storage address of the new calculation result in the second cache, and record the address access command in the circular command table.

[0148] In one implementation, the triggering module is specifically used to: when an unprocessed address access command is detected in the ring command table, send a command generated message to the host to which the codec device belongs.

[0149] In one implementation, the triggering module is specifically used to: detect the valid flag information corresponding to each address access command in the ring command table, and determine whether there are any unprocessed address access commands in the ring command table based on the valid flag information.

[0150] In one implementation, the second processing core is further configured to: reorganize the calculation results output by different RAID algorithm executors in the second cache according to the target task, so as to integrate the calculation results.

[0151] In one implementation, each RAID algorithm executor includes an enabling device; correspondingly, the enabling device is used to enable or disable the respective RAID algorithm executor.

[0152] In one implementation, the first processing core is further configured to: send a ready message to the host to which the encoding / decoding device belongs, so that the host sends the target task and the data to be processed to the first processing core according to the ready message.

[0153] In one implementation, the first processing core further includes a configuration register; correspondingly, the configuration register is used to configure the number of encoding operations and the number of decoding operations in response to configuration operations from external input, so as to adjust the type of RAID algorithm supported by the encoding / decoding device.

[0154] For more detailed information on the working process of each module and unit in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0155] As can be seen, this embodiment provides an encoding and decoding system that is applicable to various types of RAID algorithms, is highly flexible, widely used, and can improve data throughput and hardware resource utilization, as well as the execution efficiency of RAID algorithms.

[0156] Furthermore, embodiments of the present invention also provide an electronic device. The aforementioned electronic device can be, for example,... Figure 7 The encoding / decoding device 1 shown can also be as follows: Figure 8 The encoding / decoding device 2 shown. Figure 7 and Figure 8 These are all structural diagrams of an electronic device according to an exemplary embodiment, and the content in the diagrams should not be considered as any limitation on the scope of the invention.

[0157] Figure 7 This is a schematic diagram of the structure of an encoding / decoding device 1 provided in an embodiment of the present invention. The encoding / decoding device 1 specifically includes: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory stores a computer program, which is loaded and executed by the processor to implement the relevant steps in the encoding / decoding process disclosed in any of the foregoing embodiments.

[0158] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the encoding / decoding device 1; the communication interface can create a data transmission channel between the encoding / decoding device 1 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0159] In addition, the memory, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system, computer programs and data, etc., and the storage method can be temporary storage or permanent storage.

[0160] The operating system is used to manage and control the various hardware devices and computer programs on the encoding / decoding device 1, so as to enable the processor to perform operations and processing on the data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the encoding / decoding methods disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include application update information and application developer information, among other things.

[0161] Figure 8This is a schematic diagram of the structure of an encoding / decoding device 2 provided in an embodiment of the present invention. The encoding / decoding device 2 may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0162] Typically, the encoding / decoding device 2 in this embodiment includes a processor and a memory.

[0163] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0164] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory is used to store at least the following computer program, which, after being loaded and executed by the processor, is capable of implementing the relevant steps in the encoding / decoding method executed by the encoding / decoding device 2 as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, application update information.

[0165] In some embodiments, the encoding / decoding device 2 may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0166] Those skilled in the art will understand that Figure 8The structure shown does not constitute a limitation on the codec device 2 and may include more or fewer components than shown.

[0167] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0168] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processing core, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of readable storage medium known in the art.

[0169] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A codec device, characterized in that, include: A first processing core, a first cache connected to the first processing core, a plurality of RAID algorithm executors connected to the first cache, a second cache connected to each RAID algorithm executor, and a second processing core connected to the second cache; The first processing core is used to: acquire the target task and the data to be processed in the target task; the target task is used to perform encoding or decoding calculations for at least one RAID algorithm; Accordingly, the first processing core is used to: divide the data to be processed according to the time-division multiplexing principle and the type of RAID algorithm implemented by the target task to obtain multiple sub-data; The first cache is used to: store the plurality of sub-data; Each RAID algorithm executor is used to: read at least one sub-data that matches the RAID algorithm engine in itself from the first cache, and use the RAID algorithm engine to perform encoding or decoding calculations on the read sub-data; The second cache is used to store the calculation results output by each RAID algorithm executor; The second processing core is used to: generate an address access command based on the calculation results stored in the second cache, and send a command generated message to the host to which the codec device belongs, so that the host can obtain the address access command based on the command generated message, and directly read the calculation results from the second cache according to the address access command; the address access command is used to access a storage address in the second cache; The second processing core is further configured to: perform row and column reorganization on the calculation results output by different RAID algorithm executors in the second cache according to the target task, so as to integrate the calculation results; wherein, the output data integration unit built into the second processing core reorders the generated data according to the distribution order, number of verifications, number of decodings, block size information and time-division multiplexing rules, and outputs it sequentially to the time-division multiplexing output data allocation unit in the second processing core; the time-division multiplexing output data allocation unit allocates the received data to the second cache in order according to the data size and the number of verification or recovery data, and notifies the host that the task is complete; The first processing core includes: The task parsing module is used to: parse the target task to obtain the parsing result; the parsing result includes: the type of RAID algorithm implemented by the target task, the number of encodings calculated, the number of decodings calculated, and the valid data bit information, parity bit information, address list pointer, and parameter list pointer associated with the data to be processed; The task processing module is used to: divide the data to be processed when it is determined that the parsing result is correct; The task processing module includes: The time-division multiplexing control unit is used to: determine time-division multiplexing allocation rules based on the parsing results and the operating status of each RAID algorithm executor; wherein, it accelerates data processing at both large-granularity and small-granularity levels by using time-division multiplexing between different RAID algorithm executors and time-division multiplexing within a single RAID algorithm executor; at the large-granularity level, it dynamically detects the availability of idle RAID algorithm executor resources and the resource requirements of tasks in real time, and allocates tasks to idle RAID algorithm executors for processing; if all RAID algorithm executors are running at full capacity, it allows different computing engines to run using a time-division multiplexing mechanism within a single RAID algorithm executor to maximize the utilization of idle time within that single RAID algorithm executor; The address information reading unit is used to: read the storage address of the data to be processed in the host; The data parameter reading unit is used to: determine the row and column size of the data matrix to be read according to the time-division multiplexing allocation rule, read the data from the storage address of the data to be processed on the host, and write the read data into the first cache; The second processing core includes: The command generation module is used to: generate address access commands based on the calculation results stored in the second cache; The triggering module is used to: send a command generated message to the host to which the codec device belongs, so that the host obtains the address access command according to the command generated message, and directly reads the calculation result from the second cache according to the address access command; The first processing core also includes: a configuration register; Accordingly, the configuration register is used to: configure the number of encoding operations and the number of decoding operations in the encoding calculation in response to configuration operations from external input, so as to adjust the type of RAID algorithm supported by the encoding / decoding device; Each RAID algorithm executor includes: an enabling device; Accordingly, the enabling device is used to enable or disable the corresponding RAID algorithm executor.

2. The encoding / decoding device according to claim 1, characterized in that, The task processing module is further configured to: discard the target task and send a task error message to the host to which the encoding / decoding device belongs when it is determined that the parsing result is incorrect.

3. The encoding / decoding device according to claim 1, characterized in that, The first cache includes: multiple input cache units; Accordingly, each input cache unit is used to store at least one sub-data that matches the same RAID algorithm engine.

4. The encoding / decoding device according to claim 1, characterized in that, Each RAID algorithm executor includes: at least one RAID algorithm engine; Accordingly, any RAID algorithm engine is used to perform encoding or decoding calculations using RAID1, RAID2, RAID3, RAID5, RAID6, or RAIDTP algorithms.

5. The encoding / decoding device according to claim 4, characterized in that, Each RAID algorithm executor is also used to: distribute at least one sub-data read to different RAID algorithm engines within itself according to the time-division multiplexing principle, so that different RAID algorithm engines in the same RAID algorithm executor can operate according to the time-division multiplexing principle.

6. The encoding / decoding device according to claim 1, characterized in that, The second cache includes: multiple output buffer units; Accordingly, each output buffer unit is used to store the calculation results output by the same RAID algorithm executor.

7. The encoding / decoding device according to claim 1, characterized in that, The command generation module is specifically used to: when a new calculation result is detected being written to the second cache, generate a corresponding address access command based on the storage address of the new calculation result in the second cache, and record the address access command in the circular command table.

8. The encoding / decoding device according to claim 7, characterized in that, The triggering module is specifically used to: when an unprocessed address access command is detected in the ring command table, send a command generated message to the host to which the codec device belongs.

9. The encoding / decoding device according to claim 8, characterized in that, The triggering module is specifically used to: detect the valid flag information corresponding to each address access command in the ring command table, and determine whether there are any unprocessed address access commands in the ring command table based on the valid flag information.

10. The encoding / decoding apparatus according to any one of claims 1 to 9, characterized in that, The first processing core is further configured to: send a ready message to the host to which the encoding / decoding device belongs, so that the host sends the target task and the data to be processed to the first processing core according to the ready message.

11. A codec system, characterized in that, include: The host and the encoding / decoding device as described in any one of claims 1 to 10.

12. The encoding / decoding system according to claim 11, characterized in that, There are multiple encoding / decoding devices; the host is communicatively connected to each encoding / decoding device.

13. The encoding / decoding system according to claim 12, characterized in that, The host communicates with each codec device via remote direct data access technology and / or direct memory access technology.

14. The encoding / decoding system according to claim 12, characterized in that, The host is connected to each codec device via an expansion chip.

Images