Data processing method, electronic equipment and medium

Abstract

The invention discloses a data processing method, electronic equipment and a medium, and relates to the technical field of computers.The method comprises the steps that in a current data processing round, first identification information of a data storage area corresponding to a current processing task is obtained; calling a data processing parameter corresponding to the current processing task according to the first identification information; and storing the data processing parameters to a parameter cache region corresponding to the first identification information. According to the method and the device, the data processing parameters are stored in the parameter cache regions corresponding to the data storage regions, so that even if response data of each storage region is out of order due to response delay, the data processing parameters corresponding to the response data can still be found from the parameter cache regions corresponding to the data storage regions; response data does not need to be rearranged, the calculation efficiency of verification data is improved, and cache resources needed by rearrangement are released.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data processing method, an electronic device, and a medium. Background Art

[0002] Disk Array (Redundant Arrays of Independent Disks, RAID) striping technology divides data blocks and stores them dispersedly on multiple physical disks, achieving automatic balancing of I / O loads, effectively avoiding multi-process access conflicts, and greatly improving the efficiency of parallel data access. In related technologies, a parity disk mechanism is used to achieve data redundancy and fault tolerance, and data is restored based on parity data in case of disk failures to ensure storage reliability. During the calculation and writing process of parity data, data read requests need to be sent to obtain disk data scattered in multiple storage areas. However, due to the difference in response delays of each storage area, the order of the response data is inconsistent with the order of the request initiation. Subsequent data needs to wait for the previous data to be processed before it can be processed. Therefore, out-of-order data needs to be cached and re-sorted first, and this calculation process has obvious defects such as low parity data processing efficiency and cache resource occupation. Summary of the Invention

[0003] This application provides a data processing method, an electronic device, and a medium to at least solve the problems of low parity data processing efficiency and cache resource occupation.

[0004] This application provides a data processing method, including: In the current data processing round, obtain the first identification information of the data storage area corresponding to the current processing task; According to the first identification information, call the data processing parameters corresponding to the current processing task; Store the data processing parameters in the parameter cache area corresponding to the first identification information.

[0005] This application also provides a data processing device, including: An obtaining module, configured to obtain the first identification information of the data storage area corresponding to the current processing task in the current data processing round; A calling module, configured to call the data processing parameters corresponding to the current processing task according to the first identification information; A storage module, configured to store the data processing parameters in the parameter cache area corresponding to the first identification information.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above data processing methods when executing the computer program.

[0007] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above data processing methods.

[0008] The present application also provides a computer program product including a computer program, which, when executed by a processor, implements the steps of any of the above data processing methods.

[0009] Through the present application, in the process of calculating verification data, based on the current data processing round, the first identification information of the data storage area corresponding to the current processing task is obtained, and the data processing parameters corresponding to the current processing task are called based on the first identification information, and the data processing parameters are stored in the parameter cache area corresponding to the first identification information. In this way, since the data processing parameters have been pre-stored in the parameter cache area corresponding to the data storage area, even if the response data is out of order due to response delay in each data storage area, the data processing parameters corresponding to the response data can still be found in the parameter cache area corresponding to the data storage area. The calculation engine can perform partial verification calculation based on the response data and the corresponding data processing parameters, without waiting until all response data is returned and then rearranging each response data for verification calculation, improving the calculation efficiency of verification data and releasing cache resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To more clearly illustrate the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0011] Figure 1 FIG. is a schematic diagram of a scenario for calculating verification data provided by an embodiment of the present application; Figure 2 FIG. is a flowchart of a data processing method provided by an embodiment of the present application; Figure 3 FIG. is a schematic structural diagram of a data processing system provided by an embodiment of the present application; Figure 4 FIG. is a schematic diagram of storing data processing parameters in a corresponding parameter cache area provided by an embodiment of the present application; Figure 5 FIG. is a schematic diagram of reading data processing parameters from a parameter cache area based on response information provided by an embodiment of the present application; Figure 6 FIG. is a schematic diagram of a scenario for a data processing method provided by an embodiment of the present application; Figure 7A schematic diagram of another data processing method provided by an embodiment of this application; Figure 8 A schematic structural diagram of a data processing apparatus provided by an embodiment of this application; Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of this application. Detailed implementation manners

[0012] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part rather than all of the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0013] It should be noted that in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects and not to describe a specific order or sequence.

[0014] To enable those skilled in the art of this technology to better understand the solution of this application, the following further detailed description of this application will be given in conjunction with the accompanying drawings and specific implementation manners.

[0015] First, an exemplary introduction to the application scenario of the embodiment of this application will be given.

[0016] Disk array (Redundant Arrays of Independent Disks, RAID) striping technology divides continuous data blocks to obtain several small databases, and disperses several small data blocks to multiple physical disks to achieve automatic balancing of I / O load, effectively avoiding multi-process access conflicts and greatly improving the data parallel access efficiency. This technology allows multiple processes to access different parts of the data simultaneously without disk conflicts. Especially when accessing data sequentially, it can maximize the I / O parallel capability and significantly improve the data processing performance. Among them, the common RAID5 and RAID6 achieve data redundancy and fault tolerance through the parity disk mechanism, and restore data based on Galois field operations in case of disk failures to ensure storage reliability.

[0017] Specifically, in RAID5, there is 1 parity disk (P disk) and N data disks (D disks). The parity disk stores parity data, and the data disks store disk data. The following relationship holds between the parity data and the disk data: P = X1 * D1 ⊕ X2 * D2 ⊕ X3 * D3 ⊕ … ⊕ Xn * Dn Where P is the parity data, * is the multiplication operation in the Galois field, ⊕ is the exclusive OR operation, X1, X2, X3…Xn are data processing parameters (also known as multiplication coefficients, calculation parameters) used to calculate P, and D1, D2, D3, …Dn are the disk data in the data storage area, that is, the disk data on the data disks. Since the disk data is stored in the data storage area, it needs to be read from the data storage area to calculate the parity data. Therefore, in the embodiments of the present application, the disk data is referred to as the data to be read.

[0018] In the RAID5 mode, when the disk data of any one disk is lost or damaged, it can be solved through the disk data on other data disks.

[0019] RAID6 adds one more parity disk (Q disk) on the basis of RAID5. For RAID6, the following relationship holds between the parity data and the disk data: P = X1 * D1 ⊕ X2 * D2 ⊕ X3 * D3 ⊕ … ⊕ Xn * Dn Q = Y1 * D1 ⊕ Y2 * D2 ⊕ Y3 * D3 ⊕ … ⊕ Yn * Dn Where Q is another parity data, and Y1, Y2, Y3…Yn are data processing parameters used to calculate Q.

[0020] In the RAID6 mode, the data of two disks can be recovered, and the fault tolerance ability is higher than that of RAID5.

[0021] In the parity disk mechanism, any data update on a data disk requires recalculating the parity and writing the recalculated parity data to the parity disk. During the process of calculating the parity data, first, the disk data required for calculating the parity data needs to be read from each data storage area in the on-chip storage area; then, the read disk data and the corresponding data processing parameters are sent to the calculation engine so that the calculation engine calculates the parity data based on the read disk data and the corresponding data processing parameters; after the calculation engine calculates the parity data, the parity data is written back to the preset parity disk (such as the above-mentioned P disk, Q disk).

[0022] Figure 1 is a schematic diagram of a scenario for calculating parity data. In Figure 1In the process of calculating and writing check data, it involves a data reading control module, a data reordering control module, a calculation engine, and multiple caches (calculation parameter cache, reorder buffer, calculation result cache). From Figure 1 It can be seen that the data to be read for calculating check data is stored in multiple data storage areas, namely storage area 0, storage area 1,..., storage area n.

[0023] The data reading control module is used to initiate multiple data reading requests based on the bus protocol (Advanced eXtensible Interface, AXI). For different data storage areas, different AXI ID-corresponding data reading requests are sent. The data to be read (i.e., disk data) returned by the data reading requests corresponding to the same AXI ID (i.e., the data reading requests for the same data storage area) is in order, while the data to be read returned by the data reading requests corresponding to different AXI IDs (i.e., the data corresponding to different data reading requests is stored in different storage areas) may be out of order due to different response latencies of different storage areas.

[0024] The data reordering control module is used to sort each piece of data to be read. This is because the data is randomly scattered in different data storage areas, and there are differences in the response latency of each data storage area to the data reading request. The order of the data to be read returned by each data storage area is inconsistent with the order of initiating the data reading request. And the data processing parameters are obtained in a preset order, such as the order of X1, X2, X3... Xn mentioned above. That is to say, in the process of the calculation engine calculating the check data, first, a multiplication operation in the Galois field is performed based on D1 and X1, and then a multiplication operation in the Galois field is performed based on D2 and X2, and so on. Therefore, the data reordering control module sorts the data to be read in each storage area according to the data reading request.

[0025] The reorder buffer is used to cache the data to be read returned by each storage area so that the data reordering control module can reorder each piece of data to be read.

[0026] However, in the above process of calculating and writing check data, on the one hand, since the order of the data to be read returned by each storage area is inconsistent with the order of the data reading request, it is necessary to use the reorder buffer to cache each piece of data to be read, resulting in the occupation of more cache resources. On the other hand, since the out-of-order returned data to be read is written into the reorder buffer so that the data reordering control module can sort each piece of data to be read, each piece of data to be read will not be processed in time, and the processing efficiency of the check data will decrease, resulting in a significant reduction in performance.

[0027] For example, if three consecutive data reading requests are initiated to read Data 1, Data 2, and Data 3 respectively. In the worst case, the return order of each data may be Data 3, Data 2, and Data 1. After the data processing parameters corresponding to Data 1 and Data 1 are calculated and completed, Data 3 and Data 2 will be processed by the computing engine. In addition, in the related art, data reading requests corresponding to multiple tasks can be initiated simultaneously. However, since there is only one computing engine, each task must be processed sequentially according to the order of the tasks, that is, when one task is being processed in the computing engine, the next task will wait until the current task is executed and completed by the computing engine before being processed. In this case, even if the data to be read for the next task has been returned before the current task is processed, it will not be processed by the computing engine, resulting in low data processing efficiency.

[0028] In view of this, an embodiment of the present application provides a data processing method to solve the problems of low data processing efficiency and occupying cache resources during the calculation of verification data mentioned above.

[0029] It should be noted that the execution subject of the data processing method provided by the embodiment of the present invention can be a data processing device, and the data processing device can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. Among them, the electronic device can be a server or a terminal. Among them, the server in the embodiment of the present application can be a single server or a server cluster composed of multiple servers. The terminal in the embodiment of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, and other intelligent hardware devices such as a smart robot. In the following method embodiments, the execution subject is an electronic device as an example for description.

[0030] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0031] In this embodiment, a data processing method is provided, which can be used for the above-mentioned electronic devices, such as data processing servers, etc. Figure 2 is a flowchart of a data processing method provided according to an embodiment of the present invention, as Figure 2 shown, the process includes: S101: In the current data processing round, obtain the first identification information of the data storage area corresponding to the current processing task.

[0032] Specifically, a processing task refers to a task of processing data. In the embodiments of the present application, the processing task refers to calculating verification data. Specifically, the processing task specifically refers to calculating verification data based on at least one data to be read and the data processing parameters corresponding to each data to be read. Each time a data to be read used for calculating verification data and the data processing parameter corresponding to the data to be read are obtained, it is a data processing round. For the current processing task, the number of data processing rounds is the same as the number of data to be read used when calculating verification data corresponding to the current processing task.

[0033] The data storage area corresponding to the current processing task refers to the storage area storing the disk data used for calculating verification data, such as the on-chip storage area. A processing task requires at least one disk data (also referred to as data to be read), and the data processing parameters corresponding to each disk data. Among them, the disk data needs to send a data reading request to the corresponding data storage area, and in the case where the corresponding data storage area responds to the data reading request, the disk data (referred to as data to be read in the embodiments of the present application) is returned.

[0034] The first identification information of the data storage area is used to indicate the data storage area. Exemplarily, the first identification information may be the physical path of the data storage area, the logical partition number, the number of the storage area, etc. The present application does not make specific settings for the first identification information of the data storage area, and it can be defined according to the actual situation.

[0035] In a possible implementation manner, the data storage area stores the data to be read corresponding to at least one task respectively. That is to say, a data storage area can store the data to be read corresponding to multiple tasks. The multiple data to be read corresponding to one task can be stored in multiple data storage areas respectively. For example, the data storage area 1 stores the data to be read 0 and the data to be read 1 corresponding to the task 1, and the data to be read 0 and the data to be read 2 corresponding to the task 2. The data storage area 2 stores the data to be read 2 corresponding to the task 1 and the data to be read 1 corresponding to the task 2.

[0036] S102: Call the data processing parameters corresponding to the current processing task according to the first identification information.

[0037] Specifically, the data processing parameters corresponding to the current processing task refer to the parameters involved in the process of calculating verification data. Based on the above description of the verification data, it can be obtained that the data processing parameters correspond to the disk data one by one. For each processing task, the calculation of verification data is implemented based on multiple data to be read and the corresponding data processing parameters.

[0038] In the embodiments of the present application, the data processing parameters are pre-stored in the computing parameter cache. In the computing parameter cache, the data processing parameters have a binding relationship with the processing tasks. After obtaining the first identification information, the data processing parameters corresponding to the current processing task are obtained from the computing parameter cache. That is to say, the first identification information serves as an action condition, and after obtaining the first identification information, the data processing parameters corresponding to the current processing task are called.

[0039] S103: Store the data processing parameters in the parameter cache area corresponding to the first identification information.

[0040] Specifically, there is a corresponding relationship between the parameter cache area and the data storage area, and the parameter cache area corresponding to the data storage area can be found based on the first identification information. It can be understood that for multiple disk data (i.e., data to be read) stored in a data storage area, the data processing parameters corresponding to each disk data are stored in the parameter cache area corresponding to the data storage area. The specific implementation manner of the parameter cache area in the present application is not specifically limited and can be memory, solid state drive (SSD), etc.

[0041] In some embodiments, the method provided by the embodiments of the present application further includes the following content: a1: Select a computing engine corresponding to the second identification information from multiple computing engines according to the second identification information of the current processing task, and obtain the third identification information corresponding to the computing engine.

[0042] Wherein, each computing engine is used to subsequently calculate the verification data corresponding to one or more processing tasks.

[0043] Specifically, the second identification information is used to identify the current processing task. The second identification information can be the task name, task number, etc. of the current processing task. The second identification information of the current processing task in the present application is not specifically limited and can be selected according to the actual situation.

[0044] A computing engine is a hardware / software unit used to execute processing tasks and has specific computing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), etc.

[0045] The third identification information is used to indicate the computing engine. As an identifier of the computing engine, it can quickly locate and call the computing engine. Exemplarily, the third identification information can be the label, name, etc. of the computing engine.

[0046] In a possible implementation, in the above a1, the computing engine corresponding to the second identification information is selected in the following manner, and the third identification information of the computing engine is obtained: First, sort multiple processing tasks, and determine the sorting order value of the current processing task among the multiple processing tasks according to the second identification information of the current processing task.

[0047] Specifically, the multiple processing tasks can be sorted according to the priority, task request time, resources required by the task, etc. of each processing task to generate an execution queue. The sorting order value represents the position serial number of each processing task in the sorting queue and is used for subsequent computing engine allocation. For example, if the sorting order value corresponding to task A is 1, the sorting order value corresponding to task B is 2, and the sorting order value corresponding to task C is 3, it indicates that among these three tasks, task A has the highest priority, task B has medium priority, and task C has the lowest priority. That is, the computing engine is allocated to task A first, and then to task B and task C in sequence.

[0048] Then, determine the computing engine corresponding to the current processing task according to the sorting order value and the total number of computing engines, and obtain the third identification information corresponding to the computing engine.

[0049] Optionally, use the modulo algorithm to calculate the computing engine number according to the sorting order value and the total number of computing engines. Take the modulo of the sorting order value of the task by the total number of computing engines, and the obtained remainder is directly mapped to the computing engine number, that is, computing engine number = sorting order value % total number of computing engines. Specifically, assume that there are N computing engines numbered from 0 to N - 1. Divide the sorting order value of the processing task by N and take the remainder, and use the remainder as the allocated computing engine number. For example, there are 3 computing engines numbered computing engine 0, computing engine 1, and computing engine 2 respectively. If the sorting order value of a certain processing task is 5, then 5 % 3 = 2, and computing engine 2 is allocated to this processing task.

[0050] In this way, the modulo algorithm allocates computing engines for processing tasks by taking the modulo of the task sorting order value by the total number of computing engines, and can quickly determine the target computing engine without complex calculation logic, which is suitable for scenarios with high requirements for response speed. When the total number of computing engines is fixed, the modulo algorithm can ensure that processing tasks are evenly distributed to each computing engine, making the load of each computing engine relatively balanced and avoiding extreme situations such as resource limitations or overload of some computing engines.

[0051] Optionally, the sorting order value is converted into a hash value within a fixed range through a hash function, and then mapped to the list of computing engines to achieve the allocation of computing engines. Specifically, the sorting order value of the processing task is input into a preset hash function (such as MurmurHash, Cyclic Redundancy Check32 (CRC32), etc.) to obtain a hash value, and then the hash value is modulo-divided by the total number of computing engines to obtain the computing engine number corresponding to the processing task. Suppose there are 4 computing engines numbered Computing Engine 0, Computing Engine 1, Computing Engine 2, and Computing Engine 3, and the sorting order value of a certain processing task is 7. Suppose the hash function is Hash(x) = x * 31 + 17, then the hash value corresponding to this processing task is Hash(7) = 7 * 31 + 17 = 234, and 234 % 4 = 2, that is, Computing Engine 2 is allocated for this processing task.

[0052] In this way, converting the sorting order value into a hash value through a hash function can effectively disperse the task distribution. Even if there is aggregation or skewed distribution in the task order values, it can ensure that the computing engines obtain balanced loads. In the scenario where the number of computing engines changes, the consistent hashing feature of hash mapping can minimize the change in the allocation range of processing tasks and only adjust the affected processing tasks, effectively reducing the resource migration cost caused by the expansion or contraction of computing engines.

[0053] In the implementation manner of this application, the sorting order value of each processing task is determined based on the second identification information, accurately measuring the resource requirement priority of the processing task to ensure that high-priority or urgent processing tasks can obtain computing engine resources first. For example, during the process of calculating multiple pieces of verification data, the processing task of the verification data with high priority is given a higher sorting order value and computing engines are allocated preferentially. Compared with random allocation, the average processing time of the processing task is effectively shortened. At the same time, combining the total number of computing engines to allocate computing engines for processing tasks can make full use of computing engine resources, avoiding the situation where some computing engines are overloaded or some computing engines are idle, and effectively improving the overall utilization rate of each computing engine resource. In addition, by allocating computing engines according to the sorting order value of each processing task, even if some computing engines fail, the processing tasks can be quickly migrated to other available computing engines according to the task sorting order, avoiding situations such as task suspension or data loss, and ensuring the continuity and stability of verification data calculation.

[0054] In another possible implementation manner, in the above a1, the computing engine corresponding to the second identification information is selected and the third identification information of the computing engine is obtained through the following method: Select a computing engine corresponding to the second identification information from multiple computing engines based on the algorithm complexity of the verification data corresponding to the current processing task, and obtain the third identification information corresponding to the computing engine.

[0055] Specifically, the algorithm complexity of the verification data corresponding to the current processing task refers to the degree of resource consumption in terms of time / space during the calculation of the verification data. For example, the double-parity calculation of RAID6 involves Galois field multiplication and multiple exclusive OR operations, so its time complexity is higher than that of the single-parity calculation of RAID5.

[0056] Optionally, determine the algorithm complexity of the verification data corresponding to the current processing task based on the data volume of the data to be read corresponding to the current processing task.

[0057] Exemplarily, the data volume of the data to be read can be the number of all data to be read for calculating the verification data in the current processing task, the physical storage size occupied by all data to be read, the number of data storage areas where all data to be read are located, etc. Determine the algorithm complexity of the verification data corresponding to the current processing task based on a pre-constructed model in which the algorithm complexity has a linear relationship with the data volume and the data volume of the data to be read.

[0058] In this way, accurately match the computing engine according to the algorithm complexity of the verification data corresponding to the current processing task. Assign low-complexity tasks to low-performance computing engines and high-complexity tasks to high-performance computing engines to optimize the resource utilization rate of each computing engine. For example, allocating the processing task of calculating the parity data of RAID6 to an FPGA instead of a CPU can shorten the calculation duration of this processing task.

[0059] a2: Store the third identification information in the parameter cache area corresponding to the first identification information.

[0060] Optionally, there is a binding relationship between the data processing parameters in the parameter cache area and the third identification information used to indicate the computing engine. In this way, when the data processing parameters are determined, according to the binding relationship between the data processing parameters and the third identification, the computing engine used to calculate the data processing parameters can be directly determined, accelerating the calculation process of the verification data and improving the efficiency of calculating the verification data.

[0061] Of course, based on the second identification information, it can be flexibly defined according to task requirements (such as real-time performance, priority). For example, dynamically adjust the engine allocation strategy according to the load of each computing engine. During peak business hours, preferentially allocate high-priority tasks to idle and efficient engines. At the same time, when adding a new computing engine, only need to update the identification mapping relationship to quickly incorporate the newly added computing engine into the scheduling to cope with the growth of data scale and task complexity.

[0062] In the embodiments of the present application, according to the second identification information of the current processing task, the computing engine corresponding to the second identification information can be selected from multiple computing engines according to the characteristics of the current task, and the third identification information of the computing engine is stored in the parameter cache area corresponding to the first identification information, so that the verification calculation and the data processing parameter reading are coordinated. Even if the storage area response delay causes the out-of-order of the data to be read, based on the third identification information in the parameter cache area, the computing engine corresponding to the data processing parameter can be directly started, avoiding waiting for data reordering and improving the computing efficiency.

[0063] In some embodiments, in the above S101, the first identification information of the data storage area corresponding to the current processing task is obtained in the following manner: First, within the current data processing round, the address pointer of the i-th data to be read corresponding to the current processing task is obtained.

[0064] Among them, there are multiple data to be read corresponding to the current processing task, and the address pointer of one data to be read is obtained in each data processing round.

[0065] Specifically, the address pointer is an identifier pointing to the physical location of the data to be read in the data storage area. The address pointer of the data to be read can be stored in the pointer cache in advance. In this way, through the address pointer of the data to be read, the specific location of the data storage area is directly mapped, avoiding the time-consuming processes of global scanning or multi-level search that cannot be queried, and realizing the accurate positioning of the data to be read.

[0066] Then, according to the address pointer, the data storage area to which the i-th data to be read belongs is determined, and the first identification information of the data storage area is obtained.

[0067] In the embodiments of the present application, in each data processing round, the address pointer of one data to be read is obtained, the corresponding data storage area is identified, and then the corresponding data processing parameters are called and stored.

[0068] In some embodiments, after the above S101, the data processing method provided by the embodiments of the present application further includes the following content: b1: Generate a data read request according to the first identification information and the address pointer.

[0069] In the embodiments of the present application, the data read request may be an AXI data read request.

[0070] b2: Send the data read request to the data storage area corresponding to the first identification information.

[0071] In this way, after the data storage area receives the data read request, it responds to the request and returns the data to be read corresponding to the data read request.

[0072] In a possible implementation, after the above b2, the method provided by the embodiments of the present application further includes: c1: After obtaining the response information fed back by the first data storage area, extract the first data to be read and the fourth identification information corresponding to the first data storage area from the response information.

[0073] Wherein, the first data storage area is any one of a plurality of data storage areas, and the plurality of data storage areas are used to store the data to be read corresponding to at least one task respectively.

[0074] It can be understood that after sending a plurality of data read requests to a plurality of data storage areas in sequence, due to different response delays of different data storage areas, the order of the response information returned by each data storage area is not necessarily the same as the order of each data read request. Taking the example of sending 3 data read requests in sequence, for example, first send data read request 1 to data storage area a, then send data read request 2 to data storage area b, and then send data read request 3 to data storage area a. Data storage area b responds to data read request 2 and first returns the data corresponding to data read request 2. Then data storage area a responds to data read requests 1 and 3 respectively and returns the data corresponding to data read request 1 and the data corresponding to data read request 3 in sequence.

[0075] Here, the plurality of data read requests sent to the plurality of data storage areas can be data read requests for the same task or data read requests for multiple tasks, and the present application does not make specific limitations in this regard. For example, after first sending the data read requests corresponding to the multiple data to be read required in task 1, then send the data read requests corresponding to the multiple data to be read required in task 2.

[0076] c2: According to the fourth identification information, obtain the first data processing parameter corresponding to the first data to be read from the parameter cache area corresponding to the fourth identification information.

[0077] c3: Calculate the first sub-check data according to the first data to be read and the first data processing parameter.

[0078] Wherein, the first sub-check data is the sub-data for calculating the first check data, and the first check data is the check data to be generated for the task corresponding to the first data to be read.

[0079] Specifically, the first check data may be P in the above RAID5, or Q in RAID6, etc. The present application does not make specific limitations on the first check data. The sub-data in the first check data may be data obtained by performing a multiplication operation in the Galois field on a data to be read and corresponding data processing parameters, such as X1*D1, Y1*D1, etc.

[0080] Exemplarily, a computing engine may be utilized to calculate the first sub-check data according to the first data to be read and the first data processing parameter.

[0081] In the embodiment of the present application, there is a corresponding relationship between the parameter cache area where the first data processing parameter is located and the data storage area. After obtaining the response information fed back by the first data storage area, the first data processing parameter is retrieved from the parameter cache area corresponding to the first data storage area, so that the first data to be read in the response information and the first data processing parameter are strictly corresponding, avoiding the situation where the data processing parameter and the data to be read do not correspond and resulting in the inability to calculate the check data. Compared with the related art, there is no need to wait for all the data to be read returned by each data storage area and then reorder the data to be read according to the data read request. On the one hand, the cache resources required to store each data to be read are saved. On the other hand, it is realized that the check data can be calculated after obtaining the response information fed back by the storage area, improving the efficiency of calculating the check data.

[0082] Before the above c3, the data processing method provided by the embodiment of the present application further includes: First, read the fifth identification information from the parameter buffer area corresponding to the fourth identification information.

[0083] Then, determine the target computing engine according to the fifth identification information, so as to subsequently utilize the target computing engine to calculate the first sub-check data according to the first data to be read and the first data processing parameter.

[0084] Optionally, after reading the first data processing parameter and the fifth identification information, delete the first data processing parameter and the fifth identification information from the parameter cache area corresponding to the third identification information.

[0085] In this way, after the first data processing parameter and the fifth identification information are read, the first data processing parameter and the fifth identification information will be used by the computing engine to calculate the check data. At this time, the first data processing parameter and the fifth identification information are deleted from the parameter cache area to avoid logical errors caused by the reuse of these data, thereby affecting the calculation accuracy of the check data.

[0086] Optionally, the data processing method provided by the embodiment of the present application further includes the following content: After obtaining all the sub-verification data corresponding to the current processing task, verification data corresponding to the current processing task is generated based on all the sub-verification data.

[0087] In the embodiment of the present application, since there is a binding relationship between the first data processing parameter and the third identifier for indicating the computing engine, and there is also a corresponding relationship between the processing task and the computing engine, in this way, the first sub-verification data calculated by the computing engine is the sub-data in the verification data corresponding to the processing task.

[0088] According to the above description of the calculation of the verification data, the verification data is obtained based on all the sub-verification data, that is, the verification data is obtained by performing an exclusive OR operation on each sub-verification data. Therefore, after obtaining all the sub-verification data corresponding to the current processing task, perform an exclusive OR operation on each sub-verification data to obtain the verification data.

[0089] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method.

[0090] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0091] Figure 3 It is a schematic structural diagram of a data processing system. In Figure 3 it, the data processing system includes a data reading control module, a parameter distribution module, a parameter selection module, a computing engine allocation module, computing engine 0, computing engine 1, a computing result output module, and multiple caches (that is, Figure 3 the computing parameter cache, parameter sub-cache 0, parameter sub-cache 1,..., parameter sub-cache n, computing result cache 0, and computing result cache 1) in

[0092] Stored in the computing parameter cache are the data processing parameters required for calculating the verification data. Each data processing parameter corresponds to the corresponding data to be read one by one and is arranged in a preset order.

[0093] The data reading control module is used to allocate a computing engine for the processing task. Considering that a single computing engine does not support the alternate processing of two tasks, two computing engines are set in this embodiment. The allocation of the computing engines adopts the ping-pong method, that is, the first processing task is allocated to computing engine 0, the second processing task is allocated to computing engine 1, the third processing task is allocated to computing engine 0, and so on alternately.

[0094] The data reading control module is also used to initiate a data reading request. When initiating a data reading request, it is necessary to control the number of data reading requests initiated because there are two computing engines, and these two computing engines can process at most two processing tasks. Therefore, data reading requests for two tasks can be initiated. Before all the disk data of one task is returned, a data reading request for a third task cannot be initiated.

[0095] When the data reading control module sends a data reading request, it will take out a data pointer from the data pointer cache in a preset order, and then initiate a data reading request using the corresponding AXI ID (i.e., the first identification information corresponding to the data storage area). At the same time, the first identification information corresponding to the data storage area and the third identification information used to indicate the computing engine are sent to the parameter distribution module.

[0096] After obtaining the first identification information corresponding to the data storage area, the parameter distribution module reads a data processing parameter from the computing parameter cache, and writes the data processing parameter and the third identification information used to indicate the computing engine into the parameter sub-cache corresponding to the first identification information. The Figure 3 number of parameter sub-caches included is the same as the number of data storage areas storing the data to be read. Each parameter sub-cache corresponds to a data storage area.

[0097] Multiple data to be read for executing processing tasks are stored in multiple data storage areas, such as Figure 3 storage area 0, storage area 1,..., storage area n in. Each data to be read corresponds to an address pointer, which is used to indicate the specific position of the data to be read in the data storage area.

[0098] In Figure 4In this case, data reading requests for two processing tasks (i.e., Task 1 and Task 2) are initiated in sequence, that is, a data reading request for Data 0 corresponding to Task 1, a data reading request for Data 1, ……, a data reading request for Data 4, a data reading request for Data 5 corresponding to Task 2, a data reading request for Data 6, ……, a data reading request for Data 9. The computing engine 0 is used to execute the calculation of the verification data corresponding to Task 1. The computing engine 1 is used to execute the calculation of the verification data corresponding to Task 2. For Task 1, the data used to calculate the verification data includes Data 0 - Data 4. Data 0 is stored in storage area 3, Data 1 is stored in storage area 2, Data 2 is stored in storage area 3, Data 3 is stored in storage area 1, Data 4 is stored in storage area 0. Data 0 corresponds to data processing parameter 0 in the parameter cache, Data 1 corresponds to data processing parameter 1 in the parameter cache …… For Task 2, the data used to calculate the verification data includes Data 5 - Data 9. Data 5 is stored in storage area 0, Data 6 is stored in storage area 1, Data 7 is stored in storage area 2, Data 8 is stored in storage area 3, Data 9 is stored in storage area 0. Data 5 corresponds to data processing parameter 5 in the parameter cache, Data 6 corresponds to data processing parameter 6 in the parameter cache …… The address pointers of each data in each storage area are stored in the pointer cache, that is, Pointer 0 to Pointer 9.

[0099] The data reading control module first obtains the address pointer corresponding to Data 0 (i.e., Pointer 0) and the first identification information of the data storage area (i.e., storage area 3) from the pointer cache, and generates a data reading request for Data 0 based on the address pointer and the first identification information. When initiating the data reading request for Data 0, it reads the data processing parameter 0 corresponding to Parameter 0 from the calculation parameter cache and stores the data processing parameter 0 in the parameter sub-cache 3 corresponding to the first identification information. And so on, data reading requests corresponding to Data 1, Data 2, ……, Data 9 are sent in a preset order, and the corresponding data processing parameters and the identification information corresponding to the computing engine are stored in the corresponding parameter storage areas.

[0100] Such as Figure 4As shown in the figure, the parameter cache area includes parameter sub-cache 0, parameter sub-cache 1, and parameter sub-cache 2. Parameter sub-cache 3 corresponds to storage area 3. In parameter sub-cache 3, data processing parameters corresponding to data 0, data 2, and data 8 stored in storage area 3 are stored, namely data processing parameter 0, data processing parameter 2, and data processing parameter 8. In addition, identification information 0 of computing engine 0 corresponding to data processing parameter 0, identification information 0 of computing engine 0 corresponding to data processing parameter 2, and identification information 1 of computing engine 1 corresponding to data 8 are also stored in parameter sub-cache 3. Similarly, parameter sub-cache 2 corresponds to storage area 2. In parameter sub-cache 2, data processing parameters corresponding to data 1 and data 7 stored in storage area 2 are stored, namely data processing parameter 1 and data processing parameter 7. In addition, identification information 0 of computing engine 0 corresponding to data processing parameter 1 and identification information 1 of computing engine 1 corresponding to data processing parameter 7 are also stored in parameter sub-cache 2. Parameter sub-cache 1 corresponds to storage area 1. In parameter sub-cache 1, data processing parameters corresponding to data 3 and data 6 stored in storage area 1 are stored, namely data processing parameter 3 and data processing parameter 6. In addition, identification information 0 of computing engine 0 corresponding to data processing parameter 3 and identification information 1 of computing engine 1 corresponding to data processing parameter 6 are also stored in parameter sub-cache 1. Parameter sub-cache 0 corresponds to storage area 0. In parameter sub-cache 0, data processing parameters corresponding to data 4, data 5, and data 9 stored in storage area 0 are stored, namely data processing parameter 4, data processing parameter 5, and data processing parameter 9. In addition, identification information 0 of computing engine 0 corresponding to data processing parameter 4, identification information 1 of computing engine 1 corresponding to data processing parameter 5, and identification information 1 of computing engine 1 corresponding to data 9 are also stored in parameter sub-cache 0. For the data processing parameters in the same parameter sub-cache, according to the first-in, first-out principle, that is, the data processing parameters written in first will also be read out first.

[0101] The parameter selection module reads the data processing parameters and the identification information indicating the computing engine from the parameter cache area corresponding to the first identification information according to the first identification information in the response information of the data storage area, and sends the data processing parameters and the identification information corresponding to the computing engine to the computing engine allocation module.

[0102] Figure 5 The parameter sub-cache involved in Figure 4 is the parameter sub-cache involved in Figure 5When any data storage area returns response information containing data to be read, the parameter selection module can accurately find the data processing parameters corresponding to the data to be read and the identification information of the computing engine from the parameter cache area corresponding to the data storage area. For example, when storage area 0 returns response information, the data processing parameters and the identification information indicating the computing engine are read from the corresponding parameter sub-cache 0 in storage area 0.

[0103] The computing engine allocation module is used to allocate the data to be read returned by the data storage area and the corresponding data processing parameters to the corresponding computing engine. The basis for the allocation by the computing engine allocation module is the identification information of the computing engine output by the parameter selection module. Taking each computing engine in Figure 4 as an example, if the identification information is 0, the data to be read and the data processing parameters are sent to computing engine 0. If the identification information is 1, the data to be read and the data processing parameters are sent to computing engine 1. By using two computing engines to alternately execute processing tasks, the data returned by two tasks intertwined can be processed, and the situation where one task blocks another task will not occur.

[0104] Computing engine 0 calculates sub-check data based on the received data to be read and the corresponding data processing parameters, then determines the final check data according to all the sub-check data, and saves the check data to the calculation result cache 0. Computing engine 1 calculates sub-check data based on the received data to be read and the corresponding data processing parameters, then determines the final check data according to all the sub-check data, and saves the check data to the calculation result cache 1.

[0105] The function of the calculation result output module is to alternately select one of the two calculation result caches for output. The order of data output should be consistent with the order of tasks. The principle of allocating computing engines in the data reading control module is that the first task is allocated to computing engine 0, the second task is allocated to computing engine 1, the third task is allocated to computing engine 0, and so on alternately. The output order of the calculation result output module adopts the same strategy as the task allocation order, that is, the calculation result in computing engine 0 is output for the first time, the calculation result in computing engine 1 is output for the second time, and the calculation result in computing engine 0 is output for the third time.

[0106] Figure 6 and Figure 7 are the schematic diagrams of the scenarios of two data processing methods. In Figure 6 and Figure 7Among them, the data to be read corresponding to Task 1 are Data 0 and Data 1, and the data to be read corresponding to Task 2 are Data 2 and Data 3. The identification information of the data storage areas where Data 0, Data 1, Data 2, and Data 3 are located are 0, 1, 2, and 3 respectively. The data processing parameters corresponding to Data 0, Data 1, Data 2, and Data 3 are Parameter 0, Parameter 1, Parameter 2, and Parameter 3 respectively. Data read requests for the data to be read corresponding to the two tasks are initiated in sequence, that is, the data read request corresponding to Data 0, the data read request corresponding to Data 1, the data read request corresponding to Data 2, and the data read request corresponding to Data 3. The order in which each data storage area returns each data to be read is Data 3, Data 1, Data 2, Data 0, that is, each data storage area returns each data in a disordered manner.

[0107] Figure 6 is a data processing method in the related art. In Figure 6 after receiving each data to be read, it is stored in the reordering cache. After the data to be read are rearranged in the order of sending the data read requests, the rearranged data to be read and their respective corresponding data processing parameters are sent to the computing engine for the computing engine to calculate the verification data. As can be seen from Figure 6 the data 3 with a relatively early return order will not be processed immediately, but stored in the reordering cache. It will not be calculated until Data 0 and Parameter 0, Data 1 and Parameter 1, and Data 2 and Parameter 2 are all processed by the computing engine. Strictly maintaining the order of data and tasks will extend the data waiting time, resulting in a large delay in task processing and low processing efficiency.

[0108] Figure 7 is a data processing method provided according to an embodiment of the present application. After receiving the first returned Data 3, the computing engine allocation module will send Data 3 and the corresponding data processing parameter 3 of Data 3 to the computing engine 0 executing Task 2. After receiving Data 1, it will send Data 1 and the corresponding data processing parameter 1 of Data 1 to the computing engine 1 executing Task 1. After receiving Data 2, it will send Data 2 and the corresponding data processing parameter 2 of Data 2 to the computing engine 0 executing Task 2. After receiving Data 0, it will send Data 0 and the corresponding data processing parameter of Data 0 to the computing engine 1 executing Task 1. As can be seen from Figure 7 even if the order in which each data storage area returns each data is inconsistent with the order of sending the data read requests, the computing engine allocation module will send the returned data to the corresponding computing engine, enabling the computing engine to calculate the verification data based on the returned data in a timely manner without waiting for all the data to be read to be returned and rearranging the data to be read in the order of the data read requests, saving the cache space (i.e., the reordering cache resource) for storing each data to be read, shortening the calculation duration of the verification data, and improving the calculation efficiency of the verification data.

[0109] In an embodiment of the present application, a data processing device is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0110] This embodiment provides a data processing device, as Figure 8 shown, including: An acquisition module 801, configured to acquire first identification information of a data storage area corresponding to a current processing task within a current data processing round; An invocation module 802, configured to invoke data processing parameters corresponding to the current processing task according to the first identification information; A storage module 803, configured to store the data processing parameters in a parameter cache area corresponding to the first identification information.

[0111] In a possible implementation manner, the device further includes: A selection module, configured to select a computing engine corresponding to the second identification information from multiple computing engines according to the second identification information of the current processing task, and acquire third identification information corresponding to the computing engine. Each computing engine is used to subsequently calculate verification data corresponding to one or more processing tasks; The storage module 803 is further configured to store the third identification information in the parameter cache area corresponding to the first identification information.

[0112] In a possible implementation manner, the acquisition module 801 is specifically configured to, within a current data processing round, acquire an address pointer of the i-th data to be read corresponding to the current processing task, where there are multiple data to be read corresponding to the current processing task, and an address pointer of one data to be read is acquired in each data processing round; Determine the data storage area to which the i-th data to be read belongs according to the address pointer, and acquire the first identification information of the data storage area.

[0113] In a possible implementation manner, the device further includes: A generation module, configured to generate a data read request according to the first identification information and the address pointer; A sending module, configured to send the data read request to the data storage area corresponding to the first identification information to obtain the data to be read corresponding to the data read request.

[0114] In a possible implementation, the obtaining module 801 is further configured to, after obtaining the response information fed back by the first data storage area, extract the first data to be read and the fourth identification information corresponding to the first data storage area from the response information, where the first data storage area is any one of a plurality of data storage areas, and the plurality of data storage areas are used to store the data to be read corresponding to at least one task; According to the fourth identification information, obtain the first data processing parameter corresponding to the first data to be read from the parameter cache area corresponding to the fourth identification information; The apparatus further includes a calculation module, and the calculation module is configured to calculate the first sub-check data according to the first data to be read and the first data processing parameter, where the first sub-check data is the sub-data for calculating the first check data, and the first check data is the check data to be generated for the task corresponding to the first data to be read.

[0115] In a possible implementation, the generating module is further configured to, when obtaining all the sub-check data corresponding to the current processing task, generate the check data corresponding to the current processing task according to all the sub-check data.

[0116] In a possible implementation, in the apparatus, there is a binding relationship between the data processing parameter in the parameter cache area and the third identification information for indicating the calculation engine.

[0117] In a possible implementation, the obtaining module 801 is further configured to read the fifth identification information from the parameter buffer area corresponding to the fourth identification information; The apparatus further includes a determining module, and the determining module is configured to determine the target calculation engine according to the fifth identification information, so as to subsequently use the target calculation engine to calculate the first sub-check data according to the first data to be read and the first data processing parameter.

[0118] In a possible implementation, the apparatus further includes: A deleting module, configured to delete the first data processing parameter and the fifth identification information from the parameter cache area corresponding to the third identification information after reading the first data processing parameter and the fifth identification information.

[0119] In a possible implementation, when there are multiple processing tasks, the selecting module is specifically configured to sort the multiple processing tasks, and determine the sorting order value of the current processing task among the multiple processing tasks according to the second identification information of the current processing task; Determine the calculation engine corresponding to the current processing task according to the sorting order value and the total number of calculation engines, and obtain the third identification information corresponding to the calculation engine.

[0120] In a possible implementation, the selection module is specifically configured to select a computing engine corresponding to the second identification information from multiple computing engines based on the algorithm complexity of calculating the verification data corresponding to the current processing task, and obtain the third identification information corresponding to the computing engine.

[0121] In a possible implementation, in the device, the data storage area stores the data to be read corresponding to at least one task respectively.

[0122] Through the device provided by the embodiments of the present application, in the process of calculating the verification data, based on the current data processing round, the first identification information of the data storage area corresponding to the current processing task is obtained, and the data processing parameters corresponding to the current processing task are called based on the first identification information, and the data processing parameters are stored in the parameter cache area corresponding to the first identification information. In this way, since the data processing parameters have been pre-stored in the parameter cache area corresponding to the data storage area, even if the response delays of each storage area cause the response data to be disordered, the data processing parameters corresponding to the response data can still be found from the parameter cache area corresponding to the data storage area. The computing engine can perform partial verification calculations based on the response data and the corresponding data processing parameters, without waiting until all the response data is returned and then rearranging each response data for verification calculation, improving the calculation efficiency of the verification data and releasing the cache resources.

[0123] For the description of the features in the embodiments corresponding to the data processing device, reference can be made to the relevant descriptions in the embodiments corresponding to the data processing method, which will not be elaborated here one by one.

[0124] The embodiments of the present application also provide an electronic device, as Figure 9 shown, including a memory 10 and a processor 20. A computer program is stored in the memory 10, and the processor 20 is configured to run the computer program to execute the steps in any of the above data processing method embodiments.

[0125] The embodiments of the present application also provide a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above data processing method embodiments when running.

[0126] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM for short), a random access memory (RAM for short), a mobile hard disk, a magnetic disk, or an optical disc that can store computer programs.

[0127] Embodiments of the present application also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any of the above-described data processing method embodiments are implemented.

[0128] Embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-described data processing method embodiments are implemented.

[0129] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0130] The above has introduced in detail a data processing method, an electronic device, and a medium provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data processing method, characterized in that, The method is applied to a data processing server, and the method includes: In the current data processing round, obtain the first identification information of the data storage area corresponding to the current processing task; According to the first identification information, call the data processing parameters corresponding to the current processing task; Store the data processing parameters in the parameter cache area corresponding to the first identification information.

2. The method according to claim 1, characterized in that, The method further includes: According to the second identification information of the current processing task, select a computing engine corresponding to the second identification information from multiple computing engines, and obtain the third identification information corresponding to the computing engine. Each computing engine is used to calculate the verification data corresponding to one or more processing tasks subsequently; Store the third identification information in the parameter cache area corresponding to the first identification information.

3. The method according to claim 2, wherein The step of obtaining the first identification information of the data storage area corresponding to the current processing task in the current data processing round includes: In the current data processing round, obtain the address pointer of the i-th data to be read corresponding to the current processing task, where the data to be read corresponding to the current processing task includes multiple pieces, and one address pointer of a data to be read is obtained in each data processing round; According to the address pointer, determine the data storage area to which the i-th data to be read belongs, and obtain the first identification information of the data storage area.

4. The method according to claim 3, characterized in that After obtaining the first identification information of the data storage area corresponding to the current processing task in the current data processing round, the method further includes: Generate a data reading request according to the first identification information and the address pointer; Send the data reading request to the data storage area corresponding to the first identification information to obtain the data to be read corresponding to the data reading request.

5. The method according to claim 4, wherein After sending the data reading request to the data storage area corresponding to the first identification information, the method further includes: When the response information fed back by the first data storage area is obtained, extract the first data to be read and the fourth identification information corresponding to the first data storage area from the response information, where the first data storage area is any one of multiple data storage areas, and the multiple data storage areas are used to store the data to be read corresponding to at least one task respectively; According to the fourth identification information, obtain the first data processing parameter corresponding to the first data to be read from the parameter cache area corresponding to the fourth identification information; Calculate the first sub-verification data according to the first data to be read and the first data processing parameter. The first sub-verification data is the sub-data for calculating the first verification data, and the first verification data is the verification data to be generated for the task corresponding to the first data to be read.

6. The method according to claim 5, wherein The method further includes: When all the sub-verification data corresponding to the current processing task is obtained, generate the verification data corresponding to the current processing task according to all the sub-verification data.

7. The method according to claim 6, wherein There is a binding relationship between the data processing parameters in the parameter cache area and the third identification information indicating the computing engine.

8. The method according to any one of claims 5 to 7, characterized in that Before calculating the first sub-check data according to the first data to be read and the first data processing parameter, the method further includes: Read the fifth identification information from the parameter buffer area corresponding to the fourth identification information; Determine a target calculation engine according to the fifth identification information, so as to subsequently use the target calculation engine to calculate the first sub-check data according to the first data to be read and the first data processing parameter.

9. The method according to claim 8, wherein The method further includes: After reading the first data processing parameter and the fifth identification information, delete the first data processing parameter and the fifth identification information from the parameter cache area corresponding to the third identification information.

10. The method according to claim 2, characterized in that, When there are multiple processing tasks, selecting a calculation engine corresponding to the second identification information from multiple calculation engines according to the second identification information of the current processing task, and obtaining the third identification information corresponding to the calculation engine specifically includes: Sort the multiple processing tasks, and determine the sorting order value of the current processing task among the multiple processing tasks according to the second identification information of the current processing task; Determine the calculation engine corresponding to the current processing task according to the sorting order value and the total number of calculation engines, and obtain the third identification information corresponding to the calculation engine.

11. The method according to claim 2, wherein Selecting a calculation engine corresponding to the second identification information from multiple calculation engines according to the second identification information of the current processing task, and obtaining the third identification information corresponding to the calculation engine specifically includes: Select a calculation engine corresponding to the second identification information from multiple calculation engines based on the algorithm complexity of calculating the check data corresponding to the current processing task, and obtain the third identification information corresponding to the calculation engine.

12. The method according to any one of claims 1-7, 10, and 11, characterized in that, The data storage area stores the data to be read corresponding to at least one task respectively.

13. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for implementing the steps of the data processing method according to any one of claims 1-12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the data processing method according to any one of claims 1-12 when executed by a processor.

15. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the data processing method according to any one of claims 1 to 12 when executed by a processor.

Images