Data processing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a data processing method and device, computer equipment and a storage medium. The data processing method is applied to an SIMD architecture system, and the SIMD architecture system comprises a plurality of computing units. The method comprises the following steps: splitting a to-be-processed data model into a plurality of data processing sub-models; respectively sending each data processing sub-model to each calculation unit for parallel iterative operation to obtain a calculation result of each calculation unit; performing integration processing on the calculation results according to a preset data processing mode to obtain an iterative calculation result of the to-be-processed data model; for each iterative operation, sending the obtained iterative calculation result and the displacement during combination in the corresponding divide-and-conquer method to each calculation unit for parallel iterative operation, and respectively obtaining each array element; and performing integration processing on each array element to obtain an updated calculation array. The data processing efficiency in the divide-and-conquer method is improved from three dimensions.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular, to a data processing method, apparatus, computer equipment and storage medium. Background Art

[0002] Divide and Conquer is an efficient algorithm for calculating the eigenvalues ​​and eigenvectors of tridiagonal matrices. The overall idea of ​​the divide and conquer method is to decompose a large matrix into several smaller sub-matrices and solve them separately, and then merge the solutions of the sub-matrices. dlaed3 (Double-precision Linear Algebra Eigenvalue Calculation (third module), where d represents double-precision floating point numbers, la represents linear algebra, e represents eigenvalue, and d3 represents the third module for processing eigenvalue problems) is a module of the divide and conquer method in the LAPACK library (a function library used in science and technology), which is responsible for the last step of merging the eigenvalues ​​of the matrix.

[0003] The function of the dlaed3 module is mainly used for two parts of calculation: the first part is to calculate each eigenvector through iteration, and the second part is to update the merge array Q in sequence according to the eigenvector. Both parts are implemented in series, that is, each mathematical quantity to be calculated needs to be calculated in sequence before the next step of calculation can be performed. Specifically, for the first part of the calculation:

[0004] A single eigenvector v(i) is calculated using the following formula (1):

[0005] v(i)=orig(j)+tau(i)-delta(i) (1)

[0006] After the single vector v(i) is calculated by the above formula (1), the single vector v(i) needs to be substituted into the above formula (1) again to traverse the vertical column j in the matrix, that is, the following formula (2) is obtained:

[0007] v(i+1)=v(i)*(orig(j)+tau(j)-delta(i))*(1 / delta(j)-delta(i)) (2)

[0008] In formula (1)-formula (2), v(i+1) represents the single vector eigenvalue obtained by solving the i+1th matrix, v(i) represents the single vector eigenvalue obtained by solving the ith matrix, (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, orig(j) represents the initial vector eigenvalue in the vertical direction of the matrix, tau(j) represents the vertical correction scaling factor of the vector eigenvalue in the vertical direction of the matrix, delta(i) represents the horizontal displacement when merging arrays in the divide-and-conquer method, delta(j) represents the vertical displacement when merging arrays in the divide-and-conquer method, i represents the horizontal index of the matrix, j represents the vertical index of the matrix, and i≠j.

[0009] For the second part of the calculation:

[0010] The merged array Q can be updated sequentially according to the above formulas (1) and (2) by the following formula (3):

[0011] Q(i,j)=v(i)*(1 / (orig(j)-delta(i)+tau(j))) (3)

[0012] In formula (3), Q(i,j) represents the updated array, v(i) represents the single vector eigenvalue obtained by solving the i-th matrix, orig(j) represents the vertical initial vector eigenvalue of the matrix, delta(i) represents the horizontal displacement when merging arrays in the divide-and-conquer method, delta(j) represents the vertical displacement when merging arrays in the divide-and-conquer method, i represents the horizontal index of the matrix, j represents the vertical index of the matrix, and i≠j.

[0013] As described above, the first part of the calculation and the second part of the calculation are both serial calculations and are interrelated, and the calculation efficiency is relatively low.

[0014] Therefore, there is an urgent need for a method to improve the efficiency of divide-and-conquer data processing. Summary of the invention

[0015] The embodiments of the present application provide a data processing method, apparatus, computer device and storage medium.

[0016] A first aspect of an embodiment of the present application provides a data processing method, which is applied to a SIMD architecture system, wherein the SIMD architecture system includes a plurality of computing units; the method includes:

[0017] Split the data model to be processed into multiple data processing sub-models;

[0018] Sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units;

[0019] Integrate and process the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed;

[0020] For each iterative operation, the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method are sent to each of the calculation units for parallel iterative operation to obtain each array element respectively;

[0021] Each of the array elements is integrated to obtain an updated calculation array.

[0022] In an optional embodiment of the present application, the data model to be processed is a tridiagonal matrix model in the divide-and-conquer method; correspondingly, the data processing sub-model is a data iterative processing model.

[0023] In an optional embodiment of the present application, the data model to be processed is:

[0024] v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i)))

[0025] Correspondingly, the plurality of data processing sub-models include at least one of the following two data processing sub-models:

[0026] The first data processing sub-model: orig(j)+tau(j)-delta(i)

[0027] Second data processing submodel: 1 / (delta(j)-delta(i))

[0028] Among them, (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, v(i) represents the single vector eigenvalue obtained by the i-th solution, orig(j) represents the vertical initial vector eigenvalue of the matrix, tau(j) represents the vertical correction scaling factor of the vector eigenvalue in the vertical direction of the matrix, delta(i) represents the horizontal displacement when merging arrays in the divide-and-conquer method, delta(j) represents the vertical displacement when merging arrays in the divide-and-conquer method, i represents the horizontal index of the matrix, j represents the vertical index of the matrix, and i≠j.

[0029] In an optional embodiment of the present application, the plurality of data processing sub-models include the first data processing sub-model and the second data processing sub-model. Correspondingly, the calculation results are integrated and processed according to a preset data processing method to obtain the iterative calculation result of the data model to be processed, including:

[0030] The first calculation result of the first data processing sub-model and the second calculation result of the second data processing sub-model are multiplied to obtain the iterative calculation result.

[0031] In an optional embodiment of the present application, for the first data processing sub-model, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units at least includes:

[0032] Sending each of the lateral displacements to each of the calculation units respectively, and sending the first sum of the first longitudinal initial eigenvalue and the first longitudinal correction scaling factor to each of the calculation units respectively for parallel iterative calculation to obtain a first iterative calculation result;

[0033] The sums of the other longitudinal initial eigenvalues ​​and the corresponding longitudinal correction scaling factors are sequentially sent to the calculation units for parallel iterative calculations, and the iterative calculation results are sequentially obtained;

[0034] For each of the calculation units, all iterative calculation results are integrated to obtain a first calculation result of the calculation unit; wherein the first calculation result refers to the calculation result of the first data processing sub-model.

[0035] In an optional embodiment of the present application, for the second data processing sub-model, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units at least includes:

[0036] Sending each of the lateral displacements to each of the calculation units respectively;

[0037] Sending the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units respectively calculating the difference between the first longitudinal displacement and each of the lateral displacements;

[0038] The other longitudinal displacements are sent to the calculation units in sequence, and the calculation units calculate the difference between the other longitudinal displacements and the lateral displacements in sequence;

[0039] The difference values ​​obtained each time are integrated and the inverse is calculated to obtain the second calculation result of each calculation unit; wherein the second calculation result refers to the calculation result of the second data processing sub-model.

[0040] In an optional embodiment of the present application, for each iterative operation, the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method are sent to each of the computing units for parallel iterative operation to obtain each array element respectively, including:

[0041] Sending the first calculation result and the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units respectively calculates the first array element;

[0042] The other calculation results and the corresponding other longitudinal displacements are sent to each of the calculation units in turn, and each of the calculation units calculates and obtains other array elements respectively.

[0043] According to a second aspect of an embodiment of the present application, a data processing device is provided, which is applied to a SIMD architecture system, wherein the SIMD architecture system includes a plurality of computing units; the device includes:

[0044] A splitting module, used for splitting the data model to be processed into multiple data processing sub-models;

[0045] A first calculation module, used for sending each of the data processing sub-models to each of the calculation units for parallel iterative calculation, to obtain calculation results of each of the calculation units;

[0046] A second calculation module is used to integrate and process the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed;

[0047] A third calculation module is used for sending the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method to each of the calculation units for parallel iterative calculation for each iterative calculation, so as to obtain each array element respectively;

[0048] The updating module is used to integrate the array elements to obtain an updated calculation array.

[0049] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0050] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0051] An embodiment of the present application provides a data processing method, which splits a data model to be processed into data processing sub-models according to an operation method, and then sends each of the data processing sub-models to each of the computing units for parallel iterative operations to obtain a calculation result of each of the computing units, and then integrates and processes each of the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed; for each iterative operation, the obtained iterative calculation result and the corresponding displacement amount when merging in the divide-and-conquer method are sent to each computing unit for parallel iterative operations to obtain each array element; and each of the array elements is integrated to obtain the updated calculation array.

[0052] On the one hand, the embodiment of the present application uses a SIMD architecture system, which can use a single instruction of the SIMD architecture system to control multiple computing units, and multiple computing units can perform independent and parallel calculations, which can greatly improve data processing efficiency;

[0053] On the second aspect, the embodiment of the present application applies a SIMD architecture system, that is, a single instruction controls multiple computing units, thereby improving program operation efficiency;

[0054] On the third aspect, the embodiment of the present application splits the data model to be processed into data processing sub-models according to the operation method. Through the data processing sub-model, the two parts of the calculation in the dlaed3 module are split into at least four modules for independent parallel calculation, thereby further improving the data processing efficiency.

[0055] In summary, the embodiments of the present application improve the data processing efficiency in the divide-and-conquer method from three dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0057] Figure 1 A flowchart of a data processing method provided for one embodiment of the present application;

[0058] Figure 2 A flowchart of a data processing method provided for one embodiment of the present application;

[0059] Figure 3 A schematic diagram of a process for determining a first calculation result in a data processing method provided in one embodiment of the present application;

[0060] Figure 4 A flowchart of a data processing method provided for one embodiment of the present application;

[0061] Figure 5A schematic diagram of a process for determining a second calculation result in a data processing method provided in one embodiment of the present application;

[0062] Figure 6 A flowchart of a data processing method provided for one embodiment of the present application;

[0063] Figure 7 A schematic diagram of a process for determining each array element in a data processing method provided in one embodiment of the present application;

[0064] Figure 8 A schematic diagram of the structure of a data processing device provided in one embodiment of the present application;

[0065] Fig. 9 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0066] In the process of implementing the present application, the inventors found that there is an urgent need for a method to improve the efficiency of divide-and-conquer data processing.

[0067] In response to the above problems, embodiments of the present application provide a data processing method, apparatus, computer device and storage medium.

[0068] The solutions in the embodiments of the present application can be implemented using various computer languages, for example, the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0069] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0070] The following is a brief description of the application environment of the data processing method provided in the embodiment of the present application:

[0071] The data processing provided in the embodiment of the present application is applied to the divide and conquer method in the standard Lapack library to solve the matrix eigenvalue dlaed3 module, and the formation and update of the update merge array are performed based on the dlaed3 module. The divide and conquer method in the embodiment of the present application is an efficient algorithm for calculating the eigenvalues ​​and eigenvectors of a tridiagonal matrix. The overall idea of ​​the divide and conquer method is to decompose a large matrix into several smaller sub-matrices and solve them separately, and then merge the solutions of the sub-matrices.

[0072] The dlaed3 module in the embodiment of the present application refers to a special module in LAPACK (Linear Algebra PACKage), which is mainly used to solve the eigenvalues ​​and eigenvectors of symmetric tridiagonal matrices. It is an indispensable part of the divide-and-conquer method for eigenvalue problems, and is particularly suitable for eigenvalue problems of large-scale symmetric matrices. The main functions of the dlaed3 module are: merging the eigenvalues ​​and eigenvectors of sub-matrices, processing correction matrices and calculating corrected eigenvalues, performing aggregation operations to speed up calculations, ensuring numerical stability and accuracy when solving eigenvalues ​​by the divide-and-conquer method, etc. dlaed3 is one of the core steps in the divide-and-conquer method for eigenvalue calculations, and can significantly improve computational efficiency, especially when processing large-scale symmetric matrices.

[0073] The SIMD architecture system in the embodiment of the present application refers to a hardware architecture system that can implement single instruction multiple data (Single Instruction Multiple Data, SIMD). In a computer system, a single instruction is allowed to act on multiple data points at the same time, which is usually used to process large-scale data parallel tasks, such as image processing, signal processing, and scientific computing. The SIMD architecture system in the embodiment of the present application, in the same computing core in a chip, the SIMD architecture system can use a single instruction to control multiple computing units to perform the same operation (including but not limited to reading data, adding, subtracting, multiplying, storing data, etc.), and this architecture system can greatly improve the efficiency of processing large-scale data, and parallel computing can greatly improve data calculation efficiency. The data processing method provided in the embodiment of the present application is implemented based on the SIMD architecture system, and the SIMD architecture system includes multiple computing units, each computing unit operates independently, but data interaction can be performed between them.

[0074] See also Figure 1 The data processing method provided in the embodiment of the present application includes the following steps 101 to 105:

[0075] Step 101: split the data model to be processed into multiple data processing sub-models;

[0076] This splitting method can be split according to the budget method to simplify subsequent calculations, wherein there is only one operation method in one of the data processing sub-models; for example, the operations in any one of the data processing sub-models are all addition, or all subtraction, or all multiplication, or all division, and so on.

[0077] The data model to be processed refers to a matrix or data model to be solved in the divide-and-conquer method. The data model to be processed is generally a mathematical model. An exemplary data model to be processed is provided as follows:

[0078] v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) (4)

[0079] In formula (4), (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, v(i) represents the single vector eigenvalue obtained by the i-th solution, orig(j) represents the initial vector eigenvalue in the longitudinal direction of the matrix, tau(j) represents the longitudinal correction scaling factor of the vector eigenvalue in the longitudinal direction of the matrix, delta(i) represents the lateral displacement when the array is merged in the divide-and-conquer method, delta(j) represents the longitudinal displacement when the array is merged in the divide-and-conquer method, i represents the lateral index of the matrix, j represents the longitudinal index of the matrix, and i≠j.

[0080] It should be explained that the above-mentioned expression of the data model to be processed does not constitute a specific limitation on the data model to be processed in the embodiments of the present application, and the form and content of the data model to be processed can be flexibly adjusted according to actual conditions.

[0081] Step 102: Send each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units;

[0082] For example, for each data processing sub-model, independent calculations are performed based on the received data without mutual interaction. The independent calculations in the embodiments of the present application refer to iterative calculations, which require multiple loop calculations and a high number of calculations. The results obtained are owned by each calculation unit alone and do not interfere or affect each other. The specific process of the iterative calculations in the embodiments of the present application is not specifically limited here, such as solving, function calculations, basic calculations, etc., which can be flexibly adjusted according to actual conditions.

[0083] Step 103: Integrate the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed;

[0084] The preset data processing method can be: multiplication, division, inversion, function fitting, mapping, etc., which can be flexibly adjusted according to actual conditions. The embodiment of the present application does not make specific limitations, as long as it can be combined with the above step 102 to embody the purpose of splitting first and then integrating.

[0085] Step 104: for each iterative operation, the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method are sent to each of the calculation units for parallel iterative operation to obtain each array element respectively;

[0086] The parallel iterative operation in this step may be the same as or different from that in the above step 102, and is not specifically limited here, and can be flexibly adjusted according to actual conditions. For example, in the divide-and-conquer method, each numerical element can be obtained by calculating according to the iterative calculation result obtained each time and the corresponding displacement.

[0087] Step 105: Integrate the array elements to obtain the updated calculation array.

[0088] After obtaining each calculation data element, the corresponding target array can be constructed by constructing a matrix, a set, a chart, etc. The embodiment of the present application does not specifically limit the specific method of integration and the final array form, and can be flexibly adjusted according to actual conditions.

[0089] The meaning of the calculation array Q in the embodiment of the present application: In dlaed3, the calculation array Q is an orthogonal matrix used to store the eigenvectors of a matrix. Specifically, in the merge step of the divide-and-conquer method, the calculation array Q contains the eigenvectors of the smaller sub-matrices, which will be combined to form the eigenvectors of the larger matrix.

[0090] The process of updating and merging array Q may include: The updating and merging process in the dlaed3 module is to adjust and combine the eigenvectors of the sub-matrices by rotation (or transformation). This process combines the eigenvectors of the sub-matrices (stored in Q) by calculating the rotation matrix to form the eigenvectors of the final large matrix.

[0091] The updated array of the dlaed3 module indicates the update of the eigenvalue and eigenvector. The eigenvalue can describe the vibration mode and stability conditions of the system. The calculated result of the eigenvector reflects how the state or mode of the system changes. For physical systems, it is related to vibration mode, heat conduction, fluid dynamics, etc. Common applications of solving matrix eigenvalue problems: Matrix eigenvalue problems have a wide range of applications, especially in scenarios where it is necessary to analyze system stability, vibration mode, energy state, and dynamic behavior, such as the following scenarios:

[0092] a. Structural mechanics and vibration analysis

[0093] In structural mechanics, matrix eigenvalue analysis is used to determine the vibration modes and resonant frequencies of structures such as buildings, bridges, and mechanical systems. The stiffness matrix and mass matrix of a structure are often used to form a generalized eigenvalue problem. By solving the eigenvalues ​​and eigenvectors, the natural frequencies of the structure and the corresponding vibration modes can be obtained.

[0094] b. Power system stability analysis

[0095] In power systems, eigenvalue analysis is used to evaluate the stability of power networks. The eigenvalues ​​of the power system state matrix can be used to determine whether the system will stabilize and return to equilibrium when disturbed.

[0096] c. Quantum mechanics

[0097] In quantum mechanics, eigenvalue problems often arise when solving for the energy state of a quantum system. The eigenvalues ​​of the Hamiltonian operator represent the possible energy states of the system, while the corresponding eigenvectors represent the wave functions of the system in these energy states.

[0098] d. Machine Learning

[0099] In machine learning, eigenvalue decomposition is used in dimensionality reduction algorithms such as principal component analysis (PCA). PCA simplifies data representation, reduces dimensionality and retains most information by performing eigenvalue decomposition on the data covariance matrix to find the main component directions.

[0100] An embodiment of the present application provides a data processing method, which splits a data model to be processed into data processing sub-models according to an operation method, and then sends each of the data processing sub-models to each of the computing units for parallel iterative operations to obtain a calculation result of each of the computing units, and then integrates and processes each of the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed; for each iterative operation, the obtained iterative calculation result and the corresponding displacement amount when merging in the divide-and-conquer method are sent to each computing unit for parallel iterative operations to obtain each array element; and each of the array elements is integrated to obtain the updated calculation array.

[0101] On the one hand, the embodiment of the present application uses a SIMD architecture system, which can use a single instruction of the SIMD architecture system to control multiple computing units, and multiple computing units can perform independent and parallel calculations, which can greatly improve data processing efficiency;

[0102] On the second aspect, the embodiment of the present application applies a SIMD architecture system, that is, a single instruction controls multiple computing units, thereby improving program operation efficiency;

[0103] On the third aspect, the embodiment of the present application splits the data model to be processed into data processing sub-models according to the operation method. Through the data processing sub-model, the two parts of the calculation in the dlaed3 module are split into at least four modules for independent parallel calculation, thereby further improving the data processing efficiency.

[0104] In summary, the embodiments of the present application improve the data processing efficiency in the divide-and-conquer method from three dimensions.

[0105] In an optional embodiment of the present application, the data model to be processed is a tridiagonal matrix model in the divide-and-conquer method; the corresponding data processing sub-model is a data iterative processing model.

[0106] The tridiagonal matrix model refers to a matrix in which all elements except the main diagonal and the two adjacent diagonals above and below are zero. Specifically, for an n×n tridiagonal matrix A, only the following elements may be non-zero: the elements A(i,i) on the main diagonal, the diagonal elements A(i,i+1) above the main diagonal, and the diagonal elements A(i,i-1) below the main diagonal. The elements A(i,j) in other positions (when |ij|>1) are all zero.

[0107] The following is an example 5×5 tridiagonal matrix:

[0108]

[0109] That is, the present application can realize parallel processing of the tridiagonal matrix model and improve the data processing efficiency of the tridiagonal matrix model.

[0110] In an optional embodiment of the present application, the above-mentioned data model to be processed is:

[0111] v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) (4)

[0112] Correspondingly, the plurality of data processing sub-models include at least one of the following two data processing sub-models:

[0113] The first data processing submodel: orig(j)+tau(j)-delta(i)(5)

[0114] Second data processing submodel: 1 / (delta(j)-delta(i))(6)

[0115] In formula (4) to formula (6), (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, v(i) represents the i-th solved single vector eigenvalue, orig(j) represents the longitudinal initial vector eigenvalue of the matrix, tau(j) represents the longitudinal correction scaling factor of the longitudinal vector eigenvalue of the matrix, delta(i) represents the lateral displacement when merging arrays in the divide-and-conquer method, delta(j) represents the longitudinal displacement when merging arrays in the divide-and-conquer method, i represents the lateral index of the matrix, j represents the longitudinal index of the matrix, and i≠j.

[0116] That is, the embodiment of the present application provides a data processing method to quickly process the tridiagonal matrix model in the divide-and-conquer method, and splits the tridiagonal matrix model in the divide-and-conquer method into multiple data iteration processing models, and the operation symbols in each data iteration processing model are the same, for example, all are addition, or all are subtraction, or all are multiplication, or all are division, and so on.

[0117] For example, the SIMD architecture system in the embodiment of the present application has a total of m computing units, and sending a single instruction can control these computing units to perform unified addition, subtraction, multiplication, division and other operations. For the iterative operation of the n-order matrix feature vector v(i) in the first part of the dlaed3 module, the above data model to be processed can be sorted out to know:

[0118] v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) (4)

[0119] Traverse and iterate j (when i=j, let (delta(j)-delta(i)=1), split the data model to be processed again, that is, fix i and traverse j, calculate the first data processing sub-model: orig(j)+tau(j)-delta(i) multiplication, and calculate the second data processing sub-model: 1 / (delta(j)-delta(i)) multiplication (when i=j, let (delta(j)-delta(i)=1), and then multiply the two results to obtain a single vector eigenvalue v(i). The SIMD architecture system has a total of m computing units, so m single vector eigenvalues ​​v(i) can be calculated in parallel. The specific implementation method can refer to the following steps 201-203 and steps 401-step 404.

[0120] Please also see Figure 2 and Figure 3 In an optional embodiment of the present application, for the first data processing sub-model, the above step 102, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain the calculation results of each of the computing units, at least includes the following steps 201-203:

[0121] Step 201, sending each of the lateral displacements to each of the calculation units respectively, and sending the first sum of the first longitudinal initial eigenvalue and the first longitudinal correction scaling factor to each of the calculation units respectively for parallel iterative calculation to obtain a first iterative calculation result;

[0122] Step 202, sequentially sending the sum of other longitudinal initial eigenvalues ​​and corresponding longitudinal correction scaling factors to the respective computing units for parallel iterative calculations, and sequentially obtaining the iterative calculation results;

[0123] Step 203: For each of the calculation units, all iterative calculation results are integrated to obtain a first calculation result of the calculation unit; wherein the first calculation result refers to the calculation result of the first data processing sub-model.

[0124] like Figure 3 As shown, the process of the first data processing sub-model: orig(j)+tau(j)-delta(i) multiplication and parallel calculation and iterative operation can be: delta(1), delta(2), ..., delta(m) are sent to m computing units respectively, then orig(1)+tau(1) is sent to m computing units, and calculation instructions are sent, and the m computing units respectively calculate orig(1)+tau(1)-delta(1), orig(1)+tau(1)-delta(2), .... .., orig(1)+tau(1)-delta(m), retain the first iterative operation result; then send orig(2)+tau(2) to m computing units, m computing units perform multiplication operations and retain the first iterative operation result, until all iterative operation results are obtained, and all iterative operation results are multiplied to obtain the above-mentioned first calculation result, and so on, the first data processing sub-model (orig(j)+tau(j)-delta(i)) part of m single vector eigenvalues ​​v(i) can be calculated in parallel. In this way, the embodiment of the present application fully utilizes the architectural advantage of SIMD, that is, a single instruction controls multiple computing units to calculate in parallel, greatly improving computing efficiency.

[0125] Please also see Figure 4 and Figure 5 In an optional embodiment of the present application, for the second data processing sub-model, the above step 102, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain the calculation results of each of the computing units, at least includes the following steps 401-404:

[0126] Step 401, sending each of the lateral displacements to each of the calculation units respectively;

[0127] Step 402: Send the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units calculates the difference between the first longitudinal displacement and each of the lateral displacements respectively;

[0128] Step 403, sending the other longitudinal displacements to the calculation units in sequence, and the calculation units calculate the difference between the other longitudinal displacements and the lateral displacements in sequence;

[0129] Step 404: Integrate the difference values ​​obtained each time and calculate the reciprocal to obtain the second calculation result of each calculation unit.

[0130] The second calculation result refers to the calculation result of the second data processing sub-model. Figure 5 , Figure 5 The process of calculating the second calculation result of the second data processing sub-model (1 / (delta(j)-delta(i))) is as follows: first calculate the product of delta(j)-delta(i), and then find the inverse of the result. The specific process may include: sending delta(1), delta(2), ..., delta(m) to m computing units respectively, and then sending delta(1) to m computing units respectively, sending calculation instructions, and the m computing units will calculate delta(1)-delta(1), delta(1)-delta(2), ..., delta(1)-delta(m) respectively, and then sending delta(2) to m computing units, when j=i, replacing delta(j)-delta(i) with 1, performing multiplication operations on the m computing units and retaining the results, and so on, and finally taking the inverse of the multiplication results, and the second data processing sub-model (1 / (delta(j)-delta(i))) of m v(i) can be calculated in parallel. The embodiment of the present application gives full play to the architectural advantages of SIMD in this way, that is, a single instruction controls multiple computing units to calculate in parallel, thereby greatly improving computing efficiency.

[0131] In an optional embodiment of the present application, the above-mentioned multiple data processing sub-models include: the first data processing sub-model and the second data processing sub-model. Correspondingly, the above-mentioned calculation results are integrated and processed according to the preset data processing method to obtain the iterative calculation result of the data model to be processed, including the following steps:

[0132] The first calculation result of the first data processing sub-model and the second calculation result of the second data processing sub-model are multiplied to obtain the iterative calculation result.

[0133] That is, the first calculation result of the first data processing sub-model and the second calculation result of the second data processing sub-model are integrated and processed to obtain the iterative calculation result of the above-mentioned data model to be processed. The method is more simplified, and each calculation unit can be independently parallelized, which greatly improves the data processing efficiency.

[0134] Please also see Figure 6 and Figure 7In an optional embodiment of the present application, the above step 104, for each iterative operation, sends the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method to each computing unit for parallel iterative operation to obtain each array element respectively, including the following steps 601-602:

[0135] Step 601, sending the first calculation result and the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units calculates the first array element respectively;

[0136] Step 602: Send other calculation results and corresponding other longitudinal displacements to each of the calculation units in sequence, and each of the calculation units calculates other array elements respectively.

[0137] See also Figure 7 ,like Figure 7 To determine the specific process of each array element: first send v(1) and delta(1) to m computing units respectively, and calculate v(1)*(1 / (orig(1)-delta(1)+tau(1))), v(1)*(1 / (orig(2)-delta(1)+tau(2))), ..., v(1)*(1 / (orig(m)-delta(1)+tau(m))) respectively, and you can get Q(1, 1), Q(1, 2), ..., Q(1, m), and so on, you can get all the elements from the 1st column to the mth column of the array Q, and finally get all the array elements of the array Q.

[0138] In an optional embodiment of the present application, an LDPC code (Low Density Parity Check Code, LDPC code) may be generated based on the updated calculation array to establish communication between the terminal device where the SIMD architecture system is located and the satellite. The LDPC code generation method includes the following:

[0139] Constructing a check matrix to be processed of the LDPC code based on the updated calculation array, dividing the check matrix to be processed into an information bit matrix and a check bit matrix; wherein the check bit matrix is ​​a dual diagonal matrix;

[0140] Performing row transformation on the information bit matrix and the check bit matrix based on a preset row index to form a rearranged first information bit matrix and a first check bit matrix; wherein each matrix unit in the first information bit matrix is ​​a unit matrix cycle;

[0141] Performing a column transformation on the first check bit matrix based on a preset column index to form a rearranged second check bit matrix; wherein all unit matrices except one unit matrix in the second check bit matrix are unit matrix cycles;

[0142] Constructing a matrix equation for the check matrix based on the first information bit matrix and the second check bit matrix, and solving the matrix equation to obtain each target check bit value in the second check bit matrix;

[0143] A target check matrix of an LDPC code is generated based on the information bit matrix and each of the target check bit values.

[0144] In an optional embodiment of the present application, the performing row transformation on the information bit matrix and the check bit matrix based on a preset row index to form a rearranged first information bit matrix and a first check bit matrix includes:

[0145] According to the preset row index

[0146] The information bit matrix and the check bit matrix are row transformed in the manner of [0,q,2q,…(L-1)q,1,q+1,2q+1,…,(L-1)q+1,…,q-1,2q-1,3q-1,…,Lq-1] to form the rearranged first information bit matrix and the first check bit matrix; wherein L represents the number of units of the unit matrix, and q represents a matrix parameter related to the code rate and code length of the check code encoding.

[0147] In an optional embodiment of the present application, the performing column transformation on the first check bit matrix based on a preset column index to form a rearranged second check bit matrix includes:

[0148] According to the preset column index according to the column index

[0149] [0,q,2q,…(L-1)q,1,q+1,2q+1,…,(L-1)q+1,…,q-1,2q-1,3q-1,…,Lq-1] performs column transformation on the first check bit matrix to form a rearranged second check bit matrix; wherein q represents a matrix parameter related to the code rate and code length of the check code encoding.

[0150] In an optional embodiment of the present application, constructing a matrix equation for the check matrix based on the first information bit matrix and the second check bit matrix includes:

[0151]

[0152] Wherein, s represents each information bit in the first information bit matrix, p represents each check bit in the second check bit matrix, j represents the number of columns of the check matrix to be processed, n represents the number of rows of the check matrix to be processed, q represents a matrix parameter related to the code rate and code length, and different code rates in the coding protocol correspond to specific q values; Qh(n,j) (0≤n≤q-1) indicates that the identity matrix is ​​cyclically shifted right by h(n,j) positions. If h(n,j)=-1, then h(n,j) indicates an all-zero matrix.

[0153] In an optional embodiment of the present application, solving the matrix equation to obtain each target check bit value in the second check bit matrix includes:

[0154] Calculate a final check bit value of a final matrix cell in the second check bit matrix based on the matrix equation;

[0155] Calculate the first check bit value of the first matrix unit in the second check bit matrix based on the last check bit value;

[0156] Based on the bidirectional recursive method and the last check bit value and the first check bit value, other bit values ​​in the second check bit matrix are calculated to obtain the target check bit value including the first check bit value, the last check bit value and each of the other bit values.

[0157] In an optional embodiment of the present application, the generating a target check matrix of the LDPC code based on the information bit matrix and each of the target check bit values ​​includes:

[0158] Transforming each of the target check bit values ​​based on the preset column index to obtain a third check bit matrix after column restoration;

[0159] The information bit matrix and the third check bit matrix are merged to obtain the target check matrix of the LDPC code.

[0160] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0161] See also Figure 8An embodiment of the present application provides a data processing device 800, which is applied to a SIMD architecture system, wherein the SIMD architecture system includes multiple computing units; the data processing device 800 at least includes: a splitting module 810, a first computing module 820, a second computing module 830, a third computing module 840 and an updating module 850, wherein:

[0162] The splitting module 810 is used to split the data model to be processed into multiple data processing sub-models;

[0163] The first calculation module 820 is used to send each of the data processing sub-models to each of the calculation units for parallel iterative calculation to obtain calculation results of each of the calculation units;

[0164] The second calculation module 830 is used to integrate the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed;

[0165] The third calculation module 840 is used for sending the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method to each of the calculation units for parallel iterative calculation for each iterative calculation, so as to obtain each array element respectively;

[0166] The updating module 850 is used to integrate the array elements to obtain an updated calculation array.

[0167] In an optional embodiment of the present application, the data model to be processed is a tridiagonal matrix model in the divide-and-conquer method; correspondingly, the data processing sub-model is a data iterative processing model.

[0168] In an optional embodiment of the present application, the data model to be processed is:

[0169] v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i)))

[0170] Correspondingly, the plurality of data processing sub-models include at least one of the following two data processing sub-models:

[0171] The first data processing sub-model: orig(j)+tau(j)-delta(i)

[0172] Second data processing submodel: 1 / (delta(j)-delta(i))

[0173] Among them, (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, v(i) represents the single vector eigenvalue obtained by the i-th solution, orig(j) represents the vertical initial vector eigenvalue of the matrix, tau(j) represents the vertical correction scaling factor of the vector eigenvalue in the vertical direction of the matrix, delta(i) represents the horizontal displacement when merging arrays in the divide-and-conquer method, delta(j) represents the vertical displacement when merging arrays in the divide-and-conquer method, i represents the horizontal index of the matrix, j represents the vertical index of the matrix, and i≠j.

[0174] In an optional embodiment of the present application, the multiple data processing sub-models include: the first data processing sub-model and the second data processing sub-model, and the second calculation module 830 is specifically used to multiply the first calculation result of the first data processing sub-model and the second calculation result of the second data processing sub-model to obtain the iterative calculation result.

[0175] In an optional embodiment of the present application, for the first data processing sub-model, the first calculation module 820 is specifically used to send each of the lateral displacements to each of the calculation units respectively, and send the first sum of the first longitudinal initial eigenvalue and the first longitudinal correction scaling factor to each of the calculation units for parallel iterative calculations to obtain a first iterative calculation result; send other longitudinal initial eigenvalues ​​and other sums of the corresponding longitudinal correction scaling factors to each of the calculation units in turn for parallel iterative calculations to obtain each iterative calculation result in turn; for each of the calculation units, integrate all the iterative calculation results to obtain a first calculation result of the calculation unit; wherein, the first calculation result refers to the calculation result of the first data processing sub-model.

[0176] In an optional embodiment of the present application, for the second data processing sub-model, the first calculation module 820 is specifically used to send each of the lateral displacements to each of the calculation units respectively; send the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units calculates the difference between the first longitudinal displacement and each of the lateral displacements respectively; send other longitudinal displacements to each of the calculation units in turn, and each of the calculation units calculates the difference between other longitudinal displacements and each of the lateral displacements in turn; integrate the difference obtained each time and calculate the inverse to obtain the second calculation result of each of the calculation units; wherein the second calculation result refers to the calculation result of the second data processing sub-model.

[0177] In an optional embodiment of the present application, the third calculation module 840 is specifically used to send the first calculation result and the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units calculates the first array element respectively; and send other calculation results and corresponding other longitudinal displacements to each of the calculation units in turn, and each of the calculation units calculates other array elements respectively.

[0178] For the specific definition of the above-mentioned data processing device 800, please refer to the definition of the data processing method above, which will not be repeated here. Each module in the above-mentioned data processing device 800 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0179] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a data processing method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the data processing method as described above is implemented.

[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above data processing method can be implemented.

[0181] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0182] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0185] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0186] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: Applied to a SIMD architecture system, the SIMD architecture system includes a plurality of computing units; the method includes: Split the data model to be processed into multiple data processing sub-models; Sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units; Integrate and process the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed; For each iterative operation, the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method are sent to each of the calculation units for parallel iterative operation to obtain each array element respectively; Each of the array elements is integrated to obtain an updated calculation array.

2. The data processing method according to claim 1, characterized in that: The data model to be processed is a tridiagonal matrix model in the divide-and-conquer method; correspondingly, the data processing sub-model is a data iterative processing model.

3. The data processing method according to claim 2, characterized in that: The data model to be processed is: v(i)=(orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) Correspondingly, the plurality of data processing sub-models include at least one of the following two data processing sub-models: The first data processing sub-model: orig(j)+tau(j)-delta(i) Second data processing submodel: 1 / (delta(j)-delta(i)) Among them, (orig(j)+tau(j)-delta(i))*(1 / (delta(j)-delta(i))) represents the data model to be processed, v(i) represents the single vector eigenvalue obtained by the i-th solution, orig(j) represents the vertical initial vector eigenvalue of the matrix, tau(j) represents the vertical correction scaling factor of the vector eigenvalue in the vertical direction of the matrix, delta(i) represents the horizontal displacement when merging arrays in the divide-and-conquer method, delta(j) represents the vertical displacement when merging arrays in the divide-and-conquer method, i represents the horizontal index of the matrix, j represents the vertical index of the matrix, and i≠j.

4. The data processing method according to claim 3, characterized in that: The plurality of data processing sub-models, including the first data processing sub-model and the second data processing sub-model, correspondingly, the calculation results are integrated and processed according to the preset data processing method to obtain the iterative calculation result of the data model to be processed, including: The first calculation result of the first data processing sub-model and the second calculation result of the second data processing sub-model are multiplied to obtain the iterative calculation result.

5. The data processing method according to claim 3, characterized in that: For the first data processing sub-model, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units includes at least: Sending each of the lateral displacements to each of the calculation units respectively, and sending the first sum of the first longitudinal initial eigenvalue and the first longitudinal correction scaling factor to each of the calculation units respectively for parallel iterative calculation to obtain a first iterative calculation result; The sums of the other longitudinal initial eigenvalues ​​and the corresponding longitudinal correction scaling factors are sequentially sent to the calculation units for parallel iterative calculations, and the iterative calculation results are sequentially obtained; For each of the calculation units, all iterative calculation results are integrated to obtain a first calculation result of the calculation unit; wherein the first calculation result refers to the calculation result of the first data processing sub-model.

6. The data processing method according to claim 3, characterized in that: For the second data processing sub-model, sending each of the data processing sub-models to each of the computing units for parallel iterative calculations to obtain calculation results of each of the computing units includes at least: Sending each of the lateral displacements to each of the calculation units respectively; Sending the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units respectively calculating the difference between the first longitudinal displacement and each of the lateral displacements; The other longitudinal displacements are sent to the calculation units in sequence, and the calculation units calculate the difference between the other longitudinal displacements and the lateral displacements in sequence; The difference values ​​obtained each time are integrated and the inverse is calculated to obtain the second calculation result of each calculation unit; wherein the second calculation result refers to the calculation result of the second data processing sub-model.

7. The data processing method according to claim 1, characterized in that: For each iterative operation, the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method are sent to each of the computing units for parallel iterative operation to obtain each array element respectively, including: Sending the first calculation result and the first longitudinal displacement to each of the calculation units respectively, and each of the calculation units respectively calculates the first array element; The other calculation results and the corresponding other longitudinal displacements are sent to each of the calculation units in turn, and each of the calculation units calculates and obtains other array elements respectively.

8. A data processing device, characterized in that: Applied to a SIMD architecture system, the SIMD architecture system includes a plurality of computing units; the device includes: A splitting module, used for splitting the data model to be processed into multiple data processing sub-models; A first calculation module, used for sending each of the data processing sub-models to each of the calculation units for parallel iterative calculation, to obtain calculation results of each of the calculation units; A second calculation module is used to integrate and process the calculation results according to a preset data processing method to obtain an iterative calculation result of the data model to be processed; A third calculation module is used for sending the obtained iterative calculation result and the displacement amount when merging in the corresponding divide-and-conquer method to each of the calculation units for parallel iterative calculation for each iterative calculation, so as to obtain each array element respectively; The updating module is used to integrate the array elements to obtain an updated calculation array.

9. A computer device comprising: The method comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images