Spark-based distributed matrix inversion parallel operation method

A dense matrix and computing method technology, applied in complex mathematical operations, concurrent instruction execution, computing, etc., can solve the problems of large matrix, unfriendly users, fault tolerance, poor scalability, etc., achieve good fault tolerance, improve computing Efficiency, Quantity Reduction Effect

Inactive Publication Date: 2016-03-02
NANJING UNIV
View PDF4 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The two main problems addressed by the present invention are: the existing matrix scale is very large, and the traditional single computer serial opera...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Spark-based distributed matrix inversion parallel operation method
  • Spark-based distributed matrix inversion parallel operation method
  • Spark-based distributed matrix inversion parallel operation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] like figure 1 As shown, the implementation of the present invention is to convert the distributed row matrix in the distributed file system into a distributed block matrix, run the LU decomposition parallelization algorithm of the distributed block matrix, and then perform the distributed update in the decomposition result The triangular matrix and the distributed lower triangular matrix respectively run the inversion parallelization algorithm to find their inverse matrix, and then use the permutation matrix obtained by the original LU decomposition and the obtained inverse matrix of the two triangular matrices to run distributed matrix multiplication, The final result is obtained, which is the inverse matrix of the original input matrix.

[0020] The complete process of the present invention includes three parts: LU decomposition of distributed dense matrix, inversion of distributed triangular matrix (namely upper triangular matrix and lower triangular matrix as claime...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a Spark-based distributed matrix inversion parallel operation method. The method comprises the following steps: carrying out parallel LU decomposition operation on an input matrix in an iteration process; taking a distributed upper triangular matrix and a distributed lower triangular matrix obtained through the LU composition as basis so as to the inverse matrixes of the distributed upper triangular matrix and the distributed lower triangular matrix by using a recursive algorithm; and finally taking a permutation matrix and the inverse matrixes of the triangular matrixes obtained in the above two steps as basis so as to implement distributed matrix multiplication to obtain the inverse matrix of any original input matrix. According to the method, the dense matrixes with large dimensionalities can be processed, and relatively high operation efficiency as well as relatively good fault tolerance and expandability can be obtained.

Description

technical field [0001] The present invention relates to the technical field of linear algebra (LinearAlgebra) operation, in particular to a distributed parallel computing method based on the one-stop big data processing platform Spark's distributed dense matrix inversion operation. Background technique [0002] With the advent of the era of big data, the amount of data has grown explosively. The scale of data that people need to calculate and analyze is getting larger and larger, and the requirements for operating efficiency and accuracy are getting higher and higher. In many fields such as scientific computing, data mining, and machine learning, the solution to many problems can be abstracted into a series of operations centered on matrix operations. However, as an intermediate calculation step in common complex data calculation and analysis tasks, the inversion operation of large-scale matrices is a very time-consuming process. One of the main reasons is that the calculati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/16G06F9/38G06F17/30
CPCG06F17/16G06F9/3818G06F16/182
Inventor 黄宜华顾荣高兴坤
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap