The invention discloses a sparse matrix data storage method based on a ground power unit (GPU). The method comprises the following steps of: 1), sequencing the line length array length [] according to ascending order; 2), classifying the array length [] into four sections of [0, 8), [8, 16), [16, 32), [32, +infinity) according to the number of every line of non-zero element; respectively combining the 32nd, 16th, 8th, 4th lines in every section; 3], zeroizing the line in every data section and performing the line filling operation on every data section, wherein the element of the filled line is zero completely; 4], generating three one-dimensional arrays of cval [], ccol_ind [], crow_ptr [] of the SC-CSR format. In the method of the invention, the line length change amplitude of every line is reduced via segment treatment, thereby reducing the load unbalance between the thread bunch and the thread block; the adjacent lines are staggered and combined to avoid the resource waste of the thread bunch calculation when the non-zero element is less than 32, and to improve the efficiency of joint access of the CUDA display memory and decrease the step of calculating kernel and reducing lines, and therefore obviously improving the calculating performance of the vector multiplication of the sparse matrix.