Mass data parallel processing method based on big data

A mass data, parallel processing technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as large correlations, and achieve the effect of improving computing speed

Inactive Publication Date: 2018-05-01
成都博睿德科技有限公司
View PDF3 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

One of the main problems to overcome in parallel computing is data dependency, and the bitwise summation operation in the accumulation algorithm makes the algorithm itself have a big dependency problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mass data parallel processing method based on big data
  • Mass data parallel processing method based on big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0012] The following and accompanying appendices illustrating the principles of the invention Figure 1 A detailed description of one or more embodiments of the invention is provided together. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details.

[0013] One aspect of the present invention provides a massive data parallel processing method based on big data. figure 1 It is a flowchart of a method for parallel processing of massive data based on big data according to an embodimen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a mass data parallel processing method based on big data. The method comprises the steps that (1) a current available process number of a parallel computing platform is acquired; (2) a cache queue used for storing carry is allocated and initialized according to the available process number; (3) region segmentation is performed on operation tasks needing summation operation according to the available process number; (4) computing tasks in all regions are solved parallelly in multiple processes, and a last carry value is saved into an corresponding item in the cache queue;and (5) unified operation is performed on a result of each region in the updated cache queue. Through the mass data parallel processing method based on the big data, on the basis of the multi-core computing platform, a distributed parallel environment is fully utilized to increase operation speed.

Description

technical field [0001] The invention relates to parallel computing, in particular to a massive data parallel processing method. Background technique [0002] The analysis and processing of big data has become an important IT capability for enterprises. Because of the large volume, fast speed, and variety of big data, and a large amount of heterogeneous and unstructured data, there are also huge difficulties in the analysis, processing and utilization of big data. In order to solve the problem of big data computing, software developers and researchers from all over the world have done a lot of research and practice. In recent years, more and more developers have begun to pay attention to the role of software engineering. In order to reduce duplication of labor, improve software quality and code reuse, many excellent big data computing libraries have emerged. The existing big data computing library only provides mathematical computing functions, and only implements serial al...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/24532G06F16/24552
Inventor 李垚霖
Owner 成都博睿德科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap