Performance optimization method and device for distributed system platform
A distributed system and optimization method technology, applied in the field of performance optimization of distributed system platforms, can solve problems such as low efficiency in adjusting the number of partitions, affecting the performance optimization of distributed systems, etc. The effect of improving adjustment efficiency
Active Publication Date: 2021-06-08
BEIJING GRIDSUM TECH CO LTD
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] However, in the case of optimizing the performance of a distributed system, when the number of partitions needs to be continuously adjusted, the above method of setting the number of partitions requires modifying the code and repackaging each time, which will result in low efficiency in adjusting the number of partitions. In turn, it affects the performance optimization of distributed systems
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0041] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0042] A performance optimization method for a distributed system platform provided by an embodiment of the present invention, such as figure 1 As shown, the method includes:
[0043] 101. Obtain a configuration file carrying configuration information about the number of partitions.
[0044] Among them, the configuration file is passed in when the distributed system platform receives the task. The specific content in the configur...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a performance optimization method and device for a distributed system platform, which relates to the field of information technology, can improve the adjustment efficiency of the number of partitions in the distributed system platform, and can improve the performance optimization effect of the distributed system. The method includes: Obtaining a configuration file carrying the configuration information of the number of partitions, the configuration file is imported when the distributed system platform receives the task; parsing the configuration file to obtain the configuration information of the number of partitions; according to the configuration information of the number of partitions, The number of partitions in the distributed system platform is updated. The invention is applicable to the performance optimization of the distributed system platform.
Description
technical field [0001] The invention relates to the field of information technology, in particular to a performance optimization method and device for a distributed system platform. Background technique [0002] In recent years, with the rapid development of information technology, distributed systems have been widely used. Distributed systems are software systems built on the network to process various assistance tasks and then integrate the results. The parallel computing performance in a distributed system is related to the number of partitions. For example, spark is a big data distributed system computing platform. It not only implements MapReduce's operator map function, reduce function and computing The number of Partition partitions determines the parallel computing capability of Spark. The more Partition partitions, the better the concurrent capability of Spark. [0003] Currently, the number of partitions is coded and set using a corresponding API (Application Prog...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
CPCH04L67/303
Inventor 严波
Owner BEIJING GRIDSUM TECH CO LTD