Data processing method and system
A data processing system and data processing technology, applied in the field of big data, can solve the problems of occupying server memory and cpu resources, and computing tasks cannot be carried out according to the expected progress, so as to achieve the effect of independent storage resources.
Active Publication Date: 2019-04-26
CHENGDU SEFON SOFTWARE CO LTD
View PDF12 Cites 3 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In this case, spark computing, hadoop mapreduce, and hive computing will compete for memory resources, and other services will also occupy the memory and cpu resources of the server itself, which will cause computing tasks to fail to progress as expected
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all of them. The components of the embodiments of the application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.
[0043]It should b...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention provides a data processing method and system. The method comprises the steps that a computing device sends a target data table identifier and a target data feature in areceived data acquisition request to a query device; The query device queries the data table identifier of the data table in the metadatabase and the corresponding computing device identifier according to the stored data table identifier; inquiring to obtain a target computing device identifier corresponding to the target data table identifier, and sending the target data table identifier and thetarget data feature to a target computing device corresponding to the target computing device identifier to obtain target position information through the target computing device; And the computing device obtains the target data from the corresponding target storage device according to the target position information sent by the query device, and sends a processing result obtained after processing the target data according to the data obtaining request to the requesting device. Therefore, by separating the computing resources from the storage resources, it can be guaranteed that the computingtasks are carried out according to the normal progress.
Description
technical field [0001] The present application relates to the technical field of big data, in particular, to a data processing method and system. Background technique [0002] In the existing big data computing and storage structure, the same cluster is used for computing and storage at the same time, that is to say, computing resources and storage resources share a set of equipment. In this case, Spark computing, Hadoop MapReduce, and Hive computing will compete for memory resources, and other services will also occupy the memory and CPU resources of the server itself, which will cause computing tasks to fail to progress as expected. Contents of the invention [0003] In order to overcome the above-mentioned deficiencies in the prior art, the purpose of the embodiments of the present application is to provide a data processing method and system, which can make computing resources independent and storage resources independent, thereby making full use of device resources an...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F9/50
CPCG06F9/5016G06F9/5027G06F9/5061
Inventor 侯良伟王纯斌杨尧张永飞
Owner CHENGDU SEFON SOFTWARE CO LTD