Large data contour query processing method based on cyclic neural network
A recurrent neural network and big data technology, applied in the field of contour query processing for big data, can solve problems such as a lot of extra time overhead, large extra storage space, waste of storage resources, etc., to achieve short user response time, strong scalability, The effect of high profile query processing efficiency
Inactive Publication Date: 2017-06-30
TONGJI UNIV
View PDF0 Cites 4 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, we found that in the big data environment, there are at least three defects in processing contour queries by building multidimensional indexes. First, in the big data environment, multidimensional indexes need to occupy a huge amount of additional storage space, thus wasting a lot of storage resources; secondly, when the data is updated, the multidimensional index needs to be maintained and dynamically updated accordingly, which requires a lot of additional time overhead; thirdly, when the multidimensional index is built, the time complexity of the profile query is usually reduced to O(knlogn), it is not difficult to see that the complexity is nonlinear, so in the big data environment, this processing method also requires huge time overhead
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0028] In the first stage of the contour object offline learning module, the domain data object set M is generated as follows: firstly, the k-dimensional histogram is constructed by using the data distribution characteristics of the k-dimensional large data object set D in the domain, and then simulated by the k-dimensional histogram A joint distribution function F(x 1 ,x 2 ,...,x k ), then based on F(x 1 ,x 2 ,...,x k ) to generate 10,000 domain data objects s that satisfy the joint distribution function 1 [d 1 ,...,d k ],s 2 [d 1 ,...,d k ],...,s 10000 [d 1 ,...,d k ], and form a set M={s 1 [d 1 ,...,d k ],s 2 [d 1 ,...,d k ],...,s 10000 [d 1 ,...,d k ]}.
[0029] The acquisition of the contour object set SM and the non-contour object set NM on M is implemented through 4 steps: 1) For each object s in M i [d 1 ,...,d k ](1≤i≤10000), calculate the sorting operator of the object Where ln(·) is natural logarithm; 2) sort the objects in M according...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention provides a highly-efficient and highly-extensible large data contour query processing method based on the cyclic neural network, and the method is aimed at overcoming the shortcomings of the prior art. The method is summarized as follows: obtaining a contour object set on the large data through two steps of the contour object offline learning and the contour object online recognition. In the contour object offline learning phase, firstly a certain size of offline learning samples are generated based on the data distribution characteristics of the large data area, and then a cyclic neural network learning model is constructed and optimized based on the offline learning samples. In the contour object online recognition phase, for each object of the large data to be processed, the cyclic neural network learning model is used and the output value of each object model is calculated, and all contour objects on the large data are determined and output based on the model output value. The method has the advantages of being fast in speed, high in extendibility and strong in self-adapting capability, and being effectively used in the fields of Internet depth information service, intelligent transportation, e-commerce, data visualization and the like.
Description
technical field [0001] The invention relates to a contour query processing technology, in particular to a big data-oriented contour query processing technology. Background technique [0002] Skyline query processing technology is a research focus and focus of computer science in recent years, mainly because the object set of skyline query results has a wide range of applications in many fields, such as: intelligent transportation, information services, data visualization and e-commerce Wait. Given a set of k-dimensional data objects D={p 1 [d 1 ,...,d k ],p 2 [d 1 ,...,d k ],...,p n [d 1 ,...,d k ]}, where n is the number of objects, d 1 ,...,d k is the k dimensions of the object, each dimension d i (1≤i≤k) describes a feature of D, such as shelf life, price, etc., and the contour query skyQ(D) is to obtain the largest data subset SD on D that satisfies the following conditions: each object in SD will not be in all k The values in each dimension are worse than...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/245
Inventor 黄震华倪娟程久军
Owner TONGJI UNIV