Method and device for determining quantiles of data

Abstract

The embodiment of the invention discloses a method and device for determining quantiles of data. The method includes the following steps that fitting is conducted on training data selected from target data, and a probability density function p(x) corresponding to the training data is obtained; by means of the probability density function p(x), a fitting distribution function F(x) corresponding to the training data and an inverse function F-1(x) of the fitting distribution function F(x) are calculated, wherein the fitting distribution function F(x) is a nonlinear function; for fractiles contained in a preset fractile sequence P, fitting quantiles corresponding to the fractiles are calculated by means of the inverse function F-1(x), and the fitting quantiles are stored in a fitting quantile sequence B; a target data sequence D with to-be-calculated quantiles is obtained; according the fitting distribution function F(x), the inverse function F-1(x) and the fitting quantile sequence B, for the target data sequence D, the current quantiles corresponding to the fractiles are determined respectively. By means of the method and the device, errors of the determined quantiles are reduced.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method and device for determining quantiles of data. Background technique [0002] In the current era of big data, more and more data streams need to be further processed to obtain more useful information. Quantile determination of data streams is one of the important processing methods, which has a wide range of applications in the fields of computer and finance. Quantiles refer to the numerical points that divide the probability distribution range of a random variable into several equal parts. Commonly used are medians, quartiles, and percentiles. By determining its series of quantiles, the cumulative distribution function of the data stream can be observed intuitively, and then its statistical properties can be analyzed. [0003] The existing quantile determination method does not need to analyze the probability distribution model of the data stream, but extracts a...

AI Technical Summary

Problems solved by technology

[0004] However, in practical applications, the data stream often presents a heavy-tailed distribution, that is, an uneven distribution.

In this case, due to the use of the default linear function of the uniform distribution of the data stream, the determined quantile error will be large, and the statistical characteristics of the data stream cannot be accurately reflected.

Images