3 results about "Data binning" patented technology

The present specification relates to the technical field of big data processing, and provides a data binning method and device based on a big data cluster, equipment and a storage medium. The method comprises the following steps: reading a data binning configuration file to obtain user-defined information; the user-defined information comprises a bin number and an evaluation function; determining a mapping number and a reduction number of a mapping reduction task according to a node cluster state and a data volume of a feature table; in the mapping stage, each Map is caused to separate each row of data in the feature table according to columns, and an index value of each column of data in the corresponding row is determined; in the shuffle stage, a partitioner is caused to place features of a same complete column in a same partition according to the index value; in the reduction stage, each Reduce reads a column of data from a corresponding partition and calls the evaluation function to process the column of data, and the processing results of each Reduce are reduced according to the bin number. The embodiments of the present specification can improve the processing efficiency of data binning processing and reduce resource consumption.

The application relates to a driving control and data acquisition system of a digital ion trap. The application relates to the technical field of mass spectrometer control and data acquisition, and the system takes an FPGA main control unit as a core, internally has four modules of main control and timing scheduling, waveform generation, direct current control and data acquisition, is matched with a communication module, a radio frequency coupling and output interface module, a power module and an upper computer. The application binds waveform scanning parameter updating, direct current gate switching, sampling triggering and data binning operation to the same timing engine through the main control and timing scheduling module to construct a global unified clock domain and timing engine, realizes full-process synchronous linkage of driving control and data acquisition, supports configurable radio frequency driving, multi-mode and multi-channel direct current output, real-time signal processing and multiple safety protection, can eliminate alignment errors of scanning and acquisition, improves mass spectrometry precision, resolution and repeatability, and can flexibly adapt to application requirements of various ion trap structures and portable ion trap mass spectrometers.

The application relates to a data binning method, device, computer equipment and storage medium. The method comprises the following steps: acquiring business data; dividing the business data into discrete variables and continuous variables; automatically binning each continuous variable to obtain a plurality of variable binning results corresponding to the continuous variable; in response to the number of categories of the discrete variables being less than or equal to a minimum category number threshold, outputting the discrete variables; in response to there being at least one discrete variable with a category number greater than the minimum category number threshold, automatically binning each discrete variable to obtain a plurality of variable binning results corresponding to the discrete variable; according to a preset evaluation index, screening the plurality of variable binning results corresponding to each continuous variable and / or discrete variable to obtain an optimal variable binning result; and outputting the optimal variable binning result and a parameter range of a corresponding binning method. The method can reduce the system calculation pressure, and can comprehensively obtain a plurality of variable binning results of data and an optimal variable binning result.