A combined compression method with dynamic decision maker for mass spectrometry data

Abstract

The application discloses a combination compression method with a dynamic decision maker for mass spectrum data, which comprises the following steps: S1, randomly selecting multiple spectrum data from each mass spectrum file to be tested, extracting several dimension data, saving the data as a double-precision floating-point array, and converting the double-precision floating-point array into an integer array by using integer transformation coding; S2, performing combination compression test on the integer array; S3, the array compressed by different combination compressors enters a scoring system of the dynamic decision maker, and the performance of each combination of the compressors is evaluated; and S4, the scores of all different combination compressors meeting the above conditions are calculated and sorted, and the combination compressor with the optimal score is predicted to be selected as the best combination compressor. The application has the advantages that the predicted combination is consistent with the best combination of the compressors in all permutation and combination compression tests of the actual data set. Meanwhile, the algorithm created by the application can adopt different compression strategies for different dimension data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more particularly to a combined compression method for mass spectrometry data with a dynamic decision maker. Background Technology

[0002] In recent years, mass spectrometers, as instruments that ionize substances and separate and measure them based on their mass-to-charge ratio (m / z), have been widely used in various fields, such as biomedicine, petrochemicals, and food chemistry. Their broad applicability has driven the development and optimization of mass spectrometers in terms of accuracy and acquisition modes, accompanied by an explosive growth in the amount of mass spectrometry data. Mass spectrometry data can be divided into binary bytecode data and metadata. Binary data is the most important part of the mass spectrometry file, consisting of mass-to-charge ratio (m / z), ion intensity, and ion mobility. However, the manufacturer's binary bytecode files obtained directly from the mass spectrometer are difficult to read and can only be opened on specific vendor software (running on specific operating systems such as Windows), which is not free. Therefore, to facilitate data dissemination, standards organizations have introduced open formats, such as mzML, to convert manufacturer files into more extensible and applicable formats, allowing them to be used on any platform. However, mzML files are often several times, or even tens of times, larger than manufacturer files. Therefore, developing an effective compression algorithm is crucial in the field of mass spectrometry.

[0003] In the field of mass spectrometry compression, various compression algorithms have been tried, such as MS-Numpress1, MassComp2, ZDPD3, and StackZDPD4. MS-Numpress is a group of compressors that provide controlled precision algorithms, employing different compression strategies for m / z and ion intensity in mass spectrometry data. MassComp represents a lossless compressor for mass-to-charge ratio data. The ZDPD compression algorithm (a combined compression strategy using an integer compressor and a general-purpose compressor) has been tested and shows superior performance compared to MS-Numpress and MassComp in terms of compression ratio, compression time, and decompression time.3 The ZDPD algorithm first converts the double-precision raw data into a pre-set integer array of a specific precision. Taking advantage of the proximity of adjacent values ​​in mass spectrometry data, it retains the first integer bit completely, and only retains the difference (delta) between that integer bit and the previous bit. This difference is then converted into binary bytes, compressed and stored using the PforDelta algorithm library, and finally compressed using Zlib (a general-purpose compressor). Compared to compression using only the general-purpose compressor (Zlib), ZDPD-compressed files have a smaller compressed size (49% smaller than Zlib) and faster decompression speed (3 times faster than Zlib). StackZDPD, on the other hand, stacks and reassembles the mass-to-charge ratio arrays from multiple spectra based on ZDPD, and then applies ZDPD's combined compressor algorithm for compression, resulting in significant compression performance for specific files.

[0004] While the ZDPD and StackZDPD algorithms demonstrate that adding a binary packer (Zlib) before the general-purpose compressor (Zlib) can significantly improve the compression efficiency of m / z data in mass spectrometry files, testing for ion intensity and ion mobility dimensions remains lacking. Furthermore, due to the diverse mass spectrometry acquisition modes and equipment, the compression efficiency of ZDPD exhibits specificity across different datasets. Simultaneously, the compression effects of various combinations of general-purpose and binary packers remain unknown. Therefore, researchers cannot determine the optimal compressor combination type before testing the entire dataset with combined compressors. Given the numerous permutations of the two compressors and the fact that mass spectrometry datasets often range in size from several GB to tens of GB, performing a complete compressor test before compressing each dataset is time-consuming and labor-intensive.

[0005] In the current framework of combined compressors, users often can only combine two default compressors, such as Binary Packing in ZDPD plus Zlib, and apply them to all types of mass spectrometry datasets. However, the different acquisition modes or instruments used for each dataset lead to differences in results under different combined compressors. Users can only select the most suitable combination for a specific dataset after trying all possible combinations of compressors. Since some mass spectrometry data files have reached tens of gigabytes in size, this work will consume a lot of time and effort for analysts, often taking weeks to months. At the same time, because different dimensions of mass spectrometry data (mass-to-charge ratio, ion intensity, and ion mobility, etc.) have different characteristics, the ZDPD combined compressor currently only supports mass-to-charge ratio compression, but other dimensions of data are equally important. Summary of the Invention

[0006] In order to predict the most suitable compressor combination for a given dataset, this invention proposes a combined compression method with a dynamic decision-maker for mass spectrometry data. The specific technical solution is as follows: A combined compression method with a dynamic decision maker for mass spectrometry data includes the following steps: S1. Randomly select multiple spectral data from each mass spectrometry file that needs to be tested, extract data from several dimensions, save them as double-precision floating-point arrays, and use integer transformation encoding to convert them into integer arrays. S2. Perform a combined compression test on integer arrays; S3. The arrays compressed by different combinations of compressors will enter the scoring system of the dynamic decision-maker to evaluate the performance of each compressor combination. S4. Calculate and sort the scores of all different combined compressors that meet the above conditions, and the combined compressor with the best score will be predicted and selected as the best combined compressor for use.

[0007] Specifically, step S3 is as follows: S31. First, aggregate the multidimensional data of multiple spectra that need to be predicted, then compress and decompress using all compressor combinations, and finally calculate the final compressed file size (S), compression time (D), and decompression time (C), where n represents the number of all combination types available in the system. (1) S32. Using formula (2), the maximum value max(T) and minimum value min(T) of different evaluation criteria in the dataset are used. The evaluation criteria include the size of the compressed file, the compression time, and the decompression time. The feature value Tk is scaled to the interval [0, 1] and a maximum-minimum normalization transformation is performed to obtain the normalized value Norm(T). (2) S33. After normalization, we obtain three new arrays, as follows: (3) S34. Calculate the normalized average compressed file size mean(S'), and set this average value mean(S') as the baseline value, specifying the combined compressor for all final predictions. The value needs to be lower than the baseline value; then, the normalized values ​​of the multiple dimensions are integrated using formula (4) to obtain the final score of each combined compressor. (4).

[0008] Specifically, it also includes step S5, which involves using Aird, a computation-oriented mass spectrometry data format, as the final conversion format, using JSON to store controllable vocabulary metadata, and aggregating and sorting logically related spectra.

[0009] The advantages of this invention are: (1) The predicted combination is consistent with the optimal compressor combination for the actual dataset under all permutation and combination compression tests. Furthermore, the algorithm created in this invention can adopt different compression strategies for data of different dimensions.

[0010] (2) Compared to ZDPD, which only uses a combined compressor (an integer compressor paired with a general-purpose compressor) to optimize the compression of mass-to-charge ratio data, ComboComp extends this to the compression of ion intensity and ion mobility data. Therefore, ComboComp can save more memory and more effectively improve the efficiency of mass spectrometry files for computation and sharing.

[0011] (3) Since different datasets have different compression effects when compressed using different combinations of compressors, and ZDPD only provides a default compressor combination (Binary Packing with Zlib) which cannot cope with a wide variety of datasets, ComboComp has added more compressor combination options after systematic testing and evaluation. This gives ComboComp better scalability and adaptability when facing the compression needs of different datasets.

[0012] (4) Neither ZDPD nor other combined compressors can predict the most suitable combined compression strategy for a specific mass spectrometry file before fully testing the entire file. Therefore, ComboComp, after adding more selectable compressors, has built an algorithm that dynamically predicts the best combined compressor and a multi-dimensional scoring system to recommend the most suitable compressor combination for each mass spectrometry file. In practical applications, this can save users a lot of testing time. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the ComboComp combined compressor process.

[0014] Figure 2 This is a schematic diagram illustrating the principle of the ComboComp algorithm.

[0015] Figure 3 A table showing basic information about the dataset used in the compression test experiment.

[0016] Figure 4 The chart shows a comparison of ComboComp's file size with ZDPD and the original manufacturer's file size.

[0017] Figure 5 This is a comparison chart of compression times between ComboComp and ZDPD.

[0018] Figure 6 This is a comparison chart of the decompression times of ComboComp and ZDPD. Detailed Implementation

[0019] like Figure 1 As shown, a combined compression method with a dynamic decision maker for mass spectrometry data includes the following steps: S1. Randomly select multiple spectra from each mass spectrometry file to be tested, extract several dimensions of data, save them as double-precision floating-point arrays, and convert them into integer arrays using integer transformation encoding. These dimensions include mass-to-charge ratio, ion intensity, and ion mobility. Since the spectral distribution characteristics within a single mass spectrometry file are similar, the results of this method are consistent with those obtained from testing the entire mass spectrometry file. Specifically, in the integer transformation encoding step, the mass-to-charge ratio double-precision floating-point array is converted into an integer array using differential encoding and integer transformation encoding; the ion intensity double-precision floating-point array is converted into an integer array using integer transformation encoding and logarithmic encoding; and the ion mobility double-precision floating-point array is converted into an integer array by mapping to integers.

[0020] S2. Perform combined compression tests on integer arrays; each integer array is compressed using different combinations of integer compressors and byte-based compressors. The integer compressors include any one of binary packing and variable byte compressors, and the byte-based compressors include any one of zlib, Brotli, Snappy, and Zstd.

[0021] S3. The arrays compressed by different combinations of compressors will enter the scoring system of the dynamic decision-maker (ComboComp) to evaluate the performance of each compressor combination; such as Figure 2 As shown, the specific steps are as follows: S31. First, aggregate the multidimensional data of multiple spectra that need to be predicted, then compress and decompress using all compressor combinations, and finally calculate the final compressed file size (S), compression time (D), and decompression time (C), where n represents the number of all combination types available in the system. (1) S32. Using formula (2), the maximum value max(T) and minimum value min(T) of different evaluation criteria in the dataset are used. The evaluation criteria include the size of the compressed file, the compression time, and the decompression time. The feature value Tk is scaled to the interval [0, 1] and a maximum-minimum normalization transformation is performed to obtain the normalized value Norm(T). (2) S33. After normalization, we obtain three new arrays, as follows: (3) S34. To avoid excessively low compression ratios, we calculated the normalized average compressed file size mean(S'). We set this mean(S') as a baseline, stipulating that the S' of all final predicted combined compressors must be lower than the baseline mean(S'). Then, we integrated the multi-dimensional normalized values ​​using formula (4) to obtain the final score for each combined compressor. (4) S4. Calculate and sort the scores of all different combined compressors that meet the above conditions, and the combined compressor with the best score will be predicted and selected as the best combined compressor for use.

[0022] S5. Aird, a computationally oriented mass spectrometry data format, is used as the final conversion format to avoid the compressed file being read, written, and used for computation. This format uses JSON for the storage of controllable vocabulary metadata and aggregates and sorts logically related spectra, making data reading faster and facilitating data exchange and computation.

[0023] To verify the method of this application, such as Figure 3 As shown, ComboComp was applied to 11 mass spectrometry datasets from different acquisition modes, instruments, and methods to perform random sampling compressor prediction experiments. The predicted combined compressor and ZDPD were used to perform compression tests on the above 11 datasets, and all tests were run on the same test computer.

[0024] Experimental results: such as Figure 4-6 As shown, compared to ZDPD, ComboComp reduces the compressed file size by an average of 12.9% and increases the decompression speed by an average of 27.1%, while maintaining a similar average compression time. In computational applications, a file is often compressed once but needs to be decompressed multiple times; therefore, the advantages in compression ratio and decompression time make ComboComp more efficient for everyday use.

[0025] The core algorithm described above was tested on 11 mass spectrometry datasets from different acquisition modes, instrument manufacturers, and omics types. The prediction results were compared with those obtained after full file compression and those obtained after ZDPD compression. Ultimately, the results obtained by the ComboComp core prediction algorithm were consistent with the actual optimal results and outperformed ZDPD.

[0026] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A combined compression method with a dynamic decision maker for mass spectrometry data, characterized in that, Includes the following steps: S1. Randomly select multiple spectra from each mass spectrometry file to be tested, extract data from several dimensions, save them as double-precision floating-point arrays, and convert them into integer arrays using integer transformation encoding. The data from several dimensions include mass-to-charge ratio, ion intensity, and ion mobility. In the integer transformation encoding step, the mass-to-charge ratio double-precision floating-point array is converted into an integer array through differential encoding and integer transformation encoding, the ion intensity double-precision floating-point array is converted into an integer array through integer transformation encoding and logarithmic encoding, and the ion mobility double-precision floating-point array is converted into an integer array by mapping to integers. S2. Perform combined compression tests on integer arrays; each integer array is compressed and tested on different combinations of integer compressors and byte general-purpose compressors; the integer compressors include any one of binarypacking and variable byte, and the byte general-purpose compressors include any one of zlib, Brotli, Snappy, and Zstd; S3. The arrays compressed by different combinations of compressors will enter the scoring system of the dynamic decision-maker to evaluate the performance of each compressor combination. S4. Calculate and sort the scores of all different combined compressors that meet the above conditions. The combined compressor with the best score will be predicted and selected as the best combined compressor for use. Step S3 is as follows: S31. First, aggregate the multidimensional data of multiple spectra that need to be predicted, then compress and decompress using all compressor combinations, and finally calculate the final compressed file size (S), compression time (D), and decompression time (C), where n represents the number of all combination types available in the system. （1） S32. Using formula (2), the maximum value max(T) and minimum value min(T) of different evaluation criteria in the dataset are used. The evaluation criteria include the size of the compressed file, the compression time, and the decompression time. The feature value Tk is scaled to the interval [0, 1] and a maximum-minimum normalization transformation is performed to obtain the normalized value Norm(T). （2） S33. After normalization, we obtain three new arrays, as follows: （3） S34. Calculate the normalized average compressed file size mean(S'), and set this average value mean(S') as the baseline value, specifying the combined compressor for all final predictions. It needs to be lower than the baseline value; Then, the normalized values ​​from multiple dimensions are integrated using formula (4) to obtain the final score for each combined compressor. （4）。 2. The combined compression method with a dynamic decision maker for mass spectrometry data according to claim 1, characterized in that, It also includes step S5, which specifically involves: using Aird, a computation-oriented mass spectrometry data format, as the final conversion format, using JSON to store controllable vocabulary metadata, and aggregating and sorting logically related spectra.

Images