Mass spectrum data read-write method

Abstract

The invention provides a mass spectrum data read-write method, which comprises a working stage from data acquisition to data write-in, and the working stage comprises the following steps: (A1) an instrument module starts acquisition; (A2) the instrument module collects original data and stores the original data into a data buffer area; meanwhile, a multi-thread technology is adopted, data in the buffer area is processed according to the collection sequence, and data blocks are generated and transmitted to a data layer; (A3) the data layer receives the data blocks, the data blocks are written in sequence, and index information of the current data block is fed back to the index buffer area; (A4) repeating the steps (A2)-(A3) until data acquisition is completed; (A5) after the data in the data buffer area is processed, generating an index block from the data in the index buffer area, and writing the index block into the data index block; (A6) recording the position of the data index block, and replacing the index value of the data index block with the position of the data index block; and (A7) finally, writing a file identifier according to data acquisition and data writing conditions. The invention has the advantages of good performance and the like.

Description

Technical Field

[0001] The present invention relates to mass spectrometry technology, and in particular to a method for reading and writing mass spectrometry data. Background Art

[0002] Mass spectrometers, also known as mass spectrometers, are instruments that separate and detect different isotopes and are widely used in environmental testing, food safety, proteomics, metabolomics, biomedicine, clinical medicine, drug detection, and other fields. As the complexity of sample matrices continues to increase, the requirements for mass spectrometer performance, such as sensitivity, mass resolution, high throughput, and detection time, are also increasing. This has led to the alarming growth in the size of mass spectrometer raw data files. For example, tandem mass spectrometers, such as liquid chromatography-ion trap mass spectrometers and liquid chromatography-triple quadrupole mass spectrometers, used for drug detection in wastewater, can analyze a single sample for up to one hour in high-speed, high-resolution mode. In this high-speed, high-resolution mode, a single raw data file of over 10GB can be generated at a time. Hundreds to hundreds of samples need to be analyzed daily, and this massive amount of data places high demands on computer performance.

[0003] Currently, the mass spectrometry community commonly uses open source data storage formats (such as mzML, mzXML, and mzData), universal data formats (.cdf, .hdf5, and .mgf), and proprietary data formats from various instrument manufacturers (such as SCIEX's .wiff and Thermo Fisher's .RAW). These formats are typically used for offline storage of mass spectrometry data and are not compatible with real-time data storage. They require real-time online acquisition before the entire file is converted to a format suitable for further analysis and processing. Furthermore, these mainstream data storage formats lack proprietary compression algorithms designed based on the inherent characteristics of mass spectrometry data, resulting in low compression rates. Furthermore, these mainstream data storage formats lack search optimizations tailored to the acquisition mode, resulting in slow data loading and search speeds that fail to meet speed requirements. These mainstream data formats are unique and focus solely on the data itself, omitting instrument parameters (such as chromatographic method settings and retention times). This information is critical for mass spectrometry data analysis and is therefore incompatible.

[0004] It can be seen that the prior art has the following deficiencies: 1. Poor performance. Currently, conventional solutions store real-time data in a buffer and write it to a designated file at the end of the acquisition. As detection time increases, this places high demands on the buffer. Furthermore, writing data to a file at the end of the acquisition places even higher demands on the CPU and disk. More importantly, the time required to write the data severely impacts the detection throughput of tandem mass spectrometry.

[0005] 2. Poor generality. Traditional solutions all specify formats in advance, and there are certain limitations in terms of personalized requirements. For example, it is impossible to store parameters not in the specified format, etc.

[0006] 3. Large file size. Traditional formats usually store data in two-dimensional coordinates (an array of mass numbers - responses), and most formats do not use compression technology.

[0007] 4. Poor query performance. When querying data in traditional formats, all data needs to be loaded into the buffer at one time, and data is searched in the buffer. Loading the entire file takes a long time, especially during batch analysis, and the efficiency is extremely low.

[0008] 5. Incompatible with simultaneous acquisition and analysis. Traditional formats store the acquired data in the buffer and write it to the file all at once after completion, usually only supporting offline analysis.

[0009] 6. Poor data security. Traditional formats store data in plain text without security protection and inspection for data, resulting in poor security. Summary of the Invention

[0010] To address the deficiencies in the above-mentioned prior art solutions, the present invention provides a method for reading and writing mass spectrometry data.

[0011] The object of the present invention is achieved through the following technical solutions: A method for reading and writing mass spectrometry data, including a data acquisition to writing working stage, and this working stage includes the following steps: (A1) The instrument module starts acquisition. According to the pre-set file name and version number, first create a file and write the data file version number. The data index area index is temporarily written as 0; according to the set acquisition method, the current mass axis, instrument parameters, monitoring data, etc., write the basic information block; according to the system settings, determine whether to fill in the information extension block; (A2) The instrument module acquires raw data and stores it in the data buffer; at the same time, using multi-threading technology, process the data in the buffer in the acquisition order, generate data blocks, and transfer them to the data layer; (A3) The data layer receives the data blocks, writes the data blocks in order, and feeds back the index information of the current data block to the index buffer; (A4) Repeat steps (A2) - (A3) until data acquisition is completed; (A5) After the data in the data buffer is processed, generate an index block from the data in the index buffer and write it into the data index block; (A6) Record the position of the data index block and replace the value of the data index area index with it; (A7) Finally, write the file identifier according to the data acquisition and data writing situations.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. High performance. By adopting multi-threading technology, data is collected and stored simultaneously, reducing the pressure on the buffer. At the same time, the file is processed by layering and chunking, reducing the pressure of writing a large file at one time, reducing the requirements for computer performance, and greatly improving the performance. Taking 10G of data volume as an example, the traditional buffer requires at least 10G of memory, and the one-time writing time is up to several minutes, bringing a bad experience to users. Through technical improvement, the buffer can meet the working conditions at the M level (usually below 5M), and the file writing time is in the millisecond level, which is more than a hundred times higher than the traditional performance.

[0013] 2. Strong versatility. Customized extended blocks are provided, and personalized extended services can be completed as long as a specified interface is inherited; a data file version number is provided for extensions of different versions.

[0014] 3. Small data file. Only response values are stored in the data, and the mass number is calculated according to the quality equation axis, reducing the traditional data volume by 50%; at the same time, block compression technology is adopted, and the compression ratio r < 0.3; the combination of the two makes the total file size equivalent to 15% of the original on the basis of lossless.

[0015] 4. High query performance. By adopting indexing technology, data blocks are accurately located, and a certain block is loaded at a time, improving the loading speed and optimizing the query speed.

[0016] 5. Compatible with simultaneous collection and analysis. A real-time data reading interface is provided to support collection and analysis simultaneously.

[0017] 6. High security. The integrity of the entire file is quickly judged by checking the file version number and file identifier. At the same time, each block has compressed and CRC32 verification after compression, further improving the security. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Referring to the accompanying drawings, the disclosure of the present invention will become more understandable. It is easy for those skilled in the art to understand that these drawings are only used to illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention. In the figures: Figure 1 is a schematic structural diagram of a mass spectrometry data reading and writing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] Figure 1The following description and the following illustrate alternative specific embodiments of the present invention to teach those skilled in the art how to implement and reproduce the present invention. Some conventional aspects have been simplified or omitted in order to teach the technical solution of the present invention. Those skilled in the art should understand that variations or substitutions derived from these specific embodiments will fall within the scope of the present invention. Those skilled in the art should understand that the following features can be combined in various ways to form multiple variations of the present invention. Thus, the present invention is not limited to the following alternative specific embodiments, but is defined only by the claims and their equivalents.

[0020] Example 1.

[0021] The mass spectrometry data reading and writing device according to Example 1 of the present invention, as Figure 1 shown, includes: Instrument module 1, buffer module 2, data module 3, analysis module 4, data buffer 101, index buffer 102, file header layer 301, information layer 302, data layer 303, file identification layer 304, file version number 30101, index block index 30102, basic information block 30201, extension block 30202, data block 30301, index block 30302, file identification 30401.

[0022] Instrument module 1 mainly refers to the instrument device side, and is mainly responsible for establishing the acquisition method, collecting the acquisition parameters (monitoring), obtaining the mass axis parameters, and collecting the raw data, etc.

[0023] Buffer module 2 refers to a temporary space opened in the memory for storing the collected raw data and the data index list. Due to the use of the technology of collecting and storing simultaneously, generally the data buffer is not larger than 10 pieces of raw data. In the full scan mode (50 - 2000 da), one piece of raw data does not exceed 80K; the index buffer increases dynamically with the acquisition time. Calculated by 8 bytes for one index, for a 1-hour acquisition method and a collection speed of 3 pieces per second, generally only 84K of memory is required.

[0024] The data module 3 refers to the module for operations such as data storage, reading, and searching. The data module adopts a hierarchical and block-based structure, which is divided into a file header layer, an information layer, a data layer, a file identifier layer, and so on. Among them, the file header layer includes the data file version number, the data index block index, etc.; the information layer includes a basic information block and an information extension block, etc. The basic information block includes the acquisition method (including the liquid phase method), the mass axis, instrument parameters, monitoring data, etc., and the information extension block provides for users' personalized extension requirements; the data layer contains all data blocks and data index blocks. The data of each data block includes at least data auxiliary information (sampling time, serial number, etc.) and data response values, etc. The data index block stores the location index of all data blocks. The data module provides two interfaces for reading real-time data and historical data. The real-time data interface meets the analysis requirements when the instrument module is in the process of acquisition, and the historical data interface meets the analysis requirements after the instrument acquisition is completed.

[0025] A block refers to the smallest unit of data with the same or similar functions. It includes at least a compression identifier, the length before compression, the length after compression, data, data verification, etc. Each block can independently complete data compression, data writing, verification, and reverse operations.

[0026] A safe, efficient, and scalable mass spectrometry data reading and writing method according to Embodiment 1 of the present invention is specifically as follows: S1: The instrument module 1 (ion trap mass spectrometer) starts acquisition and generates the data module 3 according to the preset file name (C:\drugs.gmd); S2: Write the file version number 30101 (such as 1), and write the default value of the index block index 30102 (0, which can be any value from 0 to 264 here, and the correct value will be modified after the file is written); S3: Complete the basic information block 30201 according to the preset acquisition method (a drug method, including substance information such as ketamine, methamphetamine, cocaine, etc., where the substance information includes number, name, ion pairs (taking ketamine as an example, 238-220, 238-179), etc.), acquisition parameters (important parameters such as radio frequency voltage, dynode voltage, etc.), monitoring data (vacuum degree, liquid phase flow rate, etc.), and mass axis information (y = kx + b, the voltage and mass number equation adjusted before acquisition, where y is the voltage, x is the mass number, and b is the intercept), etc.; S4: Fill in the data of the extension block 30202 according to the preset, and this block is empty; S5: The instrument module 1 (ion trap mass spectrometer) acquires the original data (including sampling number, sampling time, response values of 8000 points, etc.) and stores them in the data buffer 101 in sequence; S6: Use multi-threading technology to process the data in the data buffer 101, generate the data block 30301, write it into the data layer 303 in sequence, and store the index of the data block 30301 in the index buffer 102; S7: Repeat S5 - S6 until the scanning is completed and the data in data buffer 101 is completely processed; S8: Generate data blocks from the data in the index buffer ([100000, 120000, 140000...]) and write them into index block 30302; S9: Modify the value of index block index 30102 with the position (1010000) of index block 30302 obtained in S8; S10: Fill in file identifier 30401 according to the current data collection and writing situation (one - byte data is used to record data availability, 0 means normal, 1 means abnormal, and the representation relationship can be customized);

[0027] S11: During the data collection process, instrument module 1 (ion trap mass spectrometer) needs to analyze specified data to check if the instrument is working properly. At this time, analysis module 4 extracts an analysis request (such as the first data) from data module 3; S12: Data module 3 obtains all the current data in index buffer 102 from instrument module 1 (ion trap mass spectrometer); S13: Data module 3 opens a file (C:\drugs.gmd, read - only), reads the data (1) of file version number 30101, and determines whether it meets the conditions (consistent with the preset and satisfying the conditions); S14: Data module 3 reads the data of basic information block 30201 and extended block 30202 for participating in data analysis; S15: Use the data in index buffer 102 obtained in S12 to temporarily fill the data of the index block ([100000, 120000, 140000...]); S16: According to the specified request (the first data) of analysis module 4, obtain the index (100000) of the first data from the index block, quickly locate the position of the data block according to the index, and read data block 30301; S17: Parse the data according to the reverse operation of the block; S18: Use the number (1) in the data obtained in S17 to obtain the ion pairs (238 - 220, 238 - 179) of the corresponding substance from the collection method in basic information block 30201, the mass axis equation (y = kx + b) in basic information block 30201, and the data parsed in S17, and parse the data block into a mass spectrum diagram (including sampling time, sampling number, mass number array, response value array); S19: Feed back the mass spectrum diagram parsed in S18 to analysis module 4 for analysis.

[0028] S20: When the instrument module 1 (ion trap mass spectrometer) needs to perform qualitative / quantitative analysis after data acquisition is completed, the analysis module 4 extracts an analysis request (all data, i.e., from the 1st to the nth sheet) from the data module 3; S21: The data module 3 opens the file (C:\drugs.gmd, read-only), reads the data of file identifier 30401, first determines the integrity and validity of the current file based on the data (0) of file identifier 30401; S22: Reads the data of file version number 30101 (1) and index block index 30102 (1010000), and then determines whether it meets the requirements based on the data of file version number 30101; S23: The data module 3 reads the data of the basic information block 30201 and the extended block 30202 for use in data analysis; S24: Uses the data of index block index 30102 in S22 to quickly locate the index block 30302, and obtains the index area data according to the reverse operation of the block; S25: According to the specified request of the analysis module 4, loop from 0 to n, obtain the index from the index area, quickly locate the position of the data block and read the data block 30301; S26: Parses the data according to the reverse operation of the block; S27: Uses the number (1) in the data of S26 to obtain the ion pairs (238 - 220, 238 - 179) of the corresponding substance from the acquisition method in the basic information block 30201, the mass axis equation (y = kx + b) in the basic information block 30201, and the data parsed in S26, and parses the data block into a mass spectrum diagram (including sampling time, sampling number, mass number array, response value array); S28: Feeds back the mass spectrum diagram parsed in S27 to the analysis module 4.

[0029] Embodiment 2.

[0030] Please refer to Figure 1 As shown, the reading and writing device of this embodiment includes an instrument module 1, a buffer module 2, a data module 3, an analysis module 4, a data buffer 101, an index buffer 102, a file header layer 301, an information layer 302, a data layer 303, a file identifier layer 304, a file version number 30101, an index block index 30102, a basic information block 30201, an extended block 30202, a data block 30301, an index block 30302, and a file identifier 30401. / / The specific name corresponding to each component marked in the attached drawing should be written in the embodiment A safe, efficient, and scalable mass spectrometry data reading and writing method according to an embodiment of the present invention is specifically as follows: S1: The instrument module 1 (triple quadrupole mass spectrometer) starts data acquisition and generates data module 3 according to the preset file name (C:\sewage.gmd). S2: Write the file version number 30101 (such as 2), and write the default value of the index block index 30102 (0, which can be any value from 0 to 264 here, and the correct value will be modified after the file is written). S3: Complete the basic information block 30201 according to the preset acquisition method (a sewage method contains multiple segments, a segment contains multiple experiments, an experiment contains multiple channels (50 channels), each channel is an ion pair (238 - 220), acquisition parameters (important parameters such as RF voltage, dynode voltage, etc.), monitoring data (vacuum degree, liquid phase flow rate, etc.) and mass axis information (y = kx + b, the voltage and mass number equation adjusted before acquisition, where y is the voltage, x is the mass number, and b is the intercept), etc. S4: Fill in the data of the extension block 30202 according to the preset. This block contains the resolution tuning table (y = kx + b, the deviation value and mass number equation adjusted before acquisition, where y is the deviation value, x is the mass number, and b is the intercept), CE voltage (1000), etc. S5: The instrument module 1 (triple quadrupole mass spectrometer) acquires the original data (including sampling number, sampling time, response values of 8000 points, etc.) and stores them in the data buffer 101 in sequence. S6: Use multi - thread technology to process the data in the data buffer 101, generate data block 30301, write it into the data layer 303 in sequence, and store the index of data block 30301 in the index buffer 102. S7: Repeat S5 - S6 until the scan ends and the data in the data buffer 101 is completely processed. S8: Generate a data block from the data in the index buffer ([100000, 120000, 140000...]) and write it into the index block 30302. S9: Modify the value of the index block index 30102 with the position (1010000) of the index block 30302 in S8. S10: Fill in the file identifier 30401 according to the current data acquisition and writing situation (one - byte data is used to record the availability of the data, 0 means normal, 1 means abnormal, and the relationship can be customized).

[0031] S11: During the acquisition process, the instrument module 1 (triple quadrupole mass spectrometer) needs to analyze the specified data to check whether the instrument is working properly. At this time, the analysis module 4 extracts the analysis request (such as the first data) from the data module 3. S12: The data module 3 retrieves all the current data of the index buffer 102 from the instrument module 1 (triple quadrupole mass spectrometer). S13: The data module 3 opens a file (C:\Drugs.gmd, read-only), reads the data (1) of the file version number 30101, and determines whether it meets the conditions (consistent with the preset and satisfies the conditions). S14: The data module 3 reads the data of the basic information block 30201 and the extended block 30202 for use in data analysis. S15: Using the data of the index buffer 102 obtained in S12, temporarily fill the data of the index block ([100000, 120000, 140000…]). S16: According to the specified request (the first data) of the analysis module 4, obtain the index (100000) of the first data from the index block, quickly locate the position of the data block according to the index, and read the data block 30301. S17: Parse the data in the reverse operation of the block. S18: Using the number (1) in the data of S17, obtain the ion pairs (238 - 220, 238 - 179) of the corresponding substance from the acquisition method in the basic information block 30201, the mass axis equation (y = kx + b) in the basic information block 30201, and the data parsed in S17, and parse the data block into a mass spectrum diagram (including sampling time, sampling number, mass number array, response value array). S19: Feed the mass spectrum diagram parsed in S18 back to the analysis module 4 for analysis.

[0032] S20: When the instrument module 1 (triple quadrupole mass spectrometer) needs to perform qualitative / quantitative analysis after data acquisition is completed, the analysis module 4 extracts an analysis request (all data, i.e., the first to the nth) from the data module 3. S21: The data module 3 opens a file (C:\Drugs.gmd, read-only), reads the data of the file identifier 30401, first determines whether the current file is complete and valid according to the data (0) of the file identifier 30401. S22: Read the data (1) of the file version number 30101 and the data (1010000) of the index block index 30102, and then determine whether it meets the requirements according to the data of the file version number 30101. S23: The data module 3 reads the data of the basic information block 30201 and the extended block 30202 for use in data analysis, where the extended block 30202 contains a resolution tuning table and a CE voltage. S24: Use the data of the index block index 30102 in S22 to quickly locate the index block 30302, and obtain the index area data in the reverse operation of the block. S25: According to the specified request of the analysis module 4, loop from 0 to n, obtain the index from the index area, quickly locate the position of the data block, and read the data block 30301; S26: Parse the data according to the reverse operation of the block; S27: Using the number (1) in the data of S26, obtain the ion pairs (238 - 220, 238 - 179) of the corresponding substance from the collection method in the basic information block 30201, the mass axis equation (y = kx + b) in the basic information block 30201, and the data parsed in S26, and parse the data block into a mass spectrometry diagram (including sampling time, sampling number, mass number array, response value array); S28: Feed back the mass spectrometry diagram parsed in S27 to the analysis module 4.

Claims

1. A method for reading and writing mass spectrometry data, including the data acquisition to writing working stage, and this working stage includes the following steps: (A1) The instrument module starts acquisition. According to the pre-set file name and version number, first create a file and write the data file version number. The data index area index is temporarily written as 0. According to the set acquisition method, the current mass axis, instrument parameters, monitored data, etc., write the basic information block. According to the system setting, determine whether to fill in the information extension block; (A2) The instrument module acquires the raw data and stores it in the data buffer. At the same time, using multi-threading technology, process the data in the buffer according to the acquisition order, generate data blocks, and transfer them to the data layer; (A3) The data layer receives the data blocks, writes the data blocks in order, and feeds back the index information of the current data block to the index buffer; (A4) Repeat steps (A2)-(A3) until the data acquisition is completed; (A5) Until the data in the data buffer is processed, generate an index block from the data in the index buffer and write it into the data index block; (A6) Record the position of the data index block and replace the value of the data index area index with it; (A7) Finally, write the file identifier according to the data acquisition and data writing situations.

2. The mass spectrometry data reading and writing method according to claim 1, wherein The reading and writing method further includes the data analysis working stage during the acquisition process, and this working stage includes the following steps: (B1) Obtain the data index from the index buffer of the instrument module; (B2) Open the specified file, read the data file version number in the file header layer, and judge whether the data file version number meets the conditions according to the preset; (B3) Read the basic information block and complete the data parsing and checking according to the reverse operation of the block; (B4) Read the extension block and complete the data parsing and checking according to the reverse operation of the block. Here, it can be empty; (B5) According to the business requirements, quickly locate the specified data block according to the data index, and complete the data parsing and checking according to the reverse operation of the data block. According to the working process from the data block to the mass spectrometry data, convert the data into a mass spectrometry graph.

3. The mass spectrometry data reading and writing method according to claim 1, characterized in that The reading and writing method further includes the data analysis working stage after the acquisition is completed, and this working stage includes the following steps: (C1) Open the specified file, read the data file version number and the data index area index in the file header layer, and judge whether the data file version number meets the conditions according to the preset; (C2) Read the file identifier and judge whether the file is complete according to the preset; (C3) Read the basic information block and complete the data parsing and checking according to the reverse operation of the block; (C4) Read the extension block and complete the data parsing and checking according to the reverse operation of the block. Here, it can be empty; (C5) According to the data index area index, quickly locate the data index block, read the data index, and obtain the data index according to the reverse operation of the block; (C6) According to the business requirements, quickly locate the specified data block according to the data index, and complete the data parsing and checking according to the reverse operation of the block. According to the working process from the data block to the mass spectrometry data, convert the data into a mass spectrometry graph.

4. The mass spectrometry data reading and writing method according to claim 1, wherein, Workflow of the block: Record the compression identifier and the length of the data before compression, compress the data according to the compression identifier, record the length of the compressed data, perform a CRC32 checksum on the compressed data, obtain the checksum and write it to the data checksum.

5. The mass spectrometry data reading and writing method according to claim 1, characterized in that Workflow of the inverse operation of the block: Read the compression identifier, the length of the data before compression, and the length of the compressed data, read the data according to the length of the compressed data, and read the data checksum; Perform a CRC32 checksum on the data, compare the obtained checksum with the data checksum to check if the data is complete; According to the compression identifier, decompress the data, and compare the length before compression with the length of the decompressed data to check if the data is complete.

6. The mass spectrometry data reading and writing method according to claim 1, characterized in that Workflow from the data block to the mass spectrometry data: The data block only stores the data auxiliary information and the original data. Obtain the corresponding acquisition method according to the serial number from the acquisition methods in the basic information block, obtain the corresponding mass axis equation according to the acquisition method from the mass axis, and calculate the mass spectrometry diagram according to the mass axis equation.

Images