A method for optimizing storage of gas chromatograph data
By structuring the gas chromatograph data and dynamic indexing accuracy control, the problem of imbalance in the allocation of indexing resources in the existing technology has been solved, and efficient data storage and retrieval performance has been improved.
Patent Information
- Application Number
- CN202510703183.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-29
- Publication Date
- 2025-08-08
- Estimated Expiration
- 2045-05-29
AI Technical Summary
In the existing gas chromatograph data optimization storage technology, the indexing accuracy cannot be dynamically regulated based on the criticality of the data segment in the overall structure, resulting in insufficient indexing capabilities of important data, redundant non-critical data resources, reduced compression efficiency, inaccurate decompression and restoration, and poor data retrieval performance.
The data collected by the gas chromatograph is divided into several data segments according to the preset boundary recognition rules. The structural behavior information is collected through the initial feature indexing operation, the criticality of each data segment is evaluated, and the evaluation results are classified, the index accuracy is dynamically regulated, and the index bitmap and compression strategy are generated.
Differentiated index resource allocation for different data segments is achieved, storage resource utilization is improved, core data retrieval accuracy and compression efficiency are ensured, and retrieval response capabilities and structural restoration capabilities in large-scale data storage scenarios are enhanced.
Smart Images

Figure CN120234344B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of gas chromatograph data storage, and in particular to a method for optimizing gas chromatograph data storage. Background Art
[0002] A gas chromatograph is an instrument commonly used to separate and analyze chemical substances, widely used in fields such as chemistry, pharmaceuticals, and environmental monitoring. It vaporizes a sample, allowing it to react with a stationary phase and a mobile phase to separate its components. The detector then outputs a time-dependent signal, typically recorded as data points, forming a series of curves reflecting chemical concentration changes. The data generated by a gas chromatograph is large, complex, and detailed, and grows exponentially with increasing sample types and the number of tests. To effectively utilize this data, traditional storage methods can encounter issues such as insufficient storage space, data redundancy, and slow access speeds. Especially when faced with massive amounts of data, real-time processing and efficient storage often cannot be guaranteed. Therefore, optimizing the storage of gas chromatograph data is crucial. Optimizing storage can reduce redundancy, improve storage efficiency, and accelerate data retrieval and analysis while ensuring data accuracy. This approach also better meets the demands of big data processing, improving overall instrument performance and experimental efficiency. Optimized storage methods not only conserve hard drive space but also increase data retrieval speed and processing accuracy, enabling researchers to perform data analysis more efficiently and accurately, ultimately enhancing the quality of research results and the reproducibility of experimental procedures.
[0003] Existing gas chromatograph data storage optimization technologies primarily utilize multiple steps to reduce data redundancy, improve storage efficiency, and increase data processing speed. First, during the data acquisition phase, signal preprocessing, such as denoising and data compression, reduces the storage space occupied by irrelevant data and noise, ensuring that only critical information is retained. Second, for data storage, efficient compression algorithms are employed to compress raw data into smaller files, saving storage space while preserving important experimental information. Furthermore, taking into account the temporal and structured nature of data, some optimization techniques utilize block-based storage or indexing, making subsequent data retrieval and access more efficient. Furthermore, some advanced technologies utilize cloud storage and distributed storage architectures for distributed data storage and backup, which not only improves data security but also optimizes data access speeds through load balancing, reducing storage bottlenecks. During data processing, storage optimization technologies often incorporate data analysis algorithms to effectively identify and categorize stored data, ensuring that different types of data can be quickly accessed based on priority and needs, thereby improving data utilization and experimental processing efficiency. Through the combined effect of these links, existing technologies can significantly improve the storage efficiency, access speed and processing accuracy of gas chromatograph data, meeting the needs of efficient data analysis and long-term storage.
[0004] The existing technology has the following deficiencies:
[0005] In the process of optimizing the storage of data collected by gas chromatographs, in order to support subsequent data compression and rapid retrieval, it is usually necessary to extract the structural features of each data segment and generate an indexing bitmap before storage. During this operation, if a uniform precision level is used for indexing, and the indexing strategy fails to distinguish the roles of different data segments in the overall structure, an imbalance in indexing resource allocation will occur. Specifically, when a data set contains both key segments that carry the main information and non-key segments that only serve auxiliary purposes, the system often applies the same feature extraction and indexing precision settings to all segments because it fails to identify the criticality of each segment in the data structure. This results in an even distribution of indexing resources, with the main segments unable to obtain high-precision indexing support, while the auxiliary segments are redundantly indexed. The existing gas chromatograph data optimization storage technology cannot dynamically adjust the indexing accuracy according to the criticality of each data segment in the overall data structure during the data feature indexing operation, resulting in insufficient indexing capabilities for important data and redundant resources occupied by non-critical data, which in turn leads to a series of problems such as decreased compression efficiency, inaccurate decompression and restoration, and poor data retrieval and positioning performance, which have a substantial impact on the data storage optimization goals.
[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention
[0007] The purpose of the present invention is to provide a method for optimizing storage of gas chromatograph data to solve the problems in the above-mentioned background technology.
[0008] In order to achieve the above object, the present invention provides the following technical solution: a method for optimizing and storing gas chromatograph data, comprising the following steps:
[0009] The raw data collected by the gas chromatograph is divided into several data segments according to a pre-set boundary recognition rule;
[0010] Performing an initial feature indexing operation on each data segment, collecting structural behavior information corresponding to each data segment during the initial feature indexing operation, evaluating the criticality of each data segment in the overall data structure based on the collected structural behavior information, and classifying each data segment according to the evaluation results;
[0011] According to the classification results, the indexing accuracy of each data segment is dynamically adjusted;
[0012] Generate a corresponding indexing bitmap based on the indexing accuracy of each data segment after dynamic adjustment;
[0013] The indexing bitmap of each data segment is associated with the data number, physical location and indexing accuracy corresponding to the data segment, the compression strategy is determined based on the indexing accuracy, the compression process is performed, and the compression result, indexing bitmap and associated information are written to the storage medium.
[0014] Preferably, an initial feature indexing operation is performed on each data segment, specifically: peak position information, peak distance information and change inflection point information in each data segment are extracted, and the extraction results are encoded in a structural tag manner.
[0015] Preferably, during the initial feature indexing operation, structural behavior information corresponding to each data segment is collected, the criticality of each data segment in the overall data structure is evaluated based on the collected structural behavior information, and each data segment is classified according to the evaluation result, which specifically includes the following steps:
[0016] During the initial feature indexing operation, the structural behavior information corresponding to each data segment is collected and preprocessed after collection;
[0017] Extracting structural coupling behavior information and characteristic load distribution information from the preprocessed structural behavior information, and analyzing them after extraction to generate structural coupling coefficients and characteristic load coefficients for each data segment respectively;
[0018] Based on the generated structural coupling coefficient and characteristic load coefficient of each data segment, a key evaluation index of each data segment is generated by weighted summation;
[0019] Determine the pre-set critical evaluation index threshold range, and compare it with the generated critical evaluation index of each data segment after determination. Evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results.
[0020] Preferably, the logic for obtaining the structural coupling coefficient of each data segment is as follows:
[0021] The structural coupling behavior information is extracted from the preprocessed structural behavior information, including the average spacing between all main peaks in each data segment during the initial feature indexing operation, the difference in the average main peak spacing between each data segment and the adjacent data segment, the difference in the signal slope between the first and last inflection points in each data segment, and the variance of the local fluctuation amplitude of each data segment, and are calibrated as 、 、 and , Indicates that during the initial feature indexing operation The average distance between all main peaks within a data segment, Indicates that during the initial feature indexing operation The average main peak distance difference between a data segment and the adjacent data segments, Indicates that during the initial feature indexing operation The signal slope difference between the first and last inflection points in a data segment, Indicates that during the initial feature indexing operation The local fluctuation amplitude variance of the data segment, , is a positive integer;
[0022] Calculate the average of the average spacing between all main peaks within all data segments during the initial feature indexing operation , according to the formula: ;
[0023] Calculate the structural coupling coefficient of each data segment. The specific calculation formula is as follows:
[0024] ;
[0025] Where, For the The structural coupling coefficient of each data segment.
[0026] Preferably, the logic for obtaining the characteristic load factor of each data segment is as follows:
[0027] The feature load distribution information is extracted from the preprocessed structural behavior information, including the number of unique main peaks identified in each data segment during the initial feature indexing operation, the total peak area of all main peaks, and the average number of times all main peaks appear in other data segments, and are marked as 、 and , Indicates that during the initial feature indexing operation The number of unique main peaks identified in the data segment, Indicates that during the initial feature indexing operation The total peak area of all main peaks in a data segment, Indicates that during the initial feature indexing operation The average number of times all main peaks in a data segment appear in other data segments, , is a positive integer;
[0028] Calculates the sum of the total peak areas of all major peaks in all data segments during the initial feature indexing operation , according to the formula: ;
[0029] Calculate the characteristic load factor of each data segment. The specific calculation formula is as follows:
[0030] ;
[0031] Where, For the The characteristic load factor of each data segment.
[0032] Preferably, the structural coupling coefficients of each data segment generated are and characteristic load factor , the key evaluation index of each data segment is generated by weighted summation. The specific calculation formula is as follows:
[0033] ;
[0034] Where, For the Key evaluation index for each data segment, and are the structural coupling coefficients of each data segment reciprocal and characteristic load factor The non-zero weight coefficient of .
[0035] Preferably, a preset key evaluation index threshold interval is determined , and after determination, generate the key evaluation index of each data segment Perform a comparison, evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results. The specific comparison analysis and classification are as follows:
[0036] like , the criticality of the data segment in the overall data structure is low criticality, and the data segment is divided into a low criticality data segment;
[0037] like , the criticality of the data segment in the overall data structure is medium criticality, and the data segment is classified as a medium critical data segment;
[0038] like , the criticality of the data segment in the overall data structure is high criticality, and the data segment is divided into a high criticality data segment.
[0039] Preferably, the indexing accuracy of each data segment is dynamically adjusted based on the classification results, specifically:
[0040] For high-key data segments, set the indexing accuracy to the highest level and index all structural fields, including peak sites, structural jump fields, and peak group boundaries.
[0041] For medium-critical data segments, set the indexing accuracy to the medium level, and only index the fields including the peak site index and structure field number;
[0042] For low-key data segments, set the indexing accuracy to the lowest level, and only index the fields including the start and end positions of the data segment.
[0043] Preferably, a corresponding indexing bitmap is generated based on the indexing accuracy of each data segment after dynamic regulation, specifically: according to the indexing accuracy level of each data segment, a field set corresponding to the level is selected to construct the indexing bitmap; when the indexing accuracy is the highest level, the indexing bitmap includes a peak site index field, a peak group boundary field, a structure field number field, and a structure jump position field; when the indexing accuracy is the intermediate level, the indexing bitmap includes a peak site index field and a structure field number field; when the indexing accuracy is the lowest level, the indexing bitmap only includes the data segment start position field and the end position field.
[0044] In the above technical solution, the technical effects and advantages provided by the present invention are:
[0045] 1. This paper introduces two quantitative indicators, the structural coupling coefficient and the characteristic load-bearing coefficient, combined with mathematical modeling and a weighted summation method to construct a key evaluation index for comprehensively assessing the structural stability and information contribution of each data segment within the overall data structure. Compared to the traditional method of uniformly indexing gas chromatograph data with precision, this method achieves refined judgment from structural behavior analysis and content density measurement to comprehensive scoring. It can accurately identify the differences between the structural backbone segments and redundant auxiliary segments, improving the accuracy and pertinence of data structure identification.
[0046] 2. This invention achieves differentiated configuration of indexing strategies by mapping key assessment indices into multi-level criticality classifications and dynamically adjusting the indexing accuracy of each data segment based on the classification results. Instead of processing all data using a fixed template, the system automatically determines the indexing field range and accuracy level based on the criticality of the data segment, ensuring that high-value segments receive complete indexing coverage and that low-value segments only record the necessary boundaries. This mechanism significantly improves storage resource utilization, avoids wasting high-density indexing resources on redundant data segments, and ensures the retrieval accuracy and compression and restoration quality of core segments.
[0047] 3. During the optimized compression and storage process, the present invention achieves criticality-driven compression path control by structurally associating the indexing bitmap with information such as data number, physical location, and indexing accuracy, and automatically matching compression strategy templates according to different accuracy levels. Ultimately, the compression results, indexing bitmap, and associated information are packaged and written to the storage medium, which not only improves compression efficiency but also enhances the system's retrieval response and structural restoration capabilities in multi-source sample and large-scale data storage scenarios. The overall solution balances high-performance compression with structural fidelity, and has excellent engineering practicality and system scalability. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0049] Figure 1 The figure is a flow chart of a method for optimizing and storing gas chromatograph data according to the present invention. DETAILED DESCRIPTION
[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.
[0051] The present invention provides Figure 1 A method for optimizing and storing gas chromatograph data is shown, comprising the following steps:
[0052] The raw data collected by the gas chromatograph is divided into several data segments according to a pre-set boundary recognition rule;
[0053] To automatically segment the raw data collected by the gas chromatograph, the system can scan and analyze the raw chromatographic signals through software, employing a signal feature recognition algorithm to identify points with boundary features and divide the continuous data stream into several structurally complete data segments. The specific segmentation method can be based on a sliding window algorithm, a signal gradient detection algorithm, or a rate-of-change threshold judgment method. The system first sets a continuous sampling interval as a sliding window and performs real-time analysis of the first-order derivative or differential value in the signal curve, identifying distinct peak start and end points, inflection points, plateaus, and sudden changes. When these boundary features are detected, the system considers the point as the start and end position of the data segment, automatically dividing the continuous data into structurally independent data segments. This segmentation method can be implemented in software within the data acquisition backend or preprocessing module, providing real-time and automated capabilities.
[0054] The pre-set boundary recognition rules refer to a set of judgment criteria for identifying paragraph boundaries that are set in advance based on the structural characteristics of the gas chromatograph signal before the data segmentation operation is performed. The rules mainly include the following types of parameters or thresholds: peak height change threshold, peak area mutation value, signal stability duration, inflection point number determination, baseline fluctuation intensity, signal-to-noise ratio change interval, etc. During operation, the system will segment and process the signal according to these rules. For example, when the change rate of N consecutive sampling points is lower than the set threshold and there is an inflection point structure before, the position can be judged as a paragraph boundary. Boundary recognition rules can be established through historical data statistical analysis, adaptive learning or manual setting, and can be flexibly configured in the software system parameters for structural division adaptation of different samples or different types of chromatographic data.
[0055] In the optimized storage process of gas chromatography data, dividing the continuous raw data stream into several structurally complete data segments is a prerequisite for achieving structured indexing and differentiated storage control. Since gas chromatography data has significant stage-by-stage variation characteristics, for example, the signal in one segment may stably express the characteristics of a single component, while another segment contains overlapping peaks of multiple components. Therefore, each data segment may differ in information structure, importance, and decompression utilization value. If effective division is not performed and the entire signal segment is treated as a unified structure, it will be impossible to perform targeted structure recognition, precision control, and compression strategies, resulting in wasted indexing resources, redundant storage, and inefficient subsequent retrieval. By achieving precise division through software, independent evaluation, indexing, and compression paths can be established for each subsequent data segment, achieving structural optimization and resource allocation at the smallest granularity unit level.
[0056] Performing an initial feature indexing operation on each data segment, collecting structural behavior information corresponding to each data segment during the initial feature indexing operation, evaluating the criticality of each data segment in the overall data structure based on the collected structural behavior information, and classifying each data segment according to the evaluation results;
[0057] In this embodiment, an initial feature indexing operation is performed on each data segment, specifically: peak position information, peak distance information and change inflection point information in each data segment are extracted, and the extraction results are encoded in a structural tagging manner.
[0058] Signal analysis algorithms can be used to analyze the one-dimensional curve structure of the numerical sequence of each data segment, extracting peak location information, peak spacing information, and inflection point information. First, a first-order difference or local extremum detection method is used to scan the continuous signal sequence to identify local maxima, which are potential peaks. False detection noise is then filtered using a set peak height threshold and minimum peak width to extract valid peak location information. Based on the extracted valid peak sequence, the sampling point spacing or time interval between adjacent peaks is calculated to form a peak spacing information sequence, which reflects the density of the signal structure and the rhythm of change. Next, by analyzing the changing trend of the second-order derivative of the data segment, turning points of the function's concavity and convexity are identified, thereby extracting inflection points in the signal. These operations are implemented using a sliding window function, digital filters (such as Savitzky–Golay filtering), and threshold judgment logic. These operations are performed in a segment-by-segment sliding manner within the algorithm module, and all calculations are efficiently and automatically performed within the software.
[0059] After extracting the key features of each data segment, the software will convert this information into a structural tag form for encoding to support subsequent structure recognition and indexing operations. The encoding process can be completed by constructing a feature vector. First, a unified structural tag template is defined, such as fields such as "peak starting point number", "peak ending point number", "peak spacing level", "inflection point type", and "inflection point position". The actual extracted feature values are then mapped to the template structure to form standardized structural tag entries. Each entry can be encoded as a specific data fragment, such as an integer displacement value, a Boolean state bit, or an index label, and organized in a structured data table or bitmap structure. During the encoding process, it can be represented in the form of a multidimensional array, a key-value pair structure, or a sparse matrix, and uniformly numbered or divided into blocks to make the structural features of different data segments comparable and searchable. The encoding results are ultimately used as the structural basis input for subsequent evaluation, classification, indexing, and other processing steps, and are represented within the system as logical bits or as input vectors to participate in subsequent model processing.
[0060] In this embodiment, during the initial feature indexing operation, structural behavior information corresponding to each data segment is collected, the criticality of each data segment in the overall data structure is evaluated based on the collected structural behavior information, and each data segment is classified according to the evaluation result. Specifically, the following steps are included:
[0061] During the initial feature indexing operation, the structural behavior information corresponding to each data segment is collected and preprocessed after collection;
[0062] During the initial feature indexing process, the collection of structural behavior information can be completed by real-time analysis and feature modeling of the chromatographic signals within each data segment through software. The collection methods include methods based on derivative analysis, threshold detection, and local statistics. Specifically, the system first traverses the data segment as a unit, performs first-order derivative calculations within a sliding window based on the numerical sequence of continuous sampling points, and extracts the signal change rate of each sampling point; at the same time, it identifies local maximum and minimum points, locates the start and end positions of the main peaks in the signal, inflection points, and fluctuation boundaries, and calculates the spacing and slope changes between each two main peaks; in addition, the data variance and peak area within the statistical window are used to quantify the volatility and signal density of the segment. All of the above structural behavior features are extracted during the intra-segment scanning process through automatic algorithms and uniformly organized into a set of structural behavior information containing multi-dimensional numerical values such as main peak distribution, inflection point slope, peak spacing, signal fluctuation, etc., which constitute the basic input for subsequent structural analysis and criticality assessment.
[0063] The purpose of preprocessing structural behavior information is to improve the accuracy and stability of subsequent parameter calculations, eliminate invalid or anomalous interference factors, unify data scaling, and construct a standard feature format for mathematical modeling. In software implementation, preprocessing typically includes three aspects: First, normalization is performed, normalizing various structural behavior information (such as peak spacing, slope, and fluctuation) to the maximum or mean value of all data segments to eliminate calculation bias caused by differences in numerical dimensions between segments; second, outlier detection and suppression is performed, using statistical analysis (such as the Z-score or IQR method) to identify data with anomalous peak spacing or slope changes, and interpolating or eliminating them to ensure that parameter results are not misleading due to extreme points; third, a unified vector format is implemented, encoding all numerical features into a consistent feature vector structure, which can be used as input variables for the subsequent calculation of structural coupling coefficients and characteristic load-bearing coefficients. The entire preprocessing process is completely automated by the software logic module, without manual intervention, and has high stability and engineering versatility.
[0064] Extracting structural coupling behavior information and characteristic load distribution information from the preprocessed structural behavior information, and analyzing them after extraction to generate structural coupling coefficients and characteristic load coefficients for each data segment respectively;
[0065] After preprocessing the structural behavior information, the extraction of structural coupling behavior information and characteristic load distribution information can be automatically completed by the software using specific feature mapping and numerical screening rules on the standardized data features. When extracting structural coupling behavior information, the system first calculates the difference in average main peak spacing between the previous and next data segments, as well as the difference in signal slope at the inflection points at the beginning and end of the segment based on the main peak position index and inflection point information of each data segment. It also calculates the local variance value of the signal within the segment within the sliding window. The above three dimensions constitute the structural coupling behavior information of the segment. When extracting characteristic load distribution information, the system calls the main peak marker index, counts the number of unique main peaks in the segment that do not appear in other data segments, calculates the sum of the peak areas corresponding to all main peaks, and retrieves the average frequency of occurrence of each main peak in other segments in the database to form the characteristic load distribution information of the segment. The entire extraction process is completed based on the software's built-in feature index logic and statistical analysis algorithm. All information is automatically matched and classified according to the data segment number, without the need for manual operation.
[0066] Based on the generated structural coupling coefficient and characteristic load coefficient of each data segment, a key evaluation index of each data segment is generated by weighted summation;
[0067] Determine the pre-set critical evaluation index threshold range, and compare it with the generated critical evaluation index of each data segment after determination. Evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results.
[0068] The critical assessment index threshold intervals can be determined by adaptive learning from historical sample datasets during the software modeling phase, or automatically generated through empirical statistical analysis, ensuring representative and discriminative classification boundaries. Specifically, the system first collects structural and behavioral information and calculates characteristic parameters from a large amount of historical chromatographic data that has been annotated or clearly classified, generating a complete critical assessment index distribution sequence. This sequence is then processed through cluster analysis, density estimation, or distribution function fitting, such as using Gaussian mixture models, K-means clustering, or kernel density estimation, to identify natural clustering intervals and boundary transition points within the critical assessment index distribution. Based on the clustering results or the cutoff points in the distribution curve, two critical values are automatically generated, serving as the low-to-medium criticality and medium-to-high criticality threshold boundaries, respectively, to construct a complete critical assessment index threshold interval. This process can be performed once by the software during initialization, or it can be periodically updated based on newly added data to ensure that the threshold boundaries dynamically align with the data structure characteristics.
[0069] In this embodiment, the logic for obtaining the structural coupling coefficient of each data segment is as follows:
[0070] The structural coupling behavior information is extracted from the preprocessed structural behavior information, including the average spacing between all main peaks in each data segment during the initial feature indexing operation, the difference in the average main peak spacing between each data segment and the adjacent data segment, the difference in the signal slope between the first and last inflection points in each data segment, and the variance of the local fluctuation amplitude of each data segment, and are calibrated as 、 、 and , Indicates that during the initial feature indexing operation The average distance between all main peaks within a data segment, Indicates that during the initial feature indexing operation The average main peak distance difference between a data segment and the adjacent data segments, Indicates that during the initial feature indexing operation The signal slope difference between the first and last inflection points in a data segment, Indicates that during the initial feature indexing operation The local fluctuation amplitude variance of the data segment, , is a positive integer;
[0071] During the initial feature indexing operation, the average spacing between all main peaks within each data segment, the difference in the average main peak spacing between adjacent data segments, the difference in signal slope between the first and last inflection points, and the variance of local fluctuation amplitude can all be automatically extracted and calculated based on continuous sampling data through software algorithms. First, the system scans the chromatographic signal curve within each data segment through the main peak recognition algorithm, identifies the local maximum points that meet the set peak height, peak width and symmetry conditions, and extracts the position indexes of these main peaks; then, the spacing between adjacent main peaks within each data segment is calculated and averaged to obtain the average spacing between the main peaks of the segment. ; To obtain the main peak spacing difference The system will simultaneously extract the average distance between the main peaks of the previous and next segments, calculate the difference with the current segment and normalize it to reflect the degree of mutation in the main peak distribution rhythm of the segment; at the same time, the system automatically identifies the first and last inflection points of each data segment based on the curve derivative operation, and calculates the derivative values of the two signals, and then calculates the slope difference. , to measure the overall transition strength of the paragraph structure; for the local fluctuation amplitude variance The system establishes a sliding window within the segment, squares the deviations between each sampling point and its local mean, and then averages them. This global mean is then aggregated across the entire segment to reflect the signal fluctuations and volatility within that segment. All of this data is automatically generated within the software through signal derivative analysis, extreme value detection, and local statistical functions, providing stable and repeatable calculation logic without the need for manual intervention.
[0072] A "primary peak" refers to a local maximum point with significant intensity characteristics and meeting morphological constraints, identified during the initial feature indexing process through feature extraction of the chromatographic signal curve. It possesses clear physical and structural significance. Specifically, a primary peak must simultaneously meet the following conditions: First, the peak height must be significantly above the local baseline, typically determined by multiplying the local window mean by a set factor. Second, the peak width must be within a preset valid range, meaning the horizontal distance between the left and right half-peaks must be neither too narrow (to exclude high-frequency noise) nor too wide (to exclude platform drift). Third, the peak shape must exhibit a distinct rise-peak-fall structure, with a positive first-to-negative trend in the first-order derivative, and possess basic symmetry or be approximated by a Gaussian function to a single peak. Fourth, the peak must form an independent extreme value within the current data segment and cannot extend across segments or adhere to boundaries. Primary peak identification is accomplished through software-based derivative analysis, extreme value search, and morphological screening of continuously sampled signals. It serves as the fundamental feature unit for subsequent extraction of key metrics such as primary peak spacing, primary peak area, and characteristic distribution. The main peak refers to a local maximum point located within each data segment during the initial feature indexing operation, with a peak height greater than a set multiple of the local baseline mean, a peak width within the effective threshold range, and a monotonically increasing and then monotonically decreasing shape.
[0073] Calculate the average of the average spacing between all main peaks within all data segments during the initial feature indexing operation , according to the formula: ;
[0074] Calculate the structural coupling coefficient of each data segment. The specific calculation formula is as follows:
[0075] ;
[0076] Where, For the The structural coupling coefficient of each data segment.
[0077] This calculation method is used for the structural coupling coefficient of each data segment in order to comprehensively evaluate the degree of dispersion of the data segment in terms of structural continuity, transition stability and internal volatility, and to quantify the structural change characteristics of multiple dimensions into a single scoring indicator. It represents the relative change amplitude of the data segment and the adjacent data segments in the main peak distribution rhythm. It is added by 1 to ensure that the value range is non-zero, and then squared to amplify the impact of the rhythm mutation. It is used to reflect the change intensity of the start and end positions of the internal structure of the data segment. The square operation enhances the recognition sensitivity of the structural mutation segment. The metric represents the intensity of microscopic fluctuations within a segment, and direct accumulation is used to reflect the negative impact of local noise or signal instability on structural coupling. The overall value is wrapped in a natural logarithmic function to compress nonlinear growth and prevent high-variability data from biasing subsequent evaluation coefficients in weighted synthesis, thereby ensuring stable gradient differentiation and consistent physical interpretation of the calculated results. This formula design ensures that each structural deviation factor is accurately captured without causing imbalance and amplification in the overall indicator, reflecting the comprehensive expression of coupling strength under multi-source heterogeneous structural perturbations.
[0078] In the overall data structure, Structural coupling coefficient of data segment The smaller the value, the closer the data segment is to its adjacent data segments in terms of structural rhythm, transition slope, and internal fluctuation, showing stronger structural continuity and coupling stability. It is usually located on the main path of the data structure and has a connecting function. Therefore, it has more value to retain in structural analysis and data compression and can be evaluated as a segment with a higher degree of criticality. On the contrary, if Larger values indicate a significant abrupt change in the main peak distribution rhythm between the segment and the preceding and following data segments, dramatic changes in the structural start and end points, or frequent internal signal fluctuations, demonstrating strong structural discontinuity and marginalization. These segments are often local anomalies, turning points, or information redundancy, and are less important to the overall structure, resulting in a correspondingly lower criticality. Therefore, numerical evaluation of the structural coupling coefficient effectively reflects the segment's embeddedness and supporting role within the structural hierarchy, providing a precise basis for the classification and assessment of criticality.
[0079] In this embodiment, the logic for obtaining the characteristic load factor of each data segment is as follows:
[0080] The feature load distribution information is extracted from the preprocessed structural behavior information, including the number of unique main peaks identified in each data segment during the initial feature indexing operation, the total peak area of all main peaks, and the average number of times all main peaks appear in other data segments, and are marked as 、 and , Indicates that during the initial feature indexing operation The number of unique main peaks identified in the data segment, Indicates that during the initial feature indexing operation The total peak area of all main peaks in a data segment, Indicates that during the initial feature indexing operation The average number of times all main peaks in a data segment appear in other data segments, , is a positive integer;
[0081] During the initial feature indexing operation, the number of unique main peaks identified in each data segment, the total peak area of all main peaks, and the average number of times all main peaks appear in other data segments can be automatically extracted and analyzed by the software. The specific implementation method is as follows: First, the system detects the main peak in each data segment through the peak recognition algorithm. The main peak must meet the standard structure of peak height higher than the local baseline, peak width in the valid range, and shape with a rising-peak-falling structure, and reflect the typical maximum value characteristics in the derivative change; the system will classify the position index, intensity, and peak type characteristics of all main peaks into the main peak feature library. Subsequently, for the data item "number of unique main peaks", the system will compare the main peak set in each data segment with the main peak set of all other data segments one by one. If a main peak does not have a similar peak in other data segments (the matching threshold is set according to the position difference, peak type similarity, and peak height relative error), then the main peak is regarded as a unique main peak, and the system counts its number as the number of unique main peaks in the segment. For the data item "Total peak area of all main peaks", the system uses the starting and ending points of each main peak as the boundary, uses the integration method to calculate the curve area under the main peak, and then sums the areas of all main peaks to obtain the total peak area of the data segment. , which reflects the overall signal strength contribution of the segment. The "average number of times all main peaks appear in other data segments" is calculated by counting the frequency of each main peak in the data segment appearing in all other data segments. The matching criteria include peak position difference, morphological overlap, etc. The average frequency of all main peaks appearing in other segments is the average number of times the segment appears. , reflecting the scarcity of this information. The above three types of data are all implemented in the software based on the main peak matching and feature comparison algorithms. They are highly automated and unambiguous, and serve as the basic data source for the subsequent calculation of the feature load coefficient.
[0082] Calculates the sum of the total peak areas of all major peaks in all data segments during the initial feature indexing operation , according to the formula: ;
[0083] Calculate the characteristic load factor of each data segment. The specific calculation formula is as follows:
[0084] ;
[0085] Where, For the The characteristic load factor of each data segment.
[0086] The core purpose of using this formula is to integrate the quantitative performance of a data segment in terms of feature strength, feature scarcity, and structural uniqueness into a single score value to evaluate its value in data compression and information retention. It indicates the ratio of the main peak area of the data segment to the total main peak area, reflecting its contribution to the overall energy or signal intensity; the second item A nonlinear amplification function representing the scarcity of the main peak, The smaller the value, the less likely the feature of this segment is repeated in other segments. The larger the index value, the more effective it is in highlighting the importance of rare segments. The addition of 1 prevents the denominator from being zero and weakens the inflation effect of extreme values. The third item The number of unique main peaks in this segment is then enhanced twice and then logarithmically compressed to balance the order of magnitude difference and avoid the score bias caused by too many main peaks. The three methods are combined to consider both the total amount of characteristic signals and their irreplaceability and uniqueness, so that the final result is The value can fully reflect the comprehensive carrying capacity of the data segment at the content information level, and support accurate judgment of the criticality.
[0087] In the overall data structure, Characteristic load factor of data segment The larger the value, the more important the data segment is in terms of content information, and thus it should be given a higher priority in the criticality assessment. A large value indicates that the data segment not only carries a high proportion of signal intensity among all segments (as reflected by the percentage of total peak area), but also contains multiple main peak features that are rarely or even uniquely present in other data segments (represented by the main peak scarcity index). Furthermore, its structural content is highly exclusive and irreplaceable (reflected by the enhanced logarithm of the number of unique main peaks). These factors collectively indicate that the data segment carries rich, unique, and recognizable critical information. Compressing or ignoring this information would severely impact the complete restoration of the data and the accuracy of subsequent retrieval. Therefore, data segments with higher feature load factors are more likely to be classified as "highly critical" in criticality assessments, making them essential for retention and high-precision indexing in optimized storage strategies.
[0088] In this embodiment, the structural coupling coefficients of each data segment are generated based on and characteristic load factor , the key evaluation index of each data segment is generated by weighted summation. The specific calculation formula is as follows:
[0089] ;
[0090] Where, For the Key evaluation index for each data segment, and are the structural coupling coefficients of each data segment reciprocal and characteristic load factor The non-zero weight coefficient of .
[0091] Generate key evaluation indices for each data segment In the process, the system uses software logic to calculate the structural coupling coefficient obtained in the early stage. and characteristic load factor Perform joint weighted calculation to achieve comprehensive evaluation of structural features and content value. Take the reciprocal form , which is used to reflect the positive contribution of the structural coupling strength. That is, the more continuous the structure and the stronger the coupling, the larger the value of this item, indicating that the structural criticality is higher. Then retain The original form is a direct reflection of the paragraph's content carrying capacity, indicating the comprehensive value of the paragraph in terms of feature strength, scarcity, and information density. In order to balance the weight influence of structure and content, the system pre-sets two weight coefficients. and ,in For controlling structural factors ( ) in the overall assessment, To control the characteristic factors ( ), both of which are non-zero real numbers and satisfy the normalization condition The two weight coefficients can be adjusted according to different application scenarios: if you focus more on structural continuity, you can set ; If you emphasize feature information density more, set Finally, through The key evaluation index calculated by the formula serves as a unified quantitative indicator to measure the criticality of each data segment, providing an accurate basis for subsequent classification processing.
[0092] In this embodiment, the preset key evaluation index threshold interval is determined , and after determination, generate the key evaluation index of each data segment Perform a comparison, evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results. The specific comparison analysis and classification are as follows:
[0093] like , the criticality of the data segment in the overall data structure is low criticality, and the data segment is divided into a low criticality data segment;
[0094] This indicates that the data segment performs poorly across both structural coupling and content-carrying capacity. It exhibits neither significant structural continuity nor valuable feature information, often representing structural edge segments, noise interference segments, or data redundancy. These data segments do not play a core role in connecting or expressing features within the overall data structure. High-precision indexing or retaining the complete signal would waste storage resources. Therefore, during storage optimization, low-precision indexing or skipped indexing can be employed, retaining only boundary locations, summary information, or pointer indexes. This minimizes data overhead while maintaining the most basic structural positioning functionality, effectively improving overall compression ratios and retrieval efficiency.
[0095] like , the criticality of the data segment in the overall data structure is medium criticality, and the data segment is classified as a medium critical data segment;
[0096] This situation indicates that it has a certain contribution to structural coupling or content carrying capacity, but is not sufficient to be identified as a core structural segment or feature trunk segment. It usually appears as an auxiliary connecting segment, information transition segment, or feature carrying segment with medium confidence. This type of data segment may provide contextual support in specific query, restoration, or partial decoding scenarios. It has a certain degree of compressibility but cannot be completely discarded. Therefore, it is advisable to adopt a medium-precision strategy during indexing and compression processing, retaining the main structural fields, significant feature sites, or peak group indexes to ensure their availability in retrieval coverage and restoration reasoning, while avoiding resource redundancy caused by high-intensity encoding.
[0097] like , the criticality of the data segment in the overall data structure is high criticality, and the data segment is divided into a high criticality data segment.
[0098] This indicates that this data segment exhibits good structural continuity and features dense, unique, and rare content. It not only serves as a backbone connection but also carries high-value characteristic signals, making it a core, key segment within the overall data structure. This type of data segment often determines the accuracy of post-compression structural restoration and the hit rate of retrieval results, making it a key target for information retention. Therefore, during optimized storage, a high-precision indexing strategy should be implemented to extract and record all structural features, main peak information, positional information, and auxiliary feature bitmaps. This ensures their maximum potential for subsequent high-precision retrieval and reconstruction, while also preventing any information loss or accidental deletion from adversely impacting system integrity.
[0099] According to the classification results, the indexing accuracy of each data segment is dynamically adjusted;
[0100] In this embodiment, the indexing accuracy of each data segment is dynamically adjusted based on the classification results, specifically:
[0101] For high-key data segments, set the indexing accuracy to the highest level and index all structural fields, including peak sites, structural jump fields, and peak group boundaries.
[0102] For data segments classified as high-key, the system uses the software to call the highest indexing template, activates all available structural fields, and performs a field-by-field traversal indexing when performing indexing precision control. Specifically, this involves extracting and recording the positional indexes of all major peaks within the segment to construct a peak site index field; identifying and annotating the start and end boundaries of all peak groups to form a peak group boundary field; analyzing the sequence of structural fields within the data segment (such as chromatographic logical groupings and sample channel labels) to generate structural field numbers; and analyzing the logical jump sequence between structural fields to construct a structural jump position field. All of these fields are accurately recorded as part of the high-precision indexing. This is because high-key segments typically carry the backbone information and express important features. Their structural information and feature density determine whether they can be restored with high fidelity after compression. Therefore, they must be fully covered and processed without omission during the indexing phase.
[0103] For medium-critical data segments, set the indexing accuracy to the medium level, and only index the fields including the peak site index and structure field number;
[0104] For medium-key data segments, the system uses a medium-key indexing template to control the indexing accuracy, limits the indexing range through software calls to the field selector, and only enables the extraction and recording of specific fields. The specific implementation is: indexing the position index field of the main peak to support peak-level retrieval and positioning; at the same time, indexing the structure field number to ensure the basic embedding logic of the segment in the structure restoration process. The remaining fields, such as structural jump positions, peak group boundaries, etc., are excluded from processing by the system at this stage to reduce the data processing burden. The implementation basis of this strategy is that although medium-key segments do not constitute the core of information, they still play an auxiliary role in structural connection or feature completion. Therefore, retaining their key identification fields helps to restore the integrity of the compressed information, but does not need to be processed as comprehensively as high-key segments, achieving a balance between processing efficiency and information retention.
[0105] For low-key data segments, set the indexing accuracy to the lowest level, and only index the fields including the start and end positions of the data segment.
[0106] For low-key data segments, the system implements a minimized indexing strategy. By calling the lowest-precision template, it turns off the extraction logic of all structure and feature fields and only retains the boundary identifier of the data segment. The specific method is: the system sets a boundary marker value at the starting sampling point and the ending sampling point of the segment to form an indexing field, which is only used to record the physical position of the segment in the data stream. High-cost fields such as main peaks, structure fields, and feature indexes are no longer extracted. The core reason for this simplified processing method is that low-key segments often only carry repetitive backgrounds, low-confidence features, or noise fragments. Preserving their structural features in their entirety does not significantly improve the compression effect or retrieval performance, but will waste computing resources and storage space. Therefore, this strategy achieves ultimate data compression and processing efficiency optimization while ensuring the integrity of the basic structure.
[0107] Generate a corresponding indexing bitmap based on the indexing accuracy of each data segment after dynamic adjustment;
[0108] In this embodiment, a corresponding indexing bitmap is generated based on the indexing accuracy of each data segment after dynamic adjustment. Specifically, according to the indexing accuracy level of each data segment, a field set corresponding to the level is selected to construct the indexing bitmap; when the indexing accuracy is the highest level, the indexing bitmap includes a peak site index field, a peak group boundary field, a structure field number field, and a structure jump position field; when the indexing accuracy is the intermediate level, the indexing bitmap includes a peak site index field and a structure field number field; when the indexing accuracy is the lowest level, the indexing bitmap only includes the data segment start position field and the end position field.
[0109] When generating an indexing bitmap corresponding to each data segment's indexing precision, the system uses a software module to pre-set multiple indexing field configuration templates. The system automatically selects the appropriate template for bitmap construction based on the segment's indexing precision level, thus achieving precision-driven structured indexing generation. Specifically, after the indexing precision is determined, the system first identifies the segment's precision level (e.g., highest, intermediate, or lowest) and then loads the corresponding field template, which contains the field type and structure definition to be represented in the indexing bitmap. For data segments with the highest precision, the software fully enables the peak index field (used to indicate the position index of the main peak in the original signal), the peak group boundary field (used to locate the start and end points of the peak group), the structure field number field (which encodes the logical structural unit in the data segment), and the structure jump position field (which marks the jump structure relationship between segments or fields), and performs bitmap encoding and assembly according to the order of field definition. For data segments with intermediate precision, only the peak index field and the structure field number field are loaded for bitmap construction, and the remaining fields are skipped to control complexity. For data segments with the lowest precision, the system only extracts the start and end sampling point indexes of the data segment to construct the minimum range of bitmap information. By generating bitmaps by mapping field sets by level, the software layer can precisely control the indexing range and density of each data segment, avoiding the unified processing of redundant fields and the resulting excessive storage overhead, while retaining the necessary information for data segments of different criticality during structure retrieval and compression and restoration, ensuring a balance between indexing efficiency and data integrity.
[0110] The indexing bitmap of each data segment is associated with the data number, physical location and indexing accuracy corresponding to the data segment, the compression strategy is determined based on the indexing accuracy, the compression process is performed, and the compression result, indexing bitmap and associated information are written to the storage medium.
[0111] In the process of associating each data segment's indexing bitmap with its corresponding data number, physical location, and indexing precision, the system uses software to construct a unified data mapping structure for organizing and indexing the indexing information of all data segments. Specifically, after generating the indexing bitmap for each data segment, the system immediately obtains the data segment's number identifier within the sampling sequence, the starting and ending sampling point locations (i.e., physical locations), and the indexing precision level used for the segment. This information is then encapsulated into a set of associated metadata. The system then maps and binds the indexing bitmap to this metadata to form a structured data segment index unit, which is stored in a unified indexing mapping table. This mapping table can be organized as a hash table or multi-level index to facilitate subsequent fast retrieval and segment-level location and precision restoration during decompression. This ensures that each indexing bitmap not only has independent structural significance but also establishes a clear correspondence with the original data structure, providing fundamental support for data management, reconstruction, and verification, and preventing information misalignment or precision confusion.
[0112] After completing the binding of indexing and metadata, the system automatically matches the corresponding compression strategy template based on the recorded indexing accuracy level, and performs differentiated compression processing on each data segment. The software system presets several compression strategies, each of which controls the compression granularity, redundant data deletion rules, feature retention range, etc. according to the indexing accuracy level. After the compression process is completed, the system packages the compressed data content, the corresponding indexing bitmap, and its bound number, location and accuracy information together, and writes them to the designated storage medium (such as a database, file system or distributed storage node) in a unified data block format. This structured, hierarchical compression writing method not only achieves the coordination of information compression rate and data integrity, but also significantly improves the efficiency of subsequent on-demand decompression and feature positioning. It is a key link in optimizing the response speed of the storage system and reducing storage resource usage.
[0113] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.
[0114] The above embodiments can be implemented in whole or in part via software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. A computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are fully or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means (e.g., infrared, wireless, microwave, etc.). A computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.
[0115] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0118] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0120] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.
Claims
1. A method for optimizing and storing gas chromatograph data, characterized in that: The specific steps include: The raw data collected by the gas chromatograph is divided into several data segments according to a pre-set boundary recognition rule; Performing an initial feature indexing operation on each data segment, collecting structural behavior information corresponding to each data segment during the initial feature indexing operation, evaluating the criticality of each data segment in the overall data structure based on the collected structural behavior information, and classifying each data segment according to the evaluation results; Determine the pre-set threshold range of key evaluation indicators , and after determination, generate the key evaluation index of each data segment Perform a comparison, evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results. The specific comparison analysis and classification are as follows: like , the criticality of the data segment in the overall data structure is low criticality, and the data segment is divided into a low criticality data segment; like , the criticality of the data segment in the overall data structure is medium criticality, and the data segment is classified as a medium critical data segment; like , the criticality of the data segment in the overall data structure is high criticality, and the data segment is divided into a high criticality data segment; According to the classification results, the indexing accuracy of each data segment is dynamically adjusted; Specifically: For high-key data segments, set the indexing accuracy to the highest level and index all structural fields, including peak sites, structural jump fields, and peak group boundaries. For medium-critical data segments, set the indexing accuracy to the medium level, and only index the fields including the peak site index and structure field number; For low-key data segments, set the indexing accuracy to the lowest level, and only index the fields including the start and end positions of the data segment; Generate a corresponding indexing bitmap based on the indexing accuracy of each data segment after dynamic adjustment; The indexing bitmap of each data segment is associated with the data number, physical location and indexing accuracy corresponding to the data segment, the compression strategy is determined based on the indexing accuracy, the compression process is performed, and the compression result, indexing bitmap and associated information are written to the storage medium.
2. The method for optimizing and storing gas chromatograph data according to claim 1, wherein: An initial feature indexing operation is performed on each data segment, specifically: peak position information, peak spacing information and change inflection point information in each data segment are extracted, and the extracted results are encoded in a structural tagging manner.
3. The method for optimizing and storing gas chromatograph data according to claim 2, wherein: During the initial feature indexing operation, the structural behavior information corresponding to each data segment is collected. Based on the collected structural behavior information, the criticality of each data segment in the overall data structure is evaluated. The data segments are classified according to the evaluation results. Specifically, the following steps are included: During the initial feature indexing operation, the structural behavior information corresponding to each data segment is collected and preprocessed after collection; Extracting structural coupling behavior information and characteristic load distribution information from the preprocessed structural behavior information, and analyzing them after extraction to generate structural coupling coefficients and characteristic load coefficients for each data segment respectively; Based on the generated structural coupling coefficient and characteristic load coefficient of each data segment, a key evaluation index of each data segment is generated by weighted summation; Determine the pre-set critical evaluation index threshold range, and compare it with the generated critical evaluation index of each data segment after determination. Evaluate the criticality of each data segment in the overall data structure based on the comparison results, and classify each data segment based on the evaluation results.
4. The method for optimizing and storing gas chromatograph data according to claim 3, wherein: The logic for obtaining the structural coupling coefficient of each data segment is as follows: The structural coupling behavior information is extracted from the preprocessed structural behavior information, including the average spacing between all main peaks in each data segment during the initial feature indexing operation, the difference in the average main peak spacing between each data segment and the adjacent data segment, the difference in the signal slope between the first and last inflection points in each data segment, and the variance of the local fluctuation amplitude of each data segment, and are calibrated as 、 、 and , Indicates that during the initial feature indexing operation The average distance between all main peaks within a data segment, Indicates that during the initial feature indexing operation The average main peak distance difference between a data segment and the adjacent data segments, Indicates that during the initial feature indexing operation The signal slope difference between the first and last inflection points in a data segment, Indicates that during the initial feature indexing operation The local fluctuation amplitude variance of the data segment, , is a positive integer; Calculate the average of the average spacing between all main peaks within all data segments during the initial feature indexing operation , according to the formula: ; Calculate the structural coupling coefficient of each data segment. The specific calculation formula is as follows: ; Where, For the The structural coupling coefficient of each data segment.
5. The method for optimizing and storing gas chromatograph data according to claim 4, wherein: The logic for obtaining the characteristic load factor of each data segment is as follows: The feature load distribution information is extracted from the preprocessed structural behavior information, including the number of unique main peaks identified in each data segment during the initial feature indexing operation, the total peak area of all main peaks, and the average number of times all main peaks appear in other data segments, and are marked as 、 and , Indicates that during the initial feature indexing operation The number of unique main peaks identified in the data segment, Indicates that during the initial feature indexing operation The total peak area of all main peaks in a data segment, Indicates that during the initial feature indexing operation The average number of times all main peaks in a data segment appear in other data segments, , is a positive integer; Calculates the sum of the total peak areas of all major peaks in all data segments during the initial feature indexing operation , according to the formula: ; Calculate the characteristic load factor of each data segment. The specific calculation formula is as follows: ; Where, For the The characteristic load factor of each data segment.
6. The method for optimizing and storing gas chromatograph data according to claim 5, characterized in that: Based on the generated structural coupling coefficients of each data segment and characteristic load factor , the key evaluation index of each data segment is generated by weighted summation. The specific calculation formula is as follows: ; Where, For the Key evaluation index for each data segment, and are the structural coupling coefficients of each data segment reciprocal and characteristic load factor The non-zero weight coefficient of .
7. The method for optimizing and storing gas chromatograph data according to claim 6, wherein: A corresponding indexing bitmap is generated based on the indexing accuracy of each data segment after dynamic regulation. Specifically, according to the indexing accuracy level of each data segment, a field set corresponding to the level is selected to construct the indexing bitmap; when the indexing accuracy is the highest level, the indexing bitmap includes a peak site index field, a peak group boundary field, a structure field number field, and a structure jump position field; when the indexing accuracy is the intermediate level, the indexing bitmap includes a peak site index field and a structure field number field; when the indexing accuracy is the lowest level, the indexing bitmap only includes the data segment start position field and the end position field.
Citation Information
Patent Citations
Gas chromatograph data optimization storage method and system
CN117785818A
Chromatographic peak detection method and device, computer equipment, storage medium and computer program product
CN119413939A