A patient data-oriented SPC multi-algorithm overall optimization and verification integrated method and system

Abstract

The application discloses a patient data-oriented SPC multi-algorithm overall optimization and verification integrated method and system. The method comprises the following steps: performing sample size self-adaptive supplement on an original patient training data set, so as to obtain a supplemented data set; performing data standardization and distribution optimization on the supplemented data set, so as to obtain an optimized data set; performing reproducible bias injection on the optimized data set, so as to obtain a training data set with reproducible bias; performing parameter search and performance scoring on an MA algorithm, an MQ algorithm and a WEWMA algorithm respectively by using the training data set with reproducible bias in a parallel mode; and obtaining a matched algorithm according to guardrail constraint information and two-stage screening. Through guardrail constraint and order optimization, the FPR is ensured to be within an upper limit while the MNPed is controlled, and the method is more consistent with clinical treatment than single weighting.

Description

Technical Field

[0001] This application relates to the field of algorithm evaluation technology, specifically to an integrated method and system for SPC multi-algorithm overall optimization and verification oriented towards patient data. Background Technology

[0002] PBRTQC has advantages over traditional IQC in terms of high frequency, low cost, and practical sensitivity, but the main difficulties in its implementation are:

[0003] 1. High parameter dependence: The data distribution varies greatly among different analytes / centers. If the parameters are not set properly, frequent false alarms or detection delays are likely to occur.

[0004] 2. Lack of integrated engineering tools: Existing work is mostly methodological research or single algorithm scripts, lacking a comprehensive solution that integrates multiple algorithms, can simulate reproducible errors, automatically adjust parameters, and generate and deploy workpieces with one click;

[0005] 3. Insufficient reproducibility and cross-center transferability: Error injection is mostly done using temporary scripts and random locations; preprocessing is inconsistent; and cross-center comparison and verification are difficult.

[0006] 4. Multi-objective conflicts lack clinical semantic optimization criteria: There are inherent conflicts between sensitivity, FPR, and MNPed, and there is a lack of interpretable optimization and guardrail mechanisms that take into account "fewer false alarms and faster detection".

[0007] Relevant literature and tools (such as the publicly available PBRTQC support scripts / frameworks by Bietenbeck et al.) provide implementation ideas for single algorithms or a small number of algorithms, but they generally have the following problems:

[0008] The engineering capability for coordinated scheduling and consensus triggering of multiple algorithms has not been developed.

[0009] The number of continuous alarm points was not systematically introduced as an optimization variable, nor was a collaborative optimization relationship established between the system and parameters such as window width;

[0010] Error simulations are mostly single injections that are not reproducible or have poor reproducibility, lacking parameterization of "segment-interval-amplitude-direction" and cross-center reuse;

[0011] Data augmentation often neglects time series structure and LIS rhythm;

[0012] Multi-objective optimization often only performs weighted averages, lacking interpretable optimization processes with clinical safeguards (such as FPR upper limit / MNPed upper limit) and order constraints;

[0013] The lack of ability to automatically generate deployable control limits and reporting packages results in high implementation costs. Summary of the Invention

[0014] Definitions:

[0015] 1. PBRTQC: Real-time quality control based on patient data. Continuous monitoring using routine patient outcomes, rather than quality control materials, to identify systematic or random errors.

[0016] 2. SPC: Statistical Process Control Framework.

[0017] 3. MA / MQ / EWMA: Three classic SPC algorithms: moving average, moving quantile, and exponentially weighted moving average.

[0018] 4. FPR: False positive rate.

[0019] 5. Sensitivity: sensitivity.

[0020] 6. MNPed: Median number of patients before error was detected, measuring detection delay.

[0021] 7. ME_Score: The comprehensive score used in this invention for automatic trade-offs and selection among multiple objectives (sensitivity, FPR, MNPed).

[0022] 8. Continuous Alarm Points (CP): The number of consecutive positive points required to trigger an alarm, used to constrain instantaneous fluctuations and stabilize the triggering mechanism.

[0023] 9. Timestamp-Preserving Upsampling (TPU): An upsampling strategy that supplements the sample size and extends the timestamp by replicating and amplifying the data with slight perturbations without disrupting the time order.

[0024] 10. Reproducible Bias Injection Engine (DSBG, Deterministic Synthetic Bias Generator): Given a random seed, it reproducibly injects positive / negative biases across centers in a manner where all elements of "segment length - interval length - amplitude - direction" are controllable.

[0025] 11. SACO (SPC Algorithmic Cluster Orchestrator): A control layer that performs unified scheduling of multiple algorithms (MA / MQ / EWMA), including mechanisms for algorithm selection / triggering.

[0026] 12. Guardrail Constraints: During the optimization and deployment phase, hard constraints are set on FPR, MNPed, etc., with upper limits / thresholds to ensure that "alarms are not too frequent and detection is not too late".

[0027] 13. Robust Truncation & Replacement Scheme (RTRS): Extreme values ​​are not removed but replaced with quantile thresholds, stabilizing control limit estimation without sacrificing sample size.

[0028] The purpose of this invention is to provide an integrated method for SPC multi-algorithm overall optimization and verification oriented towards patient data, which at least solves one of the above-mentioned technical problems.

[0029] One aspect of the present invention provides an integrated method for SPC multi-algorithm overall optimization and validation oriented towards patient data, the integrated method for SPC multi-algorithm overall optimization and validation oriented towards patient data includes:

[0030] Obtain the original patient training dataset;

[0031] The original patient training dataset is adaptively padded with additional samples to obtain the padded dataset.

[0032] The completed dataset is then standardized and its distribution optimized to obtain an optimized dataset.

[0033] Reproducible bias injection is performed on the optimized dataset to obtain a training dataset with reproducible bias.

[0034] Set guardrail constraint information during the training process;

[0035] Using the training dataset with reproducible bias, the MA algorithm, MQ algorithm, and WEWMA algorithm were performed in parallel for parameter search and performance scoring.

[0036] The matching algorithm is obtained based on the guardrail constraint information and the two-stage filtering.

[0037] Optionally, the step of adaptively padding the original patient training dataset to obtain the padded dataset includes:

[0038] Determine if the data in the current dataset is below a preset threshold; if so, then...

[0039] The detection timestamp field of all valid records is extracted from the original patient training dataset. The frequency of each timestamp is statistically analyzed using a time series analysis algorithm to generate an hour-detection frequency correspondence table.

[0040] The hour-test frequency correspondence table is used to identify consecutive hour intervals with higher test frequencies than other time periods, and these intervals are used as the actual clinical test rhythm periods.

[0041] Based on the actual clinical detection rhythm period, the rhythm period is evenly divided into several continuous time segments according to a preset fixed number;

[0042] The time interval between any two adjacent time segments is obtained. Starting from the start time of the actual clinical detection rhythm period, the timestamps of each time segment are generated sequentially according to this time interval to obtain the time segment set.

[0043] Valid detection records are randomly selected from the original patient training dataset, and perturbation amplitude amplification is performed on each selected original record to generate perturbation amplified records.

[0044] Each perturbation amplification record corresponds to an unused timestamp in the time segment set, thus forming supplementary data; the supplementary data and the original patient training dataset together constitute the completed dataset.

[0045] Optionally, the step of standardizing and optimizing the distribution of the completed dataset to obtain an optimized dataset includes:

[0046] The abnormal records in the optimized dataset are filtered out. During the filtering process, the timestamp field in the abnormal records is retained to maintain the temporal structure. After the filtering is completed, the detection values ​​of the remaining records are extracted into a sequence of values ​​to be processed.

[0047] The Box-Cox transform function is used to transform the numerical sequence to be processed, thereby generating the Box-Cox transformed numerical sequence.

[0048] The numerical sequence after the Box-Cox transformation is optimized to obtain the optimized dataset.

[0049] Optionally, optimizing the numerical sequence after the Box-Cox transformation to obtain an optimized dataset includes:

[0050] A parameter space for coarse grid search is defined, which includes the range of the power transformation factor α and the range of the translation parameter β. All combinations of α and β are generated as coarse grid candidate parameter pairs. For each candidate parameter pair, the Box-Cox transformed numerical sequence is transformed according to the formula z=(y+β)α to form the coarse optimized numerical sequence.

[0051] Plot the coarse-optimized numerical sequence and the standard QQ plot, and calculate the Pearson correlation coefficient of the points in the QQ plot. After traversing all coarse grid candidate parameter pairs, the system selects the top 10 parameter pairs with the highest QQ correlation coefficients to form a coarse-screened candidate parameter set.

[0052] A set of fine grid candidate parameter pairs is generated based on the coarse screening candidate parameter set, wherein the set of fine grid candidate parameter pairs includes at least one fine grid candidate parameter pair.

[0053] For each candidate parameter pair of the fine mesh, generate the corresponding refined and optimized numerical sequence;

[0054] Based on the various optimized numerical sequences, the single parameter pair with the highest QQ correlation coefficient was selected as the optimal parameter pair for TDO;

[0055] The optimized dataset is generated based on the optimal parameters of TDO.

[0056] Optionally, the step of injecting reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias includes:

[0057] Set the core parameters for bias;

[0058] Reproducible bias injection is performed on the optimized dataset based on the bias core parameters, thereby generating a biased numerical sequence.

[0059] Add an error flag and a segment flag to each record in the biased numerical sequence to obtain a biased numerical sequence with flags.

[0060] The biased numerical sequence with identifier bits is combined with timestamps to obtain a training dataset with reproducible bias.

[0061] Optionally, the setting of guardrail constraint information during the training process includes:

[0062] Initialize the adjustable parameter space for MA, MQ, and EWMA algorithms, and set the value range for each parameter;

[0063] Set a hard threshold for guardrail constraints.

[0064] Optionally, the step of using the training dataset with reproducible bias to perform parameter search and performance scoring for the MA algorithm, MQ algorithm, and WEWMA algorithm in parallel includes:

[0065] Separate processes were assigned to the MA, MQ, and WEWMA algorithms, and the following processing was performed on each algorithm:

[0066] The algorithm is run according to different parameter combinations, and the performance index of each parameter combination is calculated and the guardrail constraint is verified.

[0067] After all parameter combinations of the MA algorithm, MQ algorithm, and WEWMA algorithm have been searched, the total score PUMCH_Score of the effective combinations of each type of algorithm is calculated.

[0068] Optionally, the algorithm for obtaining the matched result based on guardrail constraint information and two-stage filtering includes:

[0069] The results of all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm are filtered out to remove parameter combinations that exceed the guardrail threshold, thereby obtaining the coarse-screened combination for each algorithm.

[0070] The scores for each parameter combination in the coarse screening combination are sorted according to the total score PUMCH_Score, and the first preset number of parameter combinations are retained.

[0071] The parameter combination with the highest sensitivity is selected from the preset number of parameter combinations as the optimal parameter combination, and the algorithm corresponding to the optimal parameter combination is selected as the matching algorithm.

[0072] Optionally, before performing parameter search and performance scoring on the MA, MQ, and WEWMA algorithms in parallel using the training dataset with reproducible bias, the integrated SPC multi-algorithm optimization and validation method for patient data further includes:

[0073] Extract two-dimensional features from the training dataset with reproducible bias;

[0074] Construct a two-dimensional feature map based on the two-dimensional features;

[0075] Get the preset matching rules;

[0076] Based on the preset matching rules, an algorithm pre-matching list is constructed using the two-dimensional feature map.

[0077] The parallel parameter search and performance scoring of the MA, MQ, and WEWMA algorithms using the training dataset with reproducible bias includes:

[0078] Using the training dataset with reproducible bias and the algorithm pre-matching list, the MA algorithm, MQ algorithm and WEWMA algorithm are performed in parallel for parameter search and performance scoring.

[0079] This application also provides an integrated system for SPC multi-algorithm overall optimization and validation oriented towards patient data, the integrated system for SPC multi-algorithm overall optimization and validation oriented towards patient data includes:

[0080] The original patient training dataset acquisition module is used to acquire the original patient training dataset.

[0081] The supplemented dataset acquisition module is used to adaptively supplement the original patient training dataset to obtain the supplemented dataset.

[0082] The optimized dataset acquisition module is used to perform data standardization and distribution optimization on the supplemented dataset to obtain the optimized dataset.

[0083] A training dataset acquisition module with reproducible bias is used to inject reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias.

[0084] A guardrail constraint information acquisition module, which is used to set guardrail constraint information during the training process;

[0085] The training module is used to perform parameter search and performance scoring of the MA algorithm, MQ algorithm and WEWMA algorithm in parallel using the training dataset with reproducible bias.

[0086] The final algorithm acquisition module is used to obtain the matching algorithm based on the guardrail constraint information and the two-stage screening.

[0087] The integrated SPC multi-algorithm overall optimization and validation method for patient data proposed in this application has the following advantages:

[0088] By using guardrail constraints and order optimization, FPR can simultaneously control MNPed within the upper limit, which is more clinically appropriate than single weighting.

[0089] For skewed / heavy-tailed data, the system automatically favors MQ / EWMA, and for near-normal data, it enhances MA / EWMA, reducing the need for manual parameter tuning. Attached Figure Description

[0090] Figure 1 This is a flowchart illustrating an embodiment of the SPC multi-algorithm integrated optimization and verification method for patient data according to this application.

[0091] Figure 2 This is a schematic diagram of adjustable parameters in the algorithm parameter space according to an embodiment of this application.

[0092] Figure 3 This is a schematic diagram of six indicators in one embodiment of this application.

[0093] Figure 4 This is a schematic diagram of the raw CA125 data distribution in one embodiment of this application.

[0094] Figure 5This is a schematic diagram of a near-normal distribution after Box-Cox transformation in one embodiment of this application.

[0095] Figure 6 This is a schematic diagram showing the final result distribution change after truncation and cleaning in one embodiment of this application.

[0096] Figure 7 The algorithm (using EWMA) demonstrates the detection performance of CA125 on offset segments (red) and normal segments (blue) under simulated biases of different magnitudes in the training set.

[0097] Figure 8 The algorithm (using EWMA) demonstrates the detection performance of CA125 on offset segments (red) and normal segments (blue) under simulated biases of different magnitudes in the test set.

[0098] Figure 9 The paper presents a comparison of the sensitivity performance of CA125 for three PBRTQC algorithms (EWMA, MA, MQ) on the training and test sets for different types of simulated bias.

[0099] Figure 10 The study demonstrates the performance of CA125 on three PBRTQC algorithms (EWMA, MA, and MQ) for different simulated bias types on the training and test sets.

[0100] Figure 11 The performance of CA125 for three PBRTQC algorithms (EWMA, MA, MQ) is compared on the training and test sets for different types of simulated bias and MNPed (detection latency). Detailed Implementation

[0101] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0102] like Figure 1 The integrated SPC multi-algorithm optimization and validation method for patient data, as shown, includes:

[0103] Obtain the original patient training dataset;

[0104] The original patient training dataset is adaptively padded with additional samples to obtain the padded dataset.

[0105] The completed dataset is then standardized and its distribution optimized to obtain an optimized dataset.

[0106] Reproducible bias injection is performed on the optimized dataset to obtain a training dataset with reproducible bias.

[0107] Set guardrail constraint information during the training process;

[0108] Using the training dataset with reproducible bias, the MA algorithm, MQ algorithm, and WEWMA algorithm were performed in parallel for parameter search and performance scoring.

[0109] The matching algorithm is obtained based on the guardrail constraint information and the two-stage filtering.

[0110] In this embodiment, the step of adaptively padding the original patient training dataset to obtain the padded dataset includes:

[0111] Determine if the number of data points in the current dataset is below a preset threshold (e.g., when the number of valid input samples is detected to be below a set threshold, defaulting to 8000). If so, then...

[0112] The detection timestamp field of all valid records is extracted from the original patient training dataset. The frequency of each timestamp is statistically analyzed using a time series analysis algorithm to generate an hour-to-detection frequency correspondence table. Specifically, the detection timestamp field of all valid records is extracted from the original patient training dataset (this field represents the specific detection time corresponding to each detection record, in the format of year-month-day, hour:minute:second). The frequency of these timestamps is statistically analyzed using a time series analysis algorithm. For example, the number of valid detection records in each hourly segment from 0:00 to 23:00 each day is counted to generate an hour-to-detection frequency correspondence table.

[0113] The hour-to-detection frequency correspondence table identifies consecutive hourly intervals with detection frequencies higher than other time periods. These intervals are considered as the actual clinical detection rhythm periods. Specifically, the correspondence table identifies consecutive hourly intervals with detection frequencies significantly higher than other time periods (typically characterized by hourly detection frequencies ≥ 1.5 times the daily average detection frequency within this interval, and a continuous duration ≥ 8 hours). These intervals are then determined as the actual clinical detection rhythm periods (usually concentrated between 8:00 and 18:00 daily). After inference, the specific time range of the actual clinical detection rhythm period (e.g., 8:00-18:00 daily) and the average number of detection records per hour within this period (e.g., an average of 40 records per hour) are obtained, providing rhythmic basis for subsequent generation of continuous time segments.

[0114] Based on the actual clinical detection rhythm period, the rhythm period is evenly divided into several continuous time segments according to a preset fixed number;

[0115] The system obtains the time interval between any two adjacent time segments. Starting from the start time of the actual clinical testing rhythm period, it generates timestamps for each time segment sequentially according to this time interval, thus obtaining a set of time segments. Specifically, based on the inferred actual clinical testing rhythm period (e.g., 8:00-18:00 daily, a total of 10 hours), the system evenly divides this rhythm period into several continuous time segments at a fixed number of 400 points per day. For example, first, the time interval of each time segment is calculated (e.g., 400 points per day correspond to a 10-hour rhythm period, time interval = 10 hours × 60 minutes × 60 seconds ÷ 400 = 90 seconds). Then, starting from the start time of the rhythm period (e.g., 8:00:00), the system generates timestamps for subsequent time segments sequentially according to this time interval (e.g., 8:01:30, 8:03:00... 17:58:30), ultimately forming a set of time segments with 400 continuous timestamps per day. This method can strictly follow the time distribution characteristics of the actual clinical testing rhythm and simulate the time pattern of testing behavior during actual working hours.

[0116] Valid detection records are randomly selected from the original patient training dataset, and perturbation amplitude amplification is performed on each selected original record to generate perturbation amplified records.

[0117] Each perturbation amplification record corresponds to an unused timestamp in the time segment set, thus forming supplementary data; the supplementary data and the original patient training dataset constitute the completed dataset; specifically, valid detection records are randomly selected from the original patient detection data set (the selection method is sampling without replacement to ensure that the original data distribution characteristics are not destroyed), and perturbation amplitude control is performed on each selected original record. For example, based on the detection value of the original record, a random perturbation of ±0.1% is applied (the perturbation value is generated by a uniformly distributed random function, and the perturbation value is...). The value needs to be verified through the original data distribution range, that is, to ensure that the value after perturbation does not exceed the minimum-maximum range of the detection item in the original data set, so as to avoid destroying the statistical characteristics of the data, and generate one perturbation amplification record; each perturbation amplification record corresponds to one unused timestamp in the continuous time segment set generated in the above steps, and the timestamps are sequentially allocated according to the time order of the continuous time segment set (e.g., the first perturbation amplification record is allocated 8:01:30, the second is allocated 8:03:00, etc.), to ensure that the time series continuity between the newly added perturbation amplification record and the original record is not disrupted.

[0118] In this embodiment, the step of standardizing and optimizing the distribution of the completed dataset to obtain an optimized dataset includes:

[0119] The system filters out abnormal records in the optimized dataset, retaining the timestamp field of the abnormal records to maintain the temporal sequence structure. After filtering, the detection values ​​of the remaining records are extracted into a sequence of values ​​to be processed. Specifically, the optimized dataset contains original valid detection records and perturbation amplification records. Each record contains a detection value and a sequential timestamp. The data cleaning script automatically filters out abnormal records in the set whose detection values ​​are 0 or negative, while retaining the timestamp field of all records to maintain the temporal sequence structure. After filtering, the system extracts the detection values ​​of the remaining records into an independent sequence of values ​​to be processed, and the timestamp field is temporarily stored in the cache to provide pure numerical processing objects for subsequent standardization and distribution optimization.

[0120] The Box-Cox transform function is used to transform the numerical sequence to be processed, thereby generating a Box-Cox transformed numerical sequence; specifically...

[0121] The Box-Cox transformation function in the statistical toolkit is invoked to perform transformation on the numerical sequence to be processed. First, the search range of the power parameter λ of the Box-Cox transformation is set to [-2, 2]. A sequence of candidate λ values ​​is generated with a step size of 0.1 (e.g., -2.0-1.9…2.0). For each candidate λ value, the system calculates the absolute value of skewness of the transformed numerical sequence and selects the λ value with the smallest absolute value of skewness as the optimal power parameter. If multiple candidate λ values ​​correspond to the same minimum absolute value of skewness, the λ value closest to 0 is selected (to avoid excessive transformation that could lead to data distortion). After determining the optimal λ, the system performs the transformation on all values ​​in the numerical sequence to be processed according to the Box-Cox transformation formula, generating a Box-Cox transformed numerical sequence, thus achieving preliminary skewness improvement.

[0122] The numerical sequence after the Box-Cox transformation is optimized to obtain the optimized dataset;

[0123] In this embodiment, the numerical sequence after the Box-Cox transformation is optimized to obtain an optimized dataset, including:

[0124] A parameter space for coarse grid search is defined, which includes the range of the power transformation factor α and the range of the translation parameter β. All combinations of α and β are generated as coarse grid candidate parameter pairs. For each candidate parameter pair, the Box-Cox transformed numerical sequence is transformed according to the formula z=(y+β)α to form the coarse optimized numerical sequence.

[0125] Plot the coarse-optimized numerical sequence and the standard QQ plot, and calculate the Pearson correlation coefficient of the points in the QQ plot. After traversing all coarse grid candidate parameter pairs, the system selects the top 10 parameter pairs with the highest QQ correlation coefficients to form a coarse-screened candidate parameter set.

[0126] A set of fine grid candidate parameter pairs is generated based on the coarse screening candidate parameter set, wherein the set of fine grid candidate parameter pairs includes at least one fine grid candidate parameter pair.

[0127] For each candidate parameter pair of the fine mesh, generate the corresponding refined and optimized numerical sequence;

[0128] Based on the various optimized numerical sequences, the single parameter pair with the highest QQ correlation coefficient was selected as the optimal parameter pair for TDO;

[0129] The optimized dataset is generated based on the optimal parameters of TDO.

[0130] For example, based on the definition of Two-Parameter Distribution Shape Optimization (TDO), two adjustable parameters, the power transformation factor (denoted as α) and the translation parameter (denoted as β), are introduced to further optimize the numerical sequence after the Box-Cox transformation in order to achieve approximate normality.

[0131] First, the parameter space for coarse grid search is set, with the power transformation factor α ranging from [0.5, 2.0] (step size 0.1) and the translation parameter β ranging from [0, max(y)] (max(y) is the maximum value of the numerical sequence after Box-Cox transformation, with a step size of max(y) / 20). All combinations of α and β are generated as coarse grid candidate parameter pairs. For each candidate parameter pair (α, β), the system performs a transformation on the Box-Cox transformed numerical sequence according to the formula z=(y+β)α (z is the coarsely optimized value, and y refers to a single value in the numerical sequence obtained after Box-Cox transformation). Then, a QQ plot (Quantile-Quantile Plot) of the transformed sequence and the standard normal distribution is plotted, and the Pearson correlation coefficient of the points in the QQ plot (i.e., the linear correlation between the sample quantile and the standard normal quantile) is calculated. After traversing all coarse grid candidate parameter pairs, the system selects the top 10 parameter pairs with the highest QQ correlation coefficients to form a coarse screening candidate parameter set, completing the coarse grid search stage and initially locking in the direction of parameter optimization.

[0132] Centered on the α and β values ​​of each parameter pair in the coarse screening candidate parameter set, a local fine-grid parameter space is constructed, where the fine-grid range of α is [α]. center -0.05, α center +0.05] (step size 0.01), the fine mesh range of β is [β center -0.05×max(y), β center +0.05×max(y)] (step size is max(y) / 100), ensuring that the fine grid covers the surrounding area of ​​the optimal parameters of the coarse screen; for each parameter pair (α',β') in the fine grid, repeat the numerical transformation-QQ plot drawing-correlation coefficient calculation process of the coarse grid search stage, that is, generate the fine-optimized numerical sequence according to z′=(y+β′)α′, and calculate its Pearson correlation coefficient with the standard normal distribution QQ plot;

[0133] The system selects the single parameter pair with the highest QQ correlation coefficient from all fine-mesh parameter pairs (if multiple pairs of highest correlation coefficients exist, the pair with α' closest to 1 and β' smallest is selected to reduce over-transformation) as the optimal parameter pair α for TDO. opt , β optThe process involves performing fine-grid self-calibration to further approximate the normal distribution of the numerical sequence. For each value in the numerical sequence (y sequence) after the Box-Cox transformation, the transformation is performed one by one to generate the TDO optimal parameter pair of the transformed numerical sequence (denoted as z′ sequence). This completes the fine-grid self-calibration and further approximates the normal distribution of the numerical sequence.

[0134] Outlier detection is performed on the transformed numerical sequence (z' sequence) with the optimal TDO parameters, using the 3σ criterion (i.e., values ​​exceeding the sequence mean ± 3 standard deviations are considered outliers). However, outliers are not removed; instead, the minimum value min(z') in the sequence is recorded. If min(z') ≤ 0, the system calculates the shift shift = 1 - min(z') (ensuring all values ​​are positive after shift to avoid nonlinear calculation errors caused by negative values ​​in subsequent algorithms). The shift operation z is then performed on all values ​​in the z' sequence. final =z′+shift; If min(z′)>0, then directly retain the sequence z′ as z. final Sequence; after the translation is completed, the system z final The sequence is re-associated with the corresponding timestamp field in the buffer to form an optimized dataset that is time-continuous, approximately normally distributed, and has all positive values. Finally, the system calculates the mean, standard deviation, skewness, kurtosis, and other statistics of the optimized dataset, generates a data standardization report, and outputs it along with the optimized dataset to provide a unified numerical basis for subsequent bias injection.

[0135] In this embodiment, the step of injecting reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias includes:

[0136] Set the core parameters for bias;

[0137] Reproducible bias injection is performed on the optimized dataset based on the bias core parameters, thereby generating a biased numerical sequence.

[0138] Add an error flag and a segment flag to each record in the biased numerical sequence to obtain a biased numerical sequence with flags.

[0139] The biased numerical sequence with identifier bits is combined with timestamps to obtain a training dataset with reproducible bias.

[0140] For example, configure four types of bias core parameters:

[0141] Segment Length: The number of detection records for each bias is set to be between 100 and 300. The system generates the specific length of each bias through a uniform random function (value range [100, 300], integer type). For example, if the bias length of a certain segment is 150 points, it means that the segment contains 150 consecutive detection records. Each segment length is generated independently to ensure the randomness of the short-term drift of the simulated real instrument.

[0142] Segment Interval: Set the number of normal detection records between two adjacent biases to 550-600 points (the PDF clarifies that this interval is used to simulate the natural interval between test batches). The system generates the interval length between each bias and the next bias using a uniform random function (value range [550, 600], integer type). For example, an interval of 580 points means that there are 580 normal detection records between the two biases.

[0143] Bias Magnitude: The bias magnitude levels are set to ±10%, ±30%, ±50%, ±70%, and ±90% (a total of 10 levels, including positive and negative bias). Each magnitude level corresponds to the bias amount of the base data value multiplied by the magnitude ratio (e.g., if a value in the base data is 20, the bias amount corresponding to a +10% magnitude is 20 × 10% = 2).

[0144] Bias Direction: Simultaneously configure two directions: positive bias (base data value + bias amount) and negative bias (base data value - bias amount), ensuring that each amplitude level corresponds to a set of positive bias parameters and a set of negative bias parameters.

[0145] Fixed Random Seed (SDBF Mechanism): Based on the Deterministic Seed-Controlled Bias Generation Model (SDBF), the system requires the user to input or automatically generate a fixed random seed (e.g., 12345, an integer). This seed serves as the basis for generating all random parameters (segment length, inter-segment interval, bias injection start position, etc.). The system records this random seed and stores it in the bias parameter configuration table. Clearly marking the random seed is the core identifier to ensure cross-center reproducibility. When the same random seed is run in different laboratories and on different hardware platforms, it will generate completely consistent bias segment positions, lengths, amplitudes, and directions, avoiding the problem of non-reproducibility of random positions in traditional bias injection.

[0146] Bias parameter configuration table generation: The system integrates the above-configured segment length, inter-segment interval, bias magnitude, bias direction, and random seed to generate a structured bias parameter configuration table. Each record in the table corresponds to a set of independent bias parameters (such as segment length 150 points, interval 580 points, magnitude +30%, positive bias direction, and seed 12345), and clearly marks the unique identifier of each set of parameters (such as Bias_001, Bias_002...Bias_10, corresponding to 10 types of bias magnitude levels). This table will serve as the basis for subsequent bias injection operations and will also be used for subsequent cross-center reproduction and audit verification.

[0147] Bias injection start position calculation: The system determines the position of each type of bias at z based on a fixed random seed and a pseudo-random number generation algorithm. final The starting position in the sequence, first calculate z final The total length of the sequence (denoted as L) ensures that the length of the biased segment + the interval between segments × (number of biased segments - 1) ≤ L (to avoid the biased segments exceeding the range of the basic data).

[0148] Then, using a random seed as a reference, generate a starting index for each type of bias (e.g., the starting index for Bias_001 is 1000, representing the starting index from z). final The bias is injected starting from the 1000th record of the sequence, and the starting index must satisfy the condition that the end position of the previous bias segment + the interval between segments ≤ the current starting index, to ensure that the bias segments do not overlap and the interval meets the configuration requirements.

[0149] Segment-by-segment bias injection execution: The system executes the bias injection according to the order of the bias parameter configuration table (from Bias_001 to Bias_10), targeting z. final Sequential segment-by-segment bias injection:

[0150] For a given set of bias parameters (e.g., Bias_001: segment length 150 points, amplitude +30%, starting index 1000), the system first locates the target data segment for that bias, i.e., z. final The sequence contains 150 records with indices ranging from 1000 to 1000+150-1=1149.

[0151] The injected value is calculated based on the bias direction and magnitude: If it is positive bias, the value of each record in the target data segment = original value + (original value × magnitude ratio) (e.g., if the original value is 20, after +30% magnitude it becomes 20 + 20 × 30% = 26); if it is negative bias, the value of each record in the target data segment = original value - (original value × magnitude ratio) (e.g., if the original value is 20, after -30% magnitude it becomes 20 - 20 × 30% = 14).

[0152] After injection, the system marks the target data segment as a bias segment and continues processing the next set of bias parameters until all 10 types of biases have been injected, generating a biased numerical sequence (denoted as z). bias sequence).

[0153] Data labeling (BMSR mechanism): Based on the bidirectional symmetric construction and hierarchical reproduction mechanism (BMSR), for z bias Each record in the sequence is attached with two types of labels to enable accurate tracing of biased segments:

[0154] Error flag: Set the flag value to 0 or 1. 0 represents that the record is original normal data (without injected bias), and 1 represents that the record is abnormal data after injected bias; the system iterates through z bias The sequence automatically assigns a value of 1 to the corresponding record and a value of 0 to the rest of the records based on the starting index and length of each bias.

[0155] Segment ID: Set the tag value to None or a bias-unique identifier (such as Bias_001, Bias_002). None represents normal data, and the bias identifier represents the bias segment to which the record belongs; the system also iterates through z bias For each bias, assign a unique identifier to the record corresponding to that bias (e.g., all 150 records corresponding to Bias_001 are assigned the value Bias_001), and assign None to normal data.

[0156] Labeled biased dataset integration: The system integrates biased numerical sequences (z...) bias The sequence is matched one-to-one with the additional error flag and segment ID labels, and then re-associated with the timestamp field temporarily stored in the cache (ensuring that each biased record still corresponds to the original timestamp and maintains the continuity of the time sequence), forming a labeled bias dataset. This dataset contains four types of fields: detection value, timestamp, error flag, and segment ID, which can clearly distinguish between normal data and biased data, and can trace the specific source of each bias.

[0157] In this embodiment, the labeled bias dataset can also be parsed to generate a Segment Summary Table, which contains complete structural information of each bias segment:

[0158] Unique bias identifier (e.g., Bias_001): Consistent with the identifier in the bias parameter configuration table, facilitating related queries;

[0159] Start Index: This segment is biased at z biasThe starting index in the sequence (e.g., 1000);

[0160] Segment Length: The number of records included in this bias segment (e.g., 150 points).

[0161] Previous Interval: The number of normal records between this bias segment and the previous bias segment (e.g., the interval between Bias_002 and Bias_001 is 580 points).

[0162] Bias Magnitude: The magnitude of the bias in this segment (e.g., +30%).

[0163] Bias Direction: The direction of the bias in this segment (e.g., positive bias).

[0164] End Residual: The distance from the end of the bias segment to z. bias The normal number of records at the end of the sequence (e.g., Bias_10 ends with an index of 5000, z) bias If the total sequence length is 6000, then the tail residual is 1000 points.

[0165] Dataset Type: Clearly label this dataset as biased training data to distinguish it from subsequent validation data.

[0166] In this embodiment, the setting of guardrail constraint information during the training process includes:

[0167] See Figure 2 Initialize the adjustable parameter space for the MA, MQ, and EWMA algorithms, and set the value range for each parameter; MA (Moving Average) algorithm parameter space:

[0168] Window Width: Values ​​range from 3 to 15 (integer, step size 1), used to control the number of samples used to calculate the moving average;

[0169] Control limit multiples (a, b): a ranges from 1.64 to 3.0 (floating-point, step size 0.1), b ranges from 1.64 to 3.0 (floating-point, step size 0.1), a is the upper control limit multiple, and b is the lower control limit multiple;

[0170] Truncation Factor: Values ​​range from 0 to 0.3 (floating-point, step size 0.05), used to control the strength of extreme value replacement;

[0171] Continuous alarm points (CP): The value ranges from 3 to 15 (integer type, step size 1), which is used to constrain the number of consecutive positive points required to trigger an alarm and avoid false alarms due to instantaneous fluctuations.

[0172] MQ (Moving Quantile) algorithm parameter space:

[0173] Quantile Level: Values ​​range from 0.7 to 0.95 (floating-point, step size 0.05), used to determine the basis for calculating moving quantiles (e.g., 0.9 represents the 90th quantile).

[0174] Control limit multiples (a, b): Same as MA algorithm, a takes values ​​of 1.64-3.0 (floating-point type, step size 0.1), b takes values ​​of 1.64-3.0 (floating-point type, step size 0.1).

[0175] Continuous alarm points (CP): Value range 3-15 (integer, step size 1), same as MA algorithm.

[0176] EWMA (Exponentially Weighted Moving Average) algorithm parameter space:

[0177] λ weight (EWMA Lambda): The value ranges from 0.1 to 0.5 (floating-point, step size 0.05), and is used to control the weight ratio between historical data and recent data;

[0178] Control limit multiples (a, b): Same as MA / MQ algorithm, a takes values ​​of 1.64-3.0 (floating-point type, step size 0.1), b takes values ​​of 1.64-3.0 (floating-point type, step size 0.1).

[0179] Truncation Factor: Values ​​range from 0 to 0.3 (floating-point, step size 0.05), same as the MA algorithm;

[0180] Continuous alarm points (CP): Value range 3-15 (integer, step size 1), same as MA / MQ algorithm.

[0181] Set a hard threshold for guardrail constraints.

[0182] For example, the first priority constraint is the upper limit of the false positive rate (FPR): the FPR is set to ≤ 0.005. This threshold is an insurmountable hard constraint. That is, if any combination of parameters has an FPR > 0.005 calculated on a labeled biased dataset, it will be directly screened out and will not enter the subsequent performance scoring stage to ensure that alarms are not too frequent.

[0183] Second priority constraint: Maximum detection delay (MNPed): Based on the routine clinical detection delay requirements, set MNPed≤50 (unit: number of patient test records). If the MNPed of the parameter combination is>50, it will also be screened out to ensure that the detection is not delayed.

[0184] In this embodiment, the step of using the training dataset with reproducible bias to perform parameter search and performance scoring for the MA algorithm, MQ algorithm, and WEWMA algorithm in parallel includes:

[0185] Separate processes were assigned to the MA, MQ, and WEWMA algorithms, and the following processing was performed on each algorithm:

[0186] The algorithm is run according to different parameter combinations, and the performance index of each parameter combination is calculated and the guardrail constraint is verified.

[0187] After all parameter combinations of the MA algorithm, MQ algorithm, and WEWMA algorithm have been searched, the total score PUMCH_Score of the effective combinations of each type of algorithm is calculated.

[0188] For example, the first step is to perform CPU core detection and resource allocation:

[0189] Based on the requirements of the multi-algorithm parallel scheduling (PAO) system, intelligent allocation of CPU cores is performed:

[0190] Core detection: Call the system's underlying interface (such as the detectCores() function in the parallel package of R language) to automatically detect the total number of CPU cores in the current running environment (e.g., the detection result is 16 cores);

[0191] Core reservation: By default, two cores are reserved for system processes (such as data reading and writing, memory management) to avoid system lag caused by algorithms consuming all resources in parallel.

[0192] Core Allocation: The remaining available cores (e.g., 16-2=14 cores) are allocated to the three types of algorithms based on the principle of even distribution of the number of algorithms and dynamic adjustment. Priority is given to allocating the same number of cores to each type of algorithm (e.g., 14 cores are allocated as MA=5 cores, MQ=5 cores, and EWMA=4 cores). If the number of parameter combinations of a certain type of algorithm is significantly greater than that of other algorithms (e.g., MA has 2000 sets of parameter combinations, MQ and EWMA each have 1000 sets), then 1-2 cores are adjusted from the cores of the algorithm with fewer parameter combinations to the algorithm with more parameter combinations to ensure maximum resource utilization. The allocation results are recorded in the core scheduling log, specifying the number of cores occupied and the number of each type of algorithm.

[0193] Parallel parameter search execution using multiple algorithms:

[0194] The system uses a multi-core parallel framework (such as the parLapply() function in the R language's parallel package) to initiate parallel parameter searches for three types of algorithms. Each type of algorithm executes the following process independently without interfering with the others:

[0195] Parallel search using the MA algorithm:

[0196] Create a separate process for the core assigned to MA, each process loading a labeled bias dataset with a set of MA parameters;

[0197] The MA algorithm is run according to the following parameter combination: the moving average is calculated based on the window width, and the upper and lower control limits are determined by the control limit multiple a / b (control limit = mean ± a × standard deviation / mean ± b × standard deviation). The extreme values ​​exceeding the quantile threshold are replaced by RTRS through the truncation factor. Finally, the alarm is triggered according to the number of consecutive alarm points (CP) (if CP consecutive points exceed the control limit, it is judged as abnormal).

[0198] Calculate the performance metrics for this parameter combination: based on the error flag of the labeled bias dataset, statistical sensitivity (number of correctly detected biased records / total number of biased records), FPR (number of normal records misclassified as abnormal / total number of normal records), and MNPed (median number of patient records before each bias segment was detected, calculated by referring to the bias segment position in the SGM table).

[0199] Guardrail constraint verification: If the FPR of the parameter combination is greater than 0.005 or MNPed is greater than 50, it is directly marked as an invalid combination and will not be included in the subsequent scoring; if the constraint is met, the calculation results of sensitivity, FPR and MNPed are retained and stored in the MA algorithm performance temporary cache area.

[0200] Parallel search using the MQ algorithm:

[0201] Create a separate process for the core assigned to MQ, and each process loads a labeled bias dataset with a set of MQ parameters.

[0202] Run the MQ algorithm according to this parameter combination: calculate the moving quantile (e.g., 0.9 quantile) based on the quantile level, determine the control limit by combining the control limit multiple a / b, and determine the alarm trigger according to CP;

[0203] Similar to the MA algorithm process, calculate sensitivity, FPR, and MNPed, and perform guardrail constraint verification. The performance results of the effective combination are stored in the MQ algorithm performance temporary cache area.

[0204] EWMA algorithm parallel search:

[0205] Create a separate process for the core assigned to EWMA, each process loading a labeled bias dataset with a set of EWMA parameter combinations;

[0206] The EWMA algorithm is run according to the following parameter combination: the exponentially weighted moving average is calculated based on the λ weight (high weight for recent data and low weight for historical data), the control limit is determined by the control limit multiple a / b, RTRS replacement is performed by the truncation factor, and alarm triggering is determined by CP.

[0207] Similar to the MA / MQ algorithm process, calculate performance metrics and verify constraints. The results of effective combinations are stored in the EWMA algorithm performance temporary cache area.

[0208] After all parameter combinations for the three types of algorithms have been searched, the system merges the effective combination data from the temporary performance buffer of the MA / MQ / EWMA algorithms, and calculates the PUMCH_Score for each type of algorithm's effective combination:

[0209] ;in, Default weighting ratio: =0.0005, =0.999, W3=0.0005; Sensitivity; The false positive rate; This is a penalty function for detection delay based on the Sigmoid function.

[0210] In this embodiment, the algorithm for obtaining the matched result based on the guardrail constraint information and the two-stage filtering includes:

[0211] The results of all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm are filtered out to remove parameter combinations that exceed the guardrail threshold, thereby obtaining the coarse-screened combination for each algorithm.

[0212] The scores for each parameter combination in the coarse screening combination are sorted according to the total score PUMCH_Score, and the first preset number of parameter combinations are retained.

[0213] The parameter combination with the highest sensitivity is selected from the preset number of parameter combinations as the optimal parameter combination, and the algorithm corresponding to the optimal parameter combination is selected as the matching algorithm.

[0214] For example: Execution constraint priority + two-stage filtering logic:

[0215] Phase 1: Constraint Screening: First, filter out parameter combinations that exceed the guardrail threshold (e.g., FPR>0.005), and retain only feasible solutions that meet clinical constraints;

[0216] Second stage: Coarse screening and sorting: Sort feasible solutions from high to low according to PUMCH_Score, and retain the top 10% of combinations (which can be set as needed). The specific ratio is executed according to the system's preset configuration.

[0217] Phase 3: Fine Screening Decision: Among the combinations retained in the coarse screening, the one with the highest sensitivity is selected first; if the sensitivity is tied, the one with the lowest FPR is selected; if the FPR is still tied, the one with the shortest MNPed is selected, and finally the optimal parameter combination of the algorithm after each type of pre-matching is determined.

[0218] Output results: the optimal parameter set of the algorithm after pre-matching (such as MQ, EWMA) (e.g., the optimal parameters of EWMA may be λ=0.2, a=1.64, b=1.64, cutoff factor=0, CP=10), and the full parameter evaluation result table of the training set (e.g. ma_training_performance.csv).

[0219] Examples are shown in Table 1 below:

[0220] Table 1: Summary Table of Error Segment Lengths and Intervals

[0221]

[0222] Wherein, Interval XY: represents the number of valid data points unaffected by the simulation error between the Xth and Yth error segments. Interval-1: refers to the number of valid data points existing before the first simulation error segment. Segment X Count: refers to the total number of data points within the Xth simulation error segment (i.e., the continuous data block injected with error). Interval 5-: represents the number of valid data points existing after the fifth (i.e., the last) simulation error segment.

[0223] The integrated SPC multi-algorithm optimization and validation method for patient data in this application further includes the following steps:

[0224] Validation set construction and hierarchical validation:

[0225] Validation set partitioning: Independent data are extracted from the original patient data (split chronologically with the training set in a 1:1 ratio) to form the validation set (Test Set).

[0226] Cross-error level hierarchical validation: Reproduce the 10 bias scenarios (±10%, ±30%, ±50%, ±70%, ±90%) in the above steps on the validation set, generate an error type order table (error_decrease_10 → error_increase_90), calculate the sensitivity, FPR, and MNPed for each level of bias, and quantify the detection capability of the algorithm under different bias intensities.

[0227] Closed-loop consistency verification execution: The module simultaneously applies identical parameter configurations (i.e., determined optimal parameters) to both the training set and the validation set, generates a dual-table comparison, and calculates the consistency between the two sets across six metrics through a consistency analysis (Training vs Test Comparison). Figure 3 The relative deviation on (as shown in the figure).

[0228] Multidimensional performance visualization: This application can automatically generate nine types of structured charts, including:

[0229] Sensitivity Bar Chart;

[0230] False positive rate curve (FPR Curve);

[0231] Detection delay curve (MNPed Line Plot);

[0232] Control Chart (WORD "Control Chart");

[0233] Dual Dataset Overlay (training set vs. test set)

[0234] Error Distribution Map; all charts are saved as high-resolution PNG files, and each chart includes complete parameter information (window width, control limit multiple, number of continuous alarm points, etc.) for easy tracing and reporting.

[0235] This application can generate the following six types of core files (PDF "IRAG") based on the above content:

[0236] ① Error segment structure summary: error_segments_summary.csv (including segment length, interval, bias magnitude, etc.);

[0237] ② Full parameter evaluation results of the training set: ma_training_performance.csv (containing all parameter combinations and index values);

[0238] ③ Training set mean summary: ma_training_performance_summary.csv (average performance of each parameter);

[0239] ④ Detailed test set results: test_results_detailed_ma.csv (detailed metrics for each bias scenario);

[0240] ⑤ Training vs. Testing Comparison: train_vs_test_comparison_ma.csv (dual-set performance comparison);

[0241] ⑥ Optimal parameter record: optimal_parameters.csv (parameter template for final deployment);

[0242] This application may also include version number, dataset information, performance comparison of three types of algorithms, description of optimal parameters, verification conclusions, etc.; all files are packaged and output to a unified directory to form a traceable and deliverable verification archive.

[0243] The integrated SPC multi-algorithm overall optimization and validation method for patient data proposed in this application has the following advantages:

[0244] Based on the SDBF (Deterministic Seed Control) and MDBS (Multidimensional Parametric Bias Structure) mechanisms, by explicitly specifying five-dimensional parameters—fixed random seed, segment length, interval, amplitude, and direction—the bias generation results are completely consistent across different centers and platforms (100% reproducibility consistency). Simultaneously, an SGM (Segment Statistical Table of Bias) table is output, containing structured information such as the starting position and amplitude of each bias segment, supporting cross-experimental reproducibility and audit verification, thus completely resolving the problems of randomized error injection and incomparable results in traditional PBRTQC.

[0245] Following the Guardrail Constraints and CDOF (Constraint-First Optimization) mechanism, this design prioritizes setting hard constraints such as FPR ≤ 0.005 and MNPed ≤ clinical upper limit, and then maximizes sensitivity through PUMCH_Score. Compared with traditional solutions that only perform weighted averaging, this design ensures that the algorithm does not alarm too frequently (low FPR) and does not detect too late (short MNPed), while also taking into account the ability to identify abnormalities (high sensitivity), which perfectly matches the core needs of clinical laboratories for fewer false alarms and faster detection.

[0246] The system integrates data preprocessing, bias simulation, algorithm optimization, and result output into a fully automated process. The data preprocessing module achieves self-sufficiency in sample size and distribution optimization through ATC+TDO, eliminating the need for manual data supplementation. The SACO module reduces algorithm runtime to one-third of its original time through PAO (multi-core parallel scheduling), avoiding the tedious operations of traditional solutions involving multiple tools and manual result processing, significantly lowering the barrier to laboratory implementation.

[0247] By optimizing and adapting to diverse data through RTRS (Truncation Replacement) and distribution awareness: RTRS does not remove extreme values ​​but replaces them with quantile thresholds, stabilizing control limit estimation without sacrificing sample size; at the algorithm level, MA is adapted to near-normal data, MQ is adapted to skewed / heavy-tailed data, and EWMA is adapted to time-series fluctuating data. There is no need for manual judgment of data distribution type. The system can automatically select the optimal algorithm through parameter optimization and is compatible with multiple types of analytes commonly used in clinical practice, such as NSE, CA19-9, and CA125.

[0248] In an alternative embodiment, before performing parameter search and performance scoring on the MA, MQ, and WEWMA algorithms in parallel using the training dataset with reproducible bias, the patient data-oriented SPC multi-algorithm integrated optimization and validation method further includes:

[0249] Extract two-dimensional features from the training dataset with reproducible bias;

[0250] Construct a two-dimensional feature map based on the two-dimensional features;

[0251] Get the preset matching rules;

[0252] Based on the preset matching rules, an algorithm pre-matching list is constructed using the two-dimensional feature map.

[0253] The parallel parameter search and performance scoring of the MA, MQ, and WEWMA algorithms using the training dataset with reproducible bias includes:

[0254] Using the training dataset with reproducible bias and the algorithm pre-matching list, the MA algorithm, MQ algorithm and WEWMA algorithm are performed in parallel for parameter search and performance scoring.

[0255] For example, two-dimensional features can include the following features:

[0256] First feature (biased scene feature extraction): Extract three types of quantifiable biased scene features from the bias fragment statistics table (SGM table) and the biased training dataset:

[0257] Bias segment length type: Divide the segment length into short segment (100-160 points), medium segment (161-230 points), and long segment (231-300 points), and calculate the proportion of each type of segment length to the total bias segments (if the short segment accounts for 60%, it is marked as short segment dominant).

[0258] Bias magnitude level: The magnitude is divided into low magnitude (±10%~±30%), medium magnitude (±31%~±60%), and high magnitude (±61%~±90%) according to ±10%, 30%, 50%, 70%, and 90%, respectively. The detection difficulty coefficient of each level of bias is calculated (for example, low magnitude coefficient 1.2, medium magnitude 0.8, and high magnitude 0.5).

[0259] Bias interval density: Calculate the number of bias segments per unit time (the shorter the interval, the higher the density), and classify them into low density (≤0.2 segments / thousand points), medium density (0.21-0.5 segments / thousand points), and high density (>0.5 segments / thousand points).

[0260] Second feature (data distribution feature extraction): Extract two types of data distribution features from standardized data (from the TDO mechanism and the distribution attributes after Box-Cox transformation):

[0261] Kurtosis: Calculate the kurtosis value of standardized data and classify it into low-kurtosis (kurtosis < 2), medium-kurtosis (2 ≤ kurtosis ≤ 4), and high-kurtosis (kurtosis > 4);

[0262] Distribution skewness: Based on the results of the Box-Cox transformation, the skewness value of the transformed data is calculated and divided into weak skewness (|skewness|≤0.3), moderate skewness (0.3<|skewness|≤0.8), and strong skewness (|skewness|>0.8).

[0263] Construct a two-dimensional feature map of biased scenarios and data distribution: with biased scenario features as the horizontal axis (including three dimensions: segment length type, amplitude level, and interval density) and data distribution features as the vertical axis (including two dimensions: kurtosis and skewness), each dimension is assigned a value according to the classification (e.g., short segment dominance assigned value 1, medium segment dominance assigned value 2, long segment dominance assigned value 3; weak skewness assigned value 1, medium skewness assigned value 2, strong skewness assigned value 3), forming a 5-dimensional feature vector (corresponding to short segment dominance, medium amplitude, medium density, medium kurtosis, and weak skewness, respectively).

[0264] Feature map standardization: The 5-dimensional feature vectors are standardized to the [0,1] interval using the maximum value normalization method (e.g., the amplitude level coefficient of 0.8 is standardized to 0.8 / 1.2≈0.67), generating a standardized feature map that can be directly used for matching.

[0265] Establish the following pre-matching fixed rules:

[0266] Rule 1 (Low Amplitude Bias + Medium-High Skewness Distribution): If the low amplitude bias coefficient in the standardized feature map is ≥1.0 and the skewness is ≥2 (medium / strong bias), then the MA algorithm is excluded (because MA has low sensitivity to low amplitude bias, and the MA control limit is easily shifted under medium-high skewness distribution), and only the MQ and EWMA algorithms are retained for subsequent steps.

[0267] Rule 2 (Long-segment bias + kurtosis distribution): If the long-segment bias accounts for ≥50% and the kurtosis is ≥3 (high), then the MQ algorithm is excluded (because MQ is based on quantile values, and quantile values ​​under kurtosis distribution are easily affected by extreme values, and the MQ detection delay is relatively long under long-segment bias), and only the MA and EWMA algorithms are retained for subsequent steps;

[0268] Rule 3 (High-density bias + low-kurtosis distribution): If the bias interval density is ≥0.4 segments / thousand points and the kurtosis is ≤2 (low-kurtosis), then all three types of algorithms are retained.

[0269] Output pre-matching results: Generate a list of pre-matched algorithms, clearly indicating the types of algorithms that will proceed to step 4 (e.g., retain MQ and EWMA), and include the pre-matching criteria (e.g., if they meet rule 1: low amplitude bias + moderately biased distribution, exclude MA).

[0270] In this embodiment, using a training dataset with reproducible bias and an algorithm pre-matching list, parameter search and performance scoring are performed on the MA, MQ, and WEWMA algorithms in parallel, including:

[0271] If MA is retained: Initialize MA parameters (window width, control limit multiple a / b, cutoff factor, number of continuous alarm points CP); the specific calculation process is the same as above, and will not be repeated here.

[0272] If MQ is retained: initialize MQ parameters (quantile level, control limit multiple a / b, number of continuous alarm points CP); the specific calculation process is the same as above, and will not be repeated here.

[0273] If EWMA is retained: initialize EWMA parameters (λ weight, control limit multiple a / b, number of continuous alarm points CP); the specific calculation process is the same as above, and will not be repeated here.

[0274] Setting guardrail constraints is the same as the settings above, except that you don't need to set them for certain algorithms.

[0275] Multi-core parallel parameter search and performance scoring:

[0276] Parallelized Algorithmic Orchestration (PAO):

[0277] Automatically detects the number of CPU cores in the system and reserves 2 cores by default for use by system processes;

[0278] The remaining cores are dynamically allocated for parallel evaluation of the algorithm after pre-matching, and each core independently performs parameter combination evaluation, grid search and result aggregation without interfering with each other;

[0279] The results of each algorithm are cached independently to avoid duplicate calculations.

[0280] Finally, the performance score is calculated.

[0281] In addition to the advantages described above, the alternative embodiments also have the following advantages through the above-described method:

[0282] By excluding algorithms with poor adaptability through a pre-matching list, such as excluding the MA algorithm in low-amplitude bias + medium-to-high skewness data scenarios (because MA has low sensitivity to low-amplitude bias and the control limit is prone to shift in medium-to-high skewness), only MQ and EWMA are retained for subsequent parameter search. Compared with the original solution of searching all three types of algorithms, the number of parameter combinations can be reduced by 30%-50%. In multi-center large-scale data scenarios, the parameter search time of SACO can be shortened by more than 40%, significantly improving the overall process efficiency.

[0283] Because some algorithms are excluded, CPU cores are only allocated to the reserved algorithms. For example, after pre-matching, only two types of algorithms are retained. The available cores (such as 14 cores) do not need to be distributed among the three types of algorithms. Instead, they can be centrally allocated to the reserved algorithms (such as MQ=7 cores, EWMA=7 cores), avoiding the problem of insufficient parallelism of a single algorithm due to core distribution.

[0284] To verify the functional integrity, operational stability, and cross-center applicability of the patient-data-oriented SPC multi-algorithm integrated optimization and validation method proposed in this application, multi-center clinical test data from three medical centers—a hospital in Beijing (PUMCH), a cancer hospital in Shandong Province (SCHI), and a central hospital in Xiangyang City (XYCH)—were selected for validation. Five representative analytical items included: neuron-specific enolase (NSE), carbohydrate antigens CA19-9 and CA125, type I procollagen N-terminal propeptide (TP1NP), and squamous cell carcinoma antigen (SCCAg).

[0285] I. System Operation Results and Data Characteristics

[0286] The system automatically completed the loading, cleaning, segmentation, and visualization analysis of data from each center. For each center, the most recent 7,000 valid patient test results were extracted and automatically divided into training and testing sets (1:1 ratio) in chronological order.

[0287] During operation, the system automatically generates visualizations of the distribution of the five analytes at each center (see...). Figure 4-6 The results show that the original distributions of all analytes are significantly right-skewed, and after Box-Cox transformation (λ between -0.8 and -0.25), they are approximately normalized, demonstrating the effectiveness and consistency of the system's data preprocessing module.

[0288] II. Biased Data Generation and Parameter Optimization

[0289] The system uses a built-in Deterministic Synthetic Bias Generator (DSBG) to generate ten simulated bias scenarios in the training and testing data: five positive biases (+10%, +30%, +50%, +70%, +90%) and five negative biases (-10%, -30%, -50%, -70%, -90%). Each bias scenario consists of five error segments (segment length 100–300 points), spaced 250–500 normal data points apart, to simulate the temporal characteristics of drift or sudden bias in real detection systems.

[0290] The distribution of bias segment length and interval is shown in the figure. Figure 7 as well as Figure 8 The system maintains the same bias injection location and structure between the training and test sets, ensuring that results from different algorithms can be directly compared.

[0291] During the multi-algorithm optimization process, the system uniformly calls three SPC algorithm modules: moving average (MA), moving quantile (MQ), and exponentially weighted moving average (EWMA), and performs automatic parallel grid search through the SACO (Search and Optimize Coordination Unit).

[0292] The parameter space includes window width, control limit multiples (a, b), quantile level (MQ specific), cutoff factor, number of consecutive alarm points (CP), and λ weight of EWMA.

[0293] Each parameter combination is automatically evaluated using a comprehensive scoring function (ME_Score), which takes into account sensitivity, false positive rate (FPR), and detection delay (MNPed). The upper limits of FPR and MNPed are controlled by guardrail constraints to ensure that detection is not too frequent and alarms are not too late.

[0294] III. Performance Verification Results of the Three Types of Algorithms

[0295] The performance and optimal parameter results of the three types of algorithms in three centers and five analytes are as follows: Figures 9 to 11 As shown, the main manifestations are as follows:

[0296] EWMA algorithm

[0297] Among all algorithms, it exhibits the most stable performance and the highest sensitivity. Taking CA125 as an example, the optimal parameter configuration is λ=0.2, a=1.64, b=1.64, cutoff factor=0, and CP=10. The training set sensitivity is 0.9083, and the false positive rate is 0.0017; the performance on the test set is consistent.

[0298] Figure 8 , 9 The EWMA control chart of the CA125 at the PUMCH center shows a clear separation between the red bias segment and the blue normal data, accurately identifying bias amplitudes from 10% to 90%. The relationship between sensitivity and bias amplitude is as follows. Figure 6 As shown, sensitivity gradually increases with increasing bias, approaching 95% when the bias exceeds 50%; the trend of false positive rate is as follows. Figure 10 As shown, it remains below 0.005.

[0299] MA algorithm

[0300] The MA algorithm maintained a low false positive rate and stable sensitivity across all three centers. Taking CA125 as an example, the optimal parameters were window width = 3, a = 3, b = 1.96, cutoff factor = 0.1, and CP = 5, with a training set sensitivity of 0.7872 and an FPR of 0.0013. Compared to EWMA, MA had a slightly higher detection delay (MNPed) but the results were stable, making it suitable for periodically fluctuating data. The sensitivity of the CA199 project was slightly lower, but the inter-center consistency was good.

[0301] MQ algorithm

[0302] The MQ algorithm's overall performance is comparable to MA, with an optimal quantile level of 0.9 and control limits a=b=3. In the PUMCH center CA125 training set, the sensitivity is 0.7086, the false positive rate is only 0.0006, and the results from the three centers show good consistency.

[0303] For CA199, the center sensitivity of SCHI was 0.3141, which was lower than that of PUMCH (0.5194) and XYCH (0.5373), but the false positive rate remained <0.001.

[0304] Please refer to Table 2 below for details:

[0305] Table 2: Optimal parameter combinations for three PBRTQC algorithms on the CA125 analyte and their performance on the training and test sets.

[0306]

[0307] This application also provides an integrated system for SPC multi-algorithm overall optimization and validation oriented towards patient data, the integrated system for SPC multi-algorithm overall optimization and validation oriented towards patient data includes:

[0308] The original patient training dataset acquisition module is used to acquire the original patient training dataset.

[0309] The supplemented dataset acquisition module is used to adaptively supplement the original patient training dataset to obtain the supplemented dataset.

[0310] The optimized dataset acquisition module is used to perform data standardization and distribution optimization on the supplemented dataset to obtain the optimized dataset.

[0311] A training dataset acquisition module with reproducible bias is used to inject reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias.

[0312] A guardrail constraint information acquisition module, which is used to set guardrail constraint information during the training process;

[0313] The training module is used to perform parameter search and performance scoring of the MA algorithm, MQ algorithm and WEWMA algorithm in parallel using the training dataset with reproducible bias.

[0314] The final algorithm acquisition module is used to obtain the matching algorithm based on the guardrail constraint information and the two-stage filtering.

[0315] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A multi-algorithm integrated optimization and validation method for SPC based on patient data, characterized in that, The integrated SPC multi-algorithm optimization and validation method for patient data includes: Obtain the original patient training dataset; The original patient training dataset is adaptively padded with additional samples to obtain the padded dataset. The completed dataset is then standardized and its distribution optimized to obtain an optimized dataset. Reproducible bias injection is performed on the optimized dataset to obtain a training dataset with reproducible bias. Set guardrail constraint information during the training process; Using the training dataset with reproducible bias, the MA algorithm, MQ algorithm, and WEWMA algorithm were performed in parallel for parameter search and performance scoring. The matching algorithm is obtained based on the guardrail constraint information and the two-stage filtering. The guardrail constraint information set during the training process includes: Initialize the adjustable parameter space for MA, MQ, and EWMA algorithms, and set the value range for each parameter; Set a hard threshold for guardrail constraints; The parallel parameter search and performance scoring of the MA, MQ, and WEWMA algorithms using the training dataset with reproducible bias includes: Separate processes were assigned to the MA, MQ, and WEWMA algorithms, and the following processing was performed on each algorithm: The algorithm is run according to different parameter combinations, and the performance index of each parameter combination is calculated and the guardrail constraint is verified. After all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm have been searched, calculate the total score PUMCH_Score of the effective combinations of each type of algorithm. The algorithm for obtaining the matched result based on guardrail constraint information and two-stage filtering includes: The results of all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm are filtered out to remove parameter combinations that exceed the guardrail threshold, thereby obtaining the coarse-screened combination for each algorithm. The scores for each parameter combination in the coarse screening combination are sorted according to the total score PUMCH_Score, and the first preset number of parameter combinations are retained. The parameter combination with the highest sensitivity is selected from the preset number of parameter combinations as the optimal parameter combination, and the algorithm corresponding to the optimal parameter combination is selected as the matching algorithm.

2. The integrated SPC multi-algorithm overall optimization and validation method for patient data as described in claim 1, characterized in that, The process of adaptively padding the original patient training dataset to obtain the padded dataset includes: Determine if the data in the current dataset is below a preset threshold; if so, then... The detection timestamp field of all valid records is extracted from the original patient training dataset. The frequency of each timestamp is statistically analyzed using a time series analysis algorithm to generate an hour-detection frequency correspondence table. The hour-test frequency correspondence table is used to identify consecutive hour intervals with higher test frequencies than other time periods, and these intervals are used as the actual clinical test rhythm periods. Based on the actual clinical detection rhythm period, the rhythm period is evenly divided into several continuous time segments according to a preset fixed number; The time interval between any two adjacent time segments is obtained. Starting from the start time of the actual clinical detection rhythm period, the timestamps of each time segment are generated sequentially according to this time interval to obtain the time segment set. Valid detection records are randomly selected from the original patient training dataset, and perturbation amplitude amplification is performed on each selected original record to generate perturbation amplified records. Each perturbation amplification record corresponds to an unused timestamp in the time segment set, thus forming supplementary data; the supplementary data and the original patient training dataset together constitute the completed dataset.

3. The integrated SPC multi-algorithm overall optimization and validation method for patient data as described in claim 2, characterized in that, The process of standardizing and optimizing the distribution of the completed dataset to obtain the optimized dataset includes: The abnormal records in the optimized dataset are filtered out. During the filtering process, the timestamp field in the abnormal records is retained to maintain the temporal structure. After the filtering is completed, the detection values ​​of the remaining records are extracted into a sequence of values ​​to be processed. The Box-Cox transform function is used to transform the numerical sequence to be processed, thereby generating the Box-Cox transformed numerical sequence. The numerical sequence after the Box-Cox transformation is optimized to obtain the optimized dataset.

4. The integrated SPC multi-algorithm overall optimization and validation method for patient data as described in claim 3, characterized in that, The optimization of the numerical sequence after the Box-Cox transformation to obtain the optimized dataset includes: A parameter space for coarse grid search is defined, which includes the range of the power transformation factor α and the range of the translation parameter β. All combinations of α and β are generated as coarse grid candidate parameter pairs. For each candidate parameter pair, the Box-Cox transformed numerical sequence is transformed according to the formula z=(y+β)α to form the coarse-optimized numerical sequence. Plot the coarse-optimized numerical sequence and the standard QQ plot, and calculate the Pearson correlation coefficient of the points in the QQ plot. After traversing all coarse grid candidate parameter pairs, the system selects the top 10 parameter pairs with the highest QQ correlation coefficients to form a coarse-screened candidate parameter set. A set of fine grid candidate parameter pairs is generated based on the coarse screening candidate parameter set, wherein the set of fine grid candidate parameter pairs includes at least one fine grid candidate parameter pair. For each candidate parameter pair of the fine mesh, generate the corresponding refined and optimized numerical sequence; Based on the various optimized numerical sequences, the single parameter pair with the highest QQ correlation coefficient was selected as the optimal parameter pair for TDO; The optimized dataset is generated based on the optimal parameters of TDO.

5. The integrated SPC multi-algorithm overall optimization and validation method for patient data as described in claim 4, characterized in that, The step of injecting reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias includes: Set the core parameters for bias; Reproducible bias injection is performed on the optimized dataset based on the bias core parameters, thereby generating a biased numerical sequence. Add an error flag and a segment flag to each record in the biased numerical sequence to obtain a biased numerical sequence with flags. The biased numerical sequence with identifier bits is combined with timestamps to obtain a training dataset with reproducible bias.

6. The integrated SPC multi-algorithm overall optimization and validation method for patient data as described in claim 5, characterized in that, Before using the training dataset with reproducible bias to perform parameter search and performance scoring of the MA, MQ, and WEWMA algorithms in parallel, the integrated SPC multi-algorithm optimization and validation method for patient data further includes: Extract two-dimensional features from the training dataset with reproducible bias; Construct a two-dimensional feature map based on the two-dimensional features; Get the preset matching rules; Based on the preset matching rules, an algorithm pre-matching list is constructed using the two-dimensional feature map. The parallel parameter search and performance scoring of the MA, MQ, and WEWMA algorithms using the training dataset with reproducible bias includes: Using the training dataset with reproducible bias and the algorithm pre-matching list, the MA algorithm, MQ algorithm and WEWMA algorithm are performed in parallel for parameter search and performance scoring.

7. A multi-algorithm integrated optimization and validation system for patient data-oriented SPC, characterized in that, The integrated SPC multi-algorithm overall optimization and validation system for patient data includes: The original patient training dataset acquisition module is used to acquire the original patient training dataset. The supplemented dataset acquisition module is used to adaptively supplement the original patient training dataset to obtain the supplemented dataset. The optimized dataset acquisition module is used to perform data standardization and distribution optimization on the supplemented dataset to obtain the optimized dataset. A training dataset acquisition module with reproducible bias is used to inject reproducible bias into the optimized dataset to obtain a training dataset with reproducible bias. A guardrail constraint information acquisition module, which is used to set guardrail constraint information during the training process; The training module is used to perform parameter search and performance scoring of the MA algorithm, MQ algorithm and WEWMA algorithm in parallel using the training dataset with reproducible bias. The final algorithm acquisition module is used to obtain the matching algorithm based on the guardrail constraint information and the two-stage filtering. The guardrail constraint information set during the training process includes: Initialize the adjustable parameter space for MA, MQ, and EWMA algorithms, and set the value range for each parameter; Set a hard threshold for guardrail constraints; The parallel parameter search and performance scoring of the MA, MQ, and WEWMA algorithms using the training dataset with reproducible bias includes: Separate processes were assigned to the MA, MQ, and WEWMA algorithms, and the following processing was performed on each algorithm: The algorithm is run according to different parameter combinations, and the performance index of each parameter combination is calculated and the guardrail constraint is verified. After all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm have been searched, calculate the total score PUMCH_Score of the effective combinations of each type of algorithm. The algorithm for obtaining the matched result based on guardrail constraint information and two-stage filtering includes: The results of all parameter combinations of the MA algorithm, MQ algorithm and WEWMA algorithm are filtered out to remove parameter combinations that exceed the guardrail threshold, thereby obtaining the coarse-screened combination for each algorithm. The scores for each parameter combination in the coarse screening combination are sorted according to the total score PUMCH_Score, and the first preset number of parameter combinations are retained. The parameter combination with the highest sensitivity is selected from the preset number of parameter combinations as the optimal parameter combination, and the algorithm corresponding to the optimal parameter combination is selected as the matching algorithm.

Images