Prediction model of microbial targeted sequencing positive threshold and construction method and application thereof

Abstract

The invention relates to a prediction model of a microbial targeted sequencing positive threshold and a construction method and application thereof. The method comprises the following steps: (S1) collecting information of a known microorganism sample and a tNGS detection result of the known microorganism sample as a data set; (S2) screening characteristic quantities in the tNGS detection result as a characteristic quantity set; and (S3) the data set is divided into a training set and a test set, the machine learning model is trained, the prediction label of the training set is the strain identification result of the known microorganism sample, and the feature is the feature quantity set. According to the method, interference signals are removed through data cleaning, key information is extracted through characteristic quantity selection, the optimal threshold rule is learned through a machine learning model, high consistency of the detection result and the gold standard is achieved, meanwhile, the contribution degree of each characteristic quantity to threshold judgment is determined, and support is provided for interpretability of the detection result.

Description

Technical Field

[0001] This invention belongs to the fields of pathogenic microorganism molecular detection technology and bioinformatics technology, and relates to a predictive model for the positive threshold of microbial targeted sequencing, its construction method and application. Background Technology

[0002] Targeted next-generation sequencing (tNGS) is a next-generation sequencing technology that combines pathogen targeting and enrichment with high-throughput sequencing. It uses a large number of primers targeting specific gene sequences to amplify nucleic acids in the sample via ultramultiplex polymerase chain reaction (PCR) or probe capture enrichment, obtaining a large number of target nucleic acid fragments to amplify the target pathogen signal and reduce host interference. The resulting sequences are then subjected to bioinformatics analysis, achieving high-sensitivity and high-resolution identification.

[0003] In microbial molecular detection, the judgment threshold is the core basis for distinguishing between "positive" and "negative" results, and directly determines the accuracy of the test results. Traditional judgment threshold setting methods mostly rely on manual experience (such as manually setting the threshold line based on the number of pathogen sequences detected by the standard) or fixed statistical rules (such as using 2-3 times the blank control signal value as the threshold), which have the following significant defects: (1) Poor adaptability: The signal sensitivity and background noise of different test batches (such as different operators and different experimental instruments) are different. Using a fixed threshold on different instruments for the same test item can easily lead to false positives or false negatives; (2) Weak anti-interference ability: When there are inhibitors in the sample (such as hemoglobin in blood, humic acid in soil) or the amplification signal of low concentration of target microorganisms is weak, traditional thresholds cannot effectively distinguish between effective signals and interference signals; (3) Strong subjectivity: Manually setting the threshold depends on the experience of the operator. Different people may have different judgments on the threshold of the same batch of data, resulting in poor repeatability of the test results.

[0004] With the maturation of machine learning (ML) technology in data mining and pattern recognition, it possesses the ability to learn feature patterns from massive amounts of detection data and adaptively adjust judgment rules. Based on different learning paradigms and model structures, machine learning can be mainly divided into the following categories: classic linear models, such as Logistic Regression, which are widely used in binary classification problems due to their model transparency and computational efficiency; probability-based Bayesian models, represented by Naive Bayes, which perform well in text classification and other fields based on the conditional independence assumption; and instance-based K-Nearest Neighbors (KNN) algorithms, whose intuitive approach involves classifying and regressing by measuring the similarity between samples. With the increasing demand for modeling nonlinear problems, a series of more powerful models have been proposed. Support Vector Machines (SVMs) use kernel function techniques to map low-dimensional nonlinear problems to a high-dimensional linearly separable feature space, exhibiting excellent performance in small-sample classification. Decision trees and their ensemble learning frameworks further enhance the expressive and generalization capabilities of models: Random Forest (RF) constructs multiple decision trees through bootstrap sampling and effectively reduces the risk of overfitting through its "voting" mechanism; eXtreme Gradient Boosting (XGBoost), a representative algorithm in the gradient boosting direction, corrects the prediction error of the previous round by iteratively training a series of weak learners (such as decision trees), and has proven its excellent prediction accuracy in numerous data science competitions. Furthermore, Artificial Neural Networks (ANNs), inspired by biological neural networks, are powerful tools for handling highly nonlinear relationships. Among them, the relatively basic but powerful Multilayer Perceptron (MLP), through its hierarchical structure containing an input layer, one or more hidden layers, and an output layer, combined with a nonlinear activation function, can approximate any complex continuous function with arbitrary precision, laying the foundation for the development of deep learning technology.

[0005] tNGS is a relatively cutting-edge molecular detection method for microorganisms. Existing technologies are mainly based on empirically defined positive thresholds. In addition, although some existing technologies can predict the background bacteria of sequencing, they have not optimized the selection of features unique to tNGS, and have problems such as the model being a "black box" structure and poor clinical interpretability.

[0006] In conclusion, developing a machine learning-based method for determining the threshold for microbial molecular tNGS detection is of great significance. Summary of the Invention

[0007] To address the shortcomings of existing technologies and practical needs, this invention provides a predictive model for the positive threshold of microbial targeted sequencing, its construction method, and its application. It develops a method for determining the threshold for microbial molecular detection based on machine learning technology, which is applicable to scenarios where targeted high-throughput sequencing is used to detect microorganisms. This method can accurately determine the positive or negative results of microorganisms, thereby improving the accuracy and reliability of detection.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for constructing a predictive model for a positive threshold in microbial targeted sequencing, the method comprising the following steps: (S1) Collect information on known microbial samples and their tNGS detection results as a dataset; (S2) Select the features from the tNGS detection results as the feature set; (S3) Divide the dataset into a training set and a test set, and train the machine learning model. The prediction label of the training set is the species identification result of the known microbial samples, and the feature is the feature set.

[0009] This invention, based on tNGS detection results, selects features specific to tNGS, trains and constructs a prediction model for positive thresholds, and objectively establishes reporting thresholds by integrating multiple indicators. This changes the current situation of relying on manual comprehensive judgment and significantly improves the efficiency and consistency of report interpretation.

[0010] Preferably, the information of the known microbial sample in step (S1) includes the results of strain identification.

[0011] Preferably, the tNGS detection results in step (S1) include the number of original sequences of the sample, sequencing Q30, number of normalized internal reference sequences, number of normalized pathogen sequences, and number of background pathogen sequences.

[0012] Preferably, step (S1) further includes a step of cleaning the dataset, including: when the tNGS detection results are missing in small amounts (e.g., missing <20%), data completion is performed by interpolation; when the sample is missing bacterial species identification results or too many detection results (e.g., missing >=20%), the sample is removed.

[0013] Preferably, step (S2) specifically includes: (S2-1) Construction of the initial feature set: Based on the biological principles of microbial detection, features related to the determination of positive / negative microbial results are selected to construct the initial feature set; (S2-2) Eliminate redundant features: Calculate the variance of each feature in the initial feature set, and eliminate low-variance features that do not contribute to the differentiation of samples (e.g., variance < 0.002); calculate the correlation between pairs of features, and eliminate feature vectors with low contribution among two features with a correlation lower than a preset threshold.

[0014] In this invention, irrelevant features are first eliminated through low-variance filtering and collinearity removal (reducing computational load), and then the optimal features are selected by combining feature selection with model performance (ensuring prediction accuracy), balancing "feature effectiveness" and "model complexity". Through feature contribution analysis, the core influencing factors are identified, and the "black box" problem of traditional machine learning models is solved based on model interpretation technology, meeting the requirements of clinical testing for interpretability of results.

[0015] Preferably, the method for calculating the correlation and contribution includes any one of the following: random forest contribution and Pearson correlation coefficient method, variance inflation factor method, LASSO regression method, or mutual information method; Preferably, the initial feature set includes raw_reads_num, clean_reads_num, raw_Q30, valid_reads_num, valid_micro_reads_num, sum_nc, valid_nail_rpk, valid_host_rpk, patho_reads, patho_genus_reads, patho_RPK, patho_speci fic_rpk, patho_ratio_by_clean, patho_ratio_by_valid, patho_ratio_by_micro, patho_ratio_by_genus, amp_hit_num, amp_all_num, amp_cov_ratio, patho_median_rpk, pos_ppv, patho_mean_clean, patho_i qr_mean, bio_ppv, bio_se, pos_se, pos_neg_ratio, LightGBM_ShapV, LightGBM_pathotype, patho_clinical_level, patho_intra_ntc_coef, patho_inter_ntc_coef, patho_unique_rpk, patho_intra_samnum, pa tho_intra_samratio, patho_other_runsample, patho_ratio_by_run, patho_pvalue, neighbor_size, neighbor_mean, neighbor_ratio, high_neighbor_position, high_neighbor_value and high_neighbor_ratio.

[0016] In this invention, the experimental characteristics and data signal features of tNGS ultra-multiplex PCR molecular detection are combined to screen for tNGS feature quantities from the sample level, pathogen level, and various possible sources of contamination, thereby reducing false positives or false negatives caused by traditional ROC methods (single indicators, such as only normalized sequence number).

[0017] Preferably, the machine learning model includes at least one of multilayer perceptron, logistic regression, random forest, XGBoost, Naive Bayes, support vector machine, or K-nearest neighbor algorithm.

[0018] In this invention, an appropriate model can be selected based on the actual scenario, such as sample size and feature dimensions, to avoid the limitations of a single model.

[0019] Preferably, the training in step (S3) includes: Cross-validation was used to iteratively train candidate models, and the hyperparameters of the models were tuned with the preset performance indicators on the validation set as the optimization target. Using the average performance index obtained by the cross-validation method as the evaluation criterion, the optimal combination of feature quantities is selected from all feature quantities by the feature selection algorithm as the key feature quantity set. By comparing the average performance of various machine learning models in cross-validation, the model with the highest average performance and its corresponding set of key features are selected to form a prediction model.

[0020] Preferably, the feature selection algorithm includes a recursive feature elimination method.

[0021] Preferably, the average performance metric includes at least one of average AUC, average accuracy, average F1 score, average recall, or average precision.

[0022] Secondly, the present invention provides a predictive model for a positive threshold of microbial targeted sequencing, wherein the predictive model is constructed by the method described in the first aspect for constructing a predictive model for a positive threshold of microbial targeted sequencing.

[0023] Thirdly, the present invention provides the method for constructing a prediction model for the positive threshold of microbial targeted sequencing as described in the first aspect, or the application of the prediction model for the positive threshold of microbial targeted sequencing as described in the second aspect in microbial targeted sequencing.

[0024] In this invention, based on the constructed prediction model, the threshold for judging a positive result can be accurately predicted for microbial targeted sequencing data, which helps to achieve rapid and accurate judgment results. It can be effectively applied to microbial targeted sequencing, specifically including detection for non-disease diagnosis purposes, such as detecting soil samples or other non-human samples, conducting basic research or pathogen control, etc.

[0025] Fourthly, the present invention provides an electronic device comprising one or more processors and a memory for storing executable instructions, the one or more processors being configured to invoke the executable instructions stored in the memory to implement the steps in the method for constructing a predictive model for a positive threshold for microbial targeted sequencing as described in the first aspect.

[0026] Fifthly, the present invention provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps in the method for constructing a predictive model for a positive threshold for microbial targeted sequencing as described in the first aspect.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects: This invention provides a machine learning-based method for determining thresholds in microbial molecular detection. It employs data cleaning to remove interference signals, feature selection to extract key information, and a machine learning model to learn optimal threshold rules, constructing a predictive model. This model integrates multiple indicators to objectively establish reporting thresholds, helping to change the reliance on manual judgment and improving the efficiency and consistency of report interpretation. Detection results based on model-predicted thresholds show good consistency with the gold standard, effectively reducing the workload of interpreters. The data cleaning step is adaptable to the signal characteristics of different detection instruments, and the model training process automatically adjusts the threshold based on sample data without manual intervention, making it suitable for multi-scenario detection. Furthermore, this invention uses feature contribution analysis to clarify core influencing factors, solving the "black box" problem of traditional machine learning models and facilitating clinical personnel's understanding of the threshold determination logic. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the model construction process for this invention.

[0029] Figure 2 The figure shows the comparison results of cross-validation and feature selection for multiple candidate models.

[0030] Figure 3 The ROC curve is for the optimal threshold judgment model (random forest model).

[0031] Figure 4 A bar chart showing the overall contribution of the key feature set (21 features).

[0032] Figure 5 A bar chart showing the overall contribution of all feature sets (37 feature quantities).

[0033] Figure 6 This is a diagram showing the global interpretation results based on SHAP.

[0034] Figure 7 SHAP diagram for tuberculosis-negative samples.

[0035] Figure 8 SHAP diagram for tuberculosis-positive samples. Detailed Implementation

[0036] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. However, the following examples are merely simplified examples of the present invention and do not represent or limit the scope of protection of the present invention. The scope of protection of the present invention is determined by the claims.

[0037] Where specific techniques or conditions are not specified in the examples, they shall be performed in accordance with the techniques or conditions described in the literature in this field, or in accordance with the product instructions. Reagents or instruments whose manufacturers are not specified are all conventional products that can be purchased from legitimate channels.

[0038] Unless otherwise defined, scientific and technical terms and their abbreviations used in conjunction with this invention shall have the meanings commonly understood by one of ordinary skill in the art to which this invention pertains. Some of the terms and abbreviations used in this invention are listed below.

[0039] Pearson correlation coefficient (or Pearson product-moment correlation coefficient): This coefficient measures the degree and direction of correlation between two linear variables. In collinearity analysis, the Pearson correlation coefficient between each pair of features is calculated to determine whether there is information redundancy. Calculation formula: X and Y represent two feature quantities. These are the means of X and Y, respectively.

[0040] Variance Inflation Factor (VIF): Quantifies the degree to which a feature's variance increases due to linear correlation with other features. A higher VIF value indicates more severe collinearity. A VIF > 5 or 10 is typically used as the threshold for removing that feature. Calculation formula: ; It is the coefficient of determination obtained by performing linear regression with the i-th feature as the dependent variable and all other features as independent variables.

[0041] Least Absolute Shrinkage and Selection Operator (LASSO): A regression analysis method that, by adding an L1 regularization term (the sum of the absolute values ​​of the model coefficients) to the loss function, forces the model to compress the coefficients of unimportant features to zero, thereby achieving automatic feature selection. The calculation formula is as follows: ; The target variable (dependent variable) is used as the target variable. These are the model's predicted values. For regression coefficients, This is a regularization hyperparameter (penalty term strength) that controls the degree of shrinkage of the regression coefficients. The number of independent variables.

[0042] Mutual information, originating from information theory, measures the interdependence between two variables. Unlike the Pearson coefficient, it captures non-linear correlations. A higher mutual information value indicates more shared information between the two features, potentially leading to higher redundancy. Calculation formula: p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are the marginal probability distributions.

[0043] Shapley Additive Explanations (SHAP): SHAP is a game theory-based explanatory method that helps people understand the output of complex machine learning models by calculating the marginal contribution of features to the model's predictions. It assigns a SHAP value to each feature to indicate its specific impact on the prediction result, making it an important tool for interpreting "black box" models.

[0044] LIME (Local Interpretable Model-agnostic Explanations): Perturbs input features in the local area around a single predicted sample and fits the local behavior of a complex model with a simple, interpretable model (such as linear regression or decision trees) to explain the prediction result for that sample. It focuses more on local fidelity but does not guarantee global consistency.

[0045] Partial Dependence Plot (PDP) and Individual Conditional Expectation (ICE) Plot: These plots demonstrate the marginal effect of one or two features on the model's predictions. PDP shows the average impact of a feature on the prediction, while ICE plots show the impact on each individual sample. They provide a very intuitive understanding of the model's global behavior, and are particularly suitable for continuous features.

[0046] Permutation Feature Importance: Randomly shuffle the values ​​of a feature in the dataset and observe the degree of decrease in model performance (such as accuracy and AUC). The greater the decrease, the more important the feature is.

[0047] Sensitivity / Recall: Measures a model's ability to detect positive examples. It's the proportion of true positive examples that the model correctly predicts. High sensitivity means a low false negative rate. Calculation formula: Recall = TP / (TP + FN).

[0048] Specificity: Measures a model's ability to identify negative examples. It's the proportion of samples that the model correctly excludes from all true negative samples. High specificity means a low false positive rate. Calculation formula: Specificity = TN / (TN + FP).

[0049] Precision: Measures the reliability of a model's predictions of positive examples. It represents the proportion of samples where the model correctly predicts a positive example. High precision means fewer false positives. Formula: Precision = TP / (TP + FP).

[0050] Accuracy: Measures the overall prediction accuracy of the model, i.e., the proportion of all correctly predicted samples out of the total sample. It is effective when the data is class-balanced, but can be misleading on imbalanced data. Calculation formula: Accuracy = (TP + TN) / (TP + TN + FP + FN).

[0051] F1 score: The harmonic mean of precision and recall, used to comprehensively evaluate these two metrics. It is particularly effective in reflecting the true performance of a model than accuracy, especially when there is an imbalance of positive and negative examples. Calculation formula: F1 = 2 × (Precision × Recall) / (Precision + Recall).

[0052] The area under the ROC curve (AUC) depicts the relationship between sensitivity (true positive rate) and 1-specificity (false positive rate) at different decision thresholds. The AUC value measures the overall classification and discrimination ability of the model and is independent of the threshold selection. The closer the AUC value is to 1, the better the model performance.

[0053] TP: True positive, meaning that the actual case is positive and the model also predicts it to be positive.

[0054] FP: False positive, meaning that the actual case is negative, but the model incorrectly predicts it as a positive case.

[0055] TN: True negative, meaning that the actual case is negative and the model also predicts it to be negative.

[0056] FN: False negative, meaning the actual case is positive, but the model incorrectly predicts it as negative.

[0057] This invention designs a method for constructing a predictive model for the positive threshold of microbial targeted sequencing, and trains a machine learning model, as illustrated in the flowchart below. Figure 1 As shown.

[0058] Step 1: Collect and clean the microbial sample information and tNGS detection results to remove interference signals and ensure data quality.

[0059] 1. Information collection: including but not limited to prior results of the sample (specifically, information related to pathogens that may be present in the sample obtained through clinical or laboratory testing methods. This information includes but is not limited to: (1) results of traditional microbiological methods, such as the isolation, culture and drug sensitivity test results of bacteria and fungi; (2) results of targeted molecular detection, such as the results of real-time quantitative PCR (qPCR) or digital PCR for specific pathogens; (3) results of metagenomic detection, such as the results of metagenomic sequencing (mNGS) based on high-throughput sequencing; (4) results of targeted enrichment sequencing, such as the detection results of capturing the genome sequence of specific pathogens through probe hybridization; (5) results of immunological detection, such as the serological detection results of antigens or antibodies of specific pathogens; or, (6) results of direct microscopic examination, such as the results of Gram staining and acid-fast staining microscopy.) as well as the sample detection results such as the number of original sequences, sequencing Q30, number of normalized internal control sequences, number of normalized pathogen sequences, and number of background pathogen sequences, and the prior results are used as the "gold standard results".

[0060] 2. Data Cleaning: The collected data is cleaned. When the sample has only a few missing test results, interpolation is used to complete the data; when a sample has missing prior results or too many missing test results, the sample is removed.

[0061] Step 2: Feature selection to extract key information and reduce model complexity.

[0062] 1. Initial Feature Pool Construction: Based on the biological principles of pathogen detection, important features related to pathogen negative / positive determination are screened to construct an initial feature pool.

[0063] 2. Redundant feature removal: Remove features that do not contribute to model training or may cause interference. Specific methods include: 2.1 Low variance filtering: Calculate the variance of each feature in the dataset. A low variance indicates that the feature has little difference between different samples and does not contribute to distinguishing positive and negative samples, so it is removed.

[0064] 2.2 Collinearity Analysis: To improve model stability and interpretability, highly correlated features are identified and addressed. Besides methods based on random forest contribution and Pearson correlation coefficient, other methods include, but are not limited to, variance inflation factor method, LASSO regression method, or mutual information method.

[0065] Step 3: Machine learning model training and selection, learning the optimal threshold rule.

[0066] 1. Model candidate set: Considering the clinical timeliness and multiple influencing factors of tNGS detection, the selected algorithm needs to be trained quickly under large sample and multiple feature conditions. In order to avoid the influence of the classification model itself, multiple machine learning models are selected as candidate evaluation models.

[0067] 2. Training dataset construction: The cleaned dataset is divided into a training set and a test set according to a certain ratio. The prediction label of the training set is "gold standard result", and the features are the feature set selected in step two above.

[0068] 3. Model Training and Optimization: 3.1 Model Training and Tuning: The candidate model is trained iteratively using cross-validation and the hyperparameters are tuned with the preset performance indicators on the validation set as the optimization target.

[0069] 3.2 Key Feature Set Selection: Based on the average performance index of the aforementioned cross-validation, the optimal combination of features is evaluated from all features using a feature selection algorithm, and is denoted as the "key feature set".

[0070] 3.3 Determination of the optimal model: By comprehensively comparing the average performance of each candidate model in cross-validation, the best-performing model and its corresponding set of key features are selected to form the final classification model.

[0071] Step 4: Verify the final performance based on an independent test set.

[0072] Using the optimal model and key feature set determined in step three, predictions are made on independent test set data, and the generalization performance index is calculated based on the prediction results.

[0073] Step 5: Apply model interpretation techniques (including but not limited to SHAP, LIME, partial dependency graphs, or permutation feature importance) to quantify the impact of each key feature on the final model prediction results, so as to enhance the transparency and interpretability of the model.

[0074] In one specific embodiment of the present invention, an electronic device may also be provided, the electronic device including one or more processors and a memory for storing executable instructions, the one or more processors being configured to invoke the executable instructions stored in the memory to implement the steps in the method for constructing a predictive model for a positive threshold of microbial targeted sequencing.

[0075] Those skilled in the art will understand that the device of the present invention can be obtained using various forms of hardware, software, firmware, dedicated processors, or combinations thereof.

[0076] In one specific embodiment of the present invention, a computer-readable storage medium is also provided, on which computer program instructions are stored, wherein when the computer program instructions are executed by a processor, the steps in the method for constructing a predictive model for a positive threshold for microbial targeted sequencing are implemented.

[0077] In a specific embodiment of the present invention, the technical solution of the present invention is described in detail using the results of 159 respiratory pathogens detected by ultra-multiplex PCR pathogen-targeted sequencing (tNGS) from Guangzhou Jinqirui Biotechnology Co., Ltd.

[0078] Example 1 This embodiment involves sample collection, detection, and feature selection.

[0079] 1. We collected 1243 clinical bronchoalveolar lavage fluid samples with known prior pathogen results. We used a respiratory multiplex pathogen detection kit (Guangzhou Jinqirui Biotechnology Co., Ltd., catalog number: KS608-100HXD96) to perform ultra-multiplex PCR pathogen-targeted sequencing (tNGS) detection of 159 respiratory pathogens. We obtained the pathogen detection results for each sample and statistically analyzed relevant indicators, including the normalized number of each pathogen detected in each sample, the number of internal reference sequences, and the pathogen detection status within each batch.

[0080] 2. Cleaning outlier data: 21 samples were found to have no prior results, and 16 samples had too many missing test results. A total of 37 samples (3%) were removed and not included in the sample set. 97 samples were interpolated using the signal mean method to complete the data. A total of 1206 data points were then included in the subsequent steps.

[0081] 3. Construction of the initial feature pool: Based on the biological principles of pathogen detection, important features related to pathogen negative / positive determination are screened to construct the initial feature pool. Specifically, the initial feature pools are constructed from the aspects of sample level, pathogen level, background contamination, cross-contamination, and adjacent contamination, as shown in Table 1 below.

[0082] Sample level: refers to characteristic indicators at the sample level, such as the number of sequencing sequences and the detection of internal references, to evaluate sample sampling quality, library construction quality, sequencing quality, etc. Pathogen level: refers to characteristic indicators at the pathogen level, such as the number of pathogen-detected sequences, the proportion of pathogen-detected sequences to the number of sequences in the target region, and the clinical concern level of the pathogen; Background contamination: mainly related to background noise introduced by laboratory environment, experimental reagents, experimental instruments, etc., and is generally reflected by intra-batch negative control, healthy human control, inter-batch detection, etc. Cross-contamination: This is mainly related to cross-contamination between samples within the same batch, which manifests as the detection of a certain pathogen in multiple samples from the same batch. Neighbor contamination: Due to aerosol contamination or improper operation during extraction and library preparation, nucleic acid contamination from neighboring samples may be introduced.

[0083] Through the above multi-dimensional evaluation, we comprehensively collect the features that may affect the positive or negative determination of pathogenic microorganisms in tNGS targeted high-throughput sequencing, so that the subsequent machine learning model can achieve the best results by combining and contributing the best features.

[0084] Table 1 The following are explanations of the technical terms involved: Normalized sequence count: Number of detected sequences / Number of original sample sequences × 100,000; IQR: Interquartile Range, which is the range of data values ​​between the 25th and 75th percentiles when the data is sorted in ascending order. 95th percentile: The value that is located at the 95th percentile after the data is sorted in ascending order; NTC: Abbreviation for No Template Control; Bioinformatics prediction results: grading indicators based on normalized sequence count and amplicon coverage; Clinical Pathogen Grading: The clinical pathogenicity of each pathogen is graded by a team of clinical pathogenic microbiology experts in combination with expert consensus and authoritative literature, with 1 to 10 representing progressively increasing pathogenicity. LightGBM_ShapV / LightGBM_pathotype: To improve the comparability between different models, this embodiment trains the LightGBM model (which can directly process string input) to learn 'pathogen name' and 'pathogen type', and uses the average SHAP value of its output as the standardized feature quantities LightGBM_ShapV and LightGBM_pathotype to provide consistent feature input for the evaluation of subsequent candidate models; LightGBM_pathotype: Similar to LightGBM_ShapV; raw_Q30: Q30 of the raw data. Q30 indicates that the probability of misidentification is 0.1%, i.e., the error rate is 0.1%, or the accuracy rate is 99.9%. raw_Q30 refers to the percentage of bases in the raw data with an accuracy rate of over 99.9%.

[0085] 4. Low variance filtering: Calculate the variance of each feature in different sample pathogen data. Since the variances of Q30 and patho_inter_ntc_coef are low (<0.002), these two features are removed. 5. Collinearity analysis: Calculate the contribution ranking of the initial features in the random forest, and calculate the Pearson correlation coefficient between each pair of features. Since patho_ratio_by_clean, patho_specific_rpk, patho_genus_reads, sum_nc, and clean_reads_num have a correlation >= 0.95 with other features with higher contributions, these 5 features are removed. 6. The final set of included features consists of 37 types.

[0086] Example 2 This embodiment performs model training and predicts thresholds.

[0087] 1. Candidate models: MLP, Logistic Regression, Random Forest, XGBoost, Naive Bayes, SVM, and KNN.

[0088] 2. Dataset splitting: The 1206 samples were split into a training set (844 samples) and a test set (362 samples) in a 7:3 ratio. 3. Model Training and Optimization: 3.1 Model Training: The optimal hyperparameters of the aforementioned 7 candidate models are obtained through random optimal search. The 5-fold cross-validation method is used. In each iteration, stratified sampling is used to divide the data into 5 equal parts to ensure that the proportion of each class of samples in the training set and the test set is the same as that in the original dataset. The learning model is trained using 4 of the equal parts and validated using the remaining part of the dataset.

[0089] 3.2 Feature Selection: Recursive Feature Elimination (RFE) is used for feature selection. First, features are sorted in descending order based on their average contribution to the initial feature pool (SHAP value). Then, using the candidate model's AUC value as the evaluation metric, the two features with the lowest contribution are iteratively removed from the back to the front. For example... Figure 2As shown, the AUC of the model on the cross-validation set varies with the number of features, indicating that the AUC reaches a peak of 0.971 when the number of features is 21, and further increasing the number of features does not bring significant gains. Therefore, this set of "21 features" was selected as the key feature set (patho_RPK, patho_reads, patho_ratio_by_valid, patho_ratio_by_micro, amp_cov_ratio, amp_hit_num, patho_ratio_by_run, pos_ppv, valid_micro_reads_num, LightGBM_pathotype, raw_reads_num, valid_host_rpk, patho_pvalue, bio_se, bio_ppv, valid_nail_rpk, patho_other_runsample, valid_reads_num, patho_mean_clean, patho_median_rpk, and high_neighbor_ratio). With 21 features, the cross-validation performance of each model is summarized in Table 2. The average AUC of both Random Forest and XGBoost reached 0.971, but the average accuracy, average F1-Score and average recall of Random Forest were higher than those of XGBoost. Therefore, Random Forest was finally adopted.

[0090] Table 2 4. Based on the optimal hyperparameter model and key feature set trained using the above random forest, the performance metrics of the independent test set are calculated as shown in Table 3. The ROC curves of the training set, test set, and the entire set are shown in Table 3. Figure 3 As shown.

[0091] Table 3 Specifically, the AUC of the training set was 0.996, the AUC of the independent test set was 0.969, the accuracy reached 0.909, and the recall rate was 0.913, showing good consistency with the gold standard.

[0092] 5. Comparison with existing methods: In Example 1, the positive judgment value in the kit instructions is a normalized sequence number >= 10. Using this method, pathogens with a normalized sequence number >= 10 in the independent test set are judged as positive, otherwise as negative. The difference between the results and the gold standard is calculated to obtain the performance index. Then, the judgment result based on the threshold predicted by the model of this invention is calculated and compared with the gold standard. The results of the two are shown in Table 4. It can be observed that although the sensitivity of the optimal model selected in this example, the random forest model, is relatively slightly lower, its accuracy and F1-Score are significantly improved, that is, the false positive is significantly reduced, and the overall consistency between the pathogen positive and negative judgment results and the gold standard is improved.

[0093] Table 4 Example 3 This embodiment interprets the model SHAP.

[0094] Based on the model trained in Example 2, feature contribution analysis was performed on the model based on the SHAP value.

[0095] 1. Overall Contribution of Features: The final contribution ranking of the random forest model was evaluated. The number of normalized pathogen sequences was 24%, the number of pathogen sequences was 18%, and the contribution of the key feature set (21 features) is shown below. Figure 4 The contribution of the entire feature set (37 features) can be found in [link to relevant documentation]. Figure 5 ; Conclusion: The normalized number of pathogen sequences and the number of pathogen sequences are the main features affecting the determination of the detection threshold of tNGS multiplex PCR, while other features are auxiliary features.

[0096] 2. SHAP-based global interpretation: A one-to-one correspondence is obtained between the top 6 key features and their corresponding SHAP values. This allows for a global assessment of the overall contribution of each feature to the positive pathogen classification. See details below. Figure 6 ; Conclusion: The normalized number of pathogen sequences, the number of pathogen sequences, the coverage of amplicon, and the number of amplicon amplifications all showed positive correlations, which is in line with expectations.

[0097] 3. Explanation of SHAP values ​​for a single sample: (e.g.) Figure 7 and Figure 8 The figure shows the interpretability analysis of the model prediction results for a single sample based on the SHAP method. This analysis quantifies the contribution and direction of each feature to the specific prediction result. Figure 7 This study presents a sample that was correctly predicted as "negative for Mycobacterium tuberculosis complex," with most of its features contributing negatively to the positive result. Figure 8This example demonstrates a sample that was correctly predicted as "positive," with most of its features contributing positively. This method clearly elucidates the model's decision-making basis for a single sample.

[0098] This embodiment demonstrates that the SHAP interpretation of the model can visualize the contribution and direction of its key features to the final prediction result, thereby revealing the decision-making logic within the model and confirming the rationality of its judgment.

[0099] Example 4 This embodiment involves evaluation in different laboratories.

[0100] Unlike Example 1, this example selects a sample set from a newly established laboratory in a different region to verify the effectiveness and adaptability of the technical solution of the present invention in different practical application environments.

[0101] The new laboratory's staff consisted mainly of newly hired employees with relatively insufficient operational proficiency and experience. Furthermore, the laboratory's background microbial environment and operational procedures differed significantly from those of the laboratory in Example 1. These factors collectively led to a higher contamination rate in the samples from this laboratory, and the contamination characteristics also differed from those in Example 1.

[0102] The same technical process as in Example 1 (including data preprocessing, redundant feature removal, model training and hyperparameter tuning, and key feature set screening) was used to model the historical sample data of the laboratory.

[0103] The modeling process revealed: 1. Differences in Model Selection: In this embodiment, the XGBoost model performed best in terms of average AUC and average F1-Score during cross-validation, rather than the Random Forest model in Example 1. This demonstrates that the optimal machine learning model may differ for different data features.

[0104] 2. Differences in Feature Contribution: Compared with Example 1, the "key feature set" selected by combining correlation coefficients and random forests shows significant changes in its specific feature composition and contribution ranking. For example, some operation-related features with low contribution in Example 1, such as patho_intra_ntc_coef and high_neighbor_ratio, become high-contribution features in this example due to their high correlation with novice operation errors.

[0105] Validated independently on a test set, the model trained on new laboratory data (Model 2) significantly outperformed the model trained directly in Example 1 (Model 1) on this test set in terms of overall performance (measured by AUC). Specifically, Model 2 showed significantly better sensitivity (89.5% vs 74.3%) and specificity (91.3% vs 85.8%) than Model 1. This result confirms that localized modeling using the method described in this invention can effectively improve the model's ability to capture specific contamination patterns (high sensitivity) and suppress false alarms caused by localized factors (high specificity) in new laboratory environments with different background microorganisms and operational preferences.

[0106] This embodiment demonstrates that the technical solution of the present invention does not generate a single, universal model, but rather provides a modeling process that can adapt to different application scenarios and self-learn specific data features. By performing localized modeling for specific laboratory environments, it is possible to more accurately capture their unique contamination patterns, thereby ensuring the efficiency and reliability of the model in practical applications.

[0107] In summary, this invention combines the experimental characteristics and data signal features of tNGS ultramultiplex PCR molecular detection. It screens features specific to tNGS from the sample level, pathogen level, and various possible sources of contamination. First, it removes irrelevant features through low-variance filtering and collinearity removal (reducing computational load). Then, it selects the optimal features based on feature selection and model performance (ensuring prediction accuracy), balancing "feature effectiveness" and "model complexity." Based on the actual scenario, such as sample size and feature dimensions, it selects an appropriate machine learning model. Through feature contribution analysis, it clarifies the core influencing factors, enabling the machine learning model to learn the optimal threshold rules. Ultimately, it achieves high consistency between the detection results and the gold standard, while also clarifying the contribution of each feature to threshold judgment, providing support for the interpretability of the detection results.

[0108] The applicant declares that the above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention fall within the protection and disclosure scope of the present invention.

Claims

1. A method for constructing a predictive model for a positive threshold in microbial targeted sequencing, characterized in that, The method includes the following steps: (S1) Collect information on known microbial samples and their tNGS detection results as a dataset; (S2) Select the features from the tNGS detection results as the feature set; (S3) Divide the dataset into a training set and a test set, and train the machine learning model. The prediction label of the training set is the species identification result of the known microbial samples, and the feature is the feature set.

2. The method for constructing a predictive model for the positive threshold of microbial targeted sequencing according to claim 1, characterized in that, The information on the known microbial sample in step (S1) includes the results of strain identification; Preferably, the tNGS detection results in step (S1) include the number of original sequences of the sample, sequencing Q30, number of normalized internal reference sequences, number of normalized pathogen sequences, and number of background pathogen sequences.

3. The method for constructing a predictive model for the positive threshold of microbial targeted sequencing according to claim 1 or 2, characterized in that, Step (S1) also includes a dataset cleaning step, including: when the tNGS detection results are few, data completion is performed by interpolation; when the sample is missing bacterial species identification results or too many detection results are missing, the sample is removed.

4. The method for constructing a predictive model for a positive threshold in microbial targeted sequencing according to any one of claims 1-3, characterized in that, Step (S2) specifically includes: (S2-1) Construction of the initial feature set: Based on the biological principles of microbial detection, features related to the determination of positive / negative microbial results are selected to construct the initial feature set; (S2-2) Eliminate redundant features: Calculate the variance of each feature in the initial feature set, and eliminate low-variance features that do not contribute to the differentiation of samples; calculate the correlation between pairs of features, and eliminate feature vectors with low contribution between two features whose correlation is lower than a preset threshold. Preferably, the method for calculating the correlation and contribution includes any one of the following: random forest contribution and Pearson correlation coefficient method, variance inflation factor method, LASSO regression method, or mutual information method; Preferably, the initial feature set includes raw_reads_num, clean_reads_num, raw_Q30, valid_reads_num, valid_micro_reads_num, sum_nc, valid_nail_rpk, valid_host_rpk, patho_reads, patho_genus_reads, patho_RPK, patho_speci fic_rpk, patho_ratio_by_clean, patho_ratio_by_valid, patho_ratio_by_micro, patho_ratio_by_genus, amp_hit_num, amp_all_num, amp_cov_ratio, patho_median_rpk, pos_ppv, patho_mean_clean, patho_i qr_mean, bio_ppv, bio_se, pos_se, pos_neg_ratio, LightGBM_ShapV, LightGBM_pathotype, patho_clinical_level, patho_intra_ntc_coef, patho_inter_ntc_coef, patho_unique_rpk, patho_intra_samnum, pa tho_intra_samratio, patho_other_runsample, patho_ratio_by_run, patho_pvalue, neighbor_size, neighbor_mean, neighbor_ratio, high_neighbor_position, high_neighbor_value and high_neighbor_ratio.

5. The method for constructing a predictive model for a positive threshold in microbial targeted sequencing according to any one of claims 1-4, characterized in that, The machine learning model includes at least one of the following: multilayer perceptron, logistic regression, random forest, XGBoost, Naive Bayes, support vector machine, or K-nearest neighbor algorithm.

6. The method for constructing a predictive model for a positive threshold in microbial targeted sequencing according to any one of claims 1-5, characterized in that, The training described in step (S3) includes: Cross-validation was used to iteratively train candidate models, and the hyperparameters of the models were tuned with the preset performance indicators on the validation set as the optimization target. Using the average performance index obtained by the cross-validation method as the evaluation criterion, the optimal combination of feature quantities is selected from all feature quantities by the feature selection algorithm as the key feature quantity set. Compare the average performance of each machine learning model in cross-validation, select the model with the highest average performance and its corresponding set of key features, and together they form a prediction model. Preferably, the feature selection algorithm includes a recursive feature elimination method; Preferably, the average performance metric includes at least one of average AUC, average accuracy, average F1 score, average recall, or average precision.

7. A predictive model for the positive threshold of microbial targeted sequencing, characterized in that, The prediction model is constructed by the method for constructing a prediction model for a positive threshold of microbial targeted sequencing as described in any one of claims 1-6.

8. The method for constructing a prediction model for the positive threshold of microbial targeted sequencing as described in any one of claims 1-6, or the application of the prediction model for the positive threshold of microbial targeted sequencing as described in claim 7 in microbial targeted sequencing.

9. An electronic device comprising one or more processors and a memory for storing executable instructions, characterized in that, The one or more processors are configured to invoke executable instructions stored in the memory to implement the steps in the method of constructing a predictive model for a positive threshold for microbial targeted sequencing as described in any one of claims 1-6.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps in the method for constructing a predictive model for a positive threshold for microbial targeted sequencing as described in any one of claims 1-6.

Images