System and method for automatically generating machine learning models

Abstract

The present application provides a system and method for automatically generating a machine learning model. The system generates a model for outputting a quality score to evaluate the accuracy of base calling results, including an input layer for receiving user input, a machine learning layer including a training data generation module and a training module, and an output layer for outputting the model. The training data generation module processes the user input to generate training data including a plurality of pairs of associated features and labels, the features describing base calling probabilities and / or intensities corresponding to a nucleic acid molecule at one or more positions, and the labels including two categories of whether the base calling results are consistent with the corresponding ground truth bases or not. The training module is configured to train a specified model based on part of the training data, including multiple iterations of training the specified model and evaluating the performance of the specified model after each iteration of training to obtain a quality score model. The system can automate and efficiently implement modeling, and can reduce the threshold for model development.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence-based data processing, specifically to systems and methods for automatically generating machine learning models, and more specifically to systems and methods for generating quality score models, or generating base recognition models and corresponding quality score models based on automatic training tools. Background Technology

[0002] The topics discussed in this section should not be considered prior art simply because they are mentioned here. Similarly, the technical problems mentioned in this section or related to the topics provided as background art should not be considered as having been previously recognized in the prior art. The topics in this section merely represent different methods, which themselves may correspond to specific embodiments of the technical solutions included in the claims.

[0003] In related technologies, sequencing generally involves detecting relevant signals to identify the type of nucleotides or bases linked to one or more nucleotide positions in a nucleic acid molecule; that is, determining at least a portion of the nucleotide sequence of the nucleic acid molecule through base calling. Changes in the signal and / or signal intensity corresponding to a specific position of a nucleotide linked to the nucleic acid molecule can indicate the type of base at that position. For example, different nucleotides can be labeled with different fluorescent molecules or optically detectable luminescent tags. In one round of sequencing, multiple nucleotides are contacted with the nucleic acid molecule to be tested, and then the luminescent signals are detected to distinguish the types of nucleotides or bases linked to the template in that round of sequencing. Typical examples include sequencing platforms from ILLUMINA or MGI. Alternatively, optical signal detection and nucleotide incorporation or linking can be separated into independent steps to achieve one round of sequencing, such as the sequencing platform from Element Bioscience; and others include Thermo Fisher's Ion... Torrent sequencing platforms do not rely on fluorescence or optical signals for detection. Instead, they use electrochemical sensors to detect pH changes caused by the release of H+ ions when nucleotides are incorporated into the nucleic acid molecule to identify the type of incorporated nucleotide or base. Other platforms, such as PacBio and Oxford Nanopore, are based on various nanopores for single-molecule sequencing. Compared to the sequencing wheels mentioned above, these platforms provide continuous sequencing. For example, they use zero-mode waveguide (ZMW) nanopores to observe the fluorescence signals emitted during the incorporation of phosphorylated modified nucleotides with fluorescent labels into a single-molecule template (the nucleic acid molecule to be tested) to identify the type of incorporated base in real time. Alternatively, they may not involve biochemical enzyme catalysis processes such as polymerization reactions and directly determine the base sequence of nucleic acid molecules by physically detecting the electrical signals generated when different bases pass through the nanopore.

[0004] Currently, all commercially available sequencing platforms (also known as sequencing systems or sequencers) include base identification software adapted to the specific platform, model, or detection principle. This software identifies the base types in the nucleic acid molecule sequence based on the detection and analysis of relevant signals. Furthermore, each sequencing platform generally includes quality scoring software to quantitatively evaluate the accuracy or reliability of bases or base combinations identified in one or more sequencing runs. Related technologies include, for example, the widely accepted Phred quality Q score (also known as quality score or Q value) [Ewing B, Green P. "Base-calling of automated sequencerraces using Phred. I. Accuracy assessment." Genome Research. 1998; 8(3): 175-185.], with scores such as Q20, Q30, or Q40, expressed as Q = -10log 10 P is logarithmically correlated with the probability of base classification errors. Q value is used to characterize the quality of bases or DNA sequences and can be used to compare the effectiveness of detection results obtained from different sequencing methods or platforms; moreover, the advantage of real-time calculation of quality scores during base identification or detection also includes the ability to terminate defective sequencing runs early, which can significantly save resources.

[0005] In related technologies, machine learning includes various algorithms, such as traditional machine learning algorithms like linear regression, decision trees, and support vector machines, as well as neural networks and deep learning. Applying machine learning to sequencing or gene detection fields, such as developing base recognition algorithms, base recognition result classification scoring, or quality score algorithms using machine learning, has been publicly disclosed in some cases. For example, Wei-Chun Kao, et al.: "naiveBayesCall: An Efficient Model-Based Base-Calling Algorithm for High-Throughput Sequencing", 2010-04-25, RESEARCH INCOMPUTATIONAL MOLECULAR BIOLOGY, SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, pp. 233-247, XP019141683, ISBN: 978-3-642-12682-6; CN118942549A, CN112789680, CN118429967A, CN118429965A, CN118116469A, CN117976042A and US10068053B, etc.

[0006] Machine learning models used in common sequencing or related applications, including the model training, development, and usage processes outlined in the publicly available solutions, often involve significant manual intervention. For example, manual data collection and preprocessing, model selection, hyperparameter tuning, model training, and evaluation are frequently required in the early stages. This manual intervention inevitably leads to the need to configure and optimize machine learning models for different real-world scenarios, resulting in intensive manual operations. Intensive manual operations are generally prone to errors, inefficiency, and management difficulties. Furthermore, this process typically requires personnel with extensive relevant professional knowledge and experience, such as algorithm development experience or a professional background, especially when the machine learning model is complex and involves configuring and optimizing different types of algorithms. Summary of the Invention

[0007] The embodiments of this application are intended to at least solve one of the above-mentioned technical problems or at least provide a practical commercial option.

[0008] This application provides a system for automatically generating machine learning models. The generated machine learning models output quality scores to evaluate the accuracy of base recognition results. The system includes: an input layer for receiving user input from a user, including sequencing results and optionally one or more tasks. The sequencing results are obtained by detecting nucleic acid molecules linked at multiple locations on a surface using surface imaging sequencing technology. The sequencing results include base recognition results and corresponding base recognition probabilities. A machine learning layer includes a training data generation module and a training module. The training data generation module processes the user input from the input layer to generate training data, which includes multiple pairs of associated features. The system comprises a label, wherein the feature description corresponds to the probability of one or more base recognitions and / or the strength of a nucleic acid molecule at one or more positions; the label includes two categories: base recognition results consistent with and inconsistent with the corresponding benchmark true bases; the training module is used to train a specified model based on a portion of the training data to obtain a quality score model, including iterative training of the specified model using the training set and performance evaluation of the specified model after each iteration using the validation set to obtain a quality score model; the training set and validation set are each independent and non-overlapping portions of the training data; and an output layer is used to output a quality score model that meets preset requirements.

[0009] This application also provides a method for developing a quality score model, the method comprising accessing user input using a system for automatically generating machine learning models in any embodiment, and running the system using at least one hardware processor to present the best quality score model to the user, wherein the user input includes sequencing results and optionally one or more tasks, the sequencing results being obtained by detecting nucleic acid molecules linked at multiple locations on a surface using surface imaging sequencing technology, and the sequencing results including base recognition results and base recognition probabilities.

[0010] This application also provides a quality score model generated using a system for automatically generating machine learning models in any of the embodiments.

[0011] This application also provides a computer-readable storage medium storing a plurality of instructions for controlling a processor to execute a method for developing a quality score model in any embodiment.

[0012] This application also provides a system for automatically generating machine learning models, the system including a memory storing programs and one or more processors coupled to the memory, the processors running programs to implement the method for developing quality score models in any embodiment.

[0013] This application also provides a sequencing system or sequencing platform that includes the system for automatically generating machine learning models in any of the above embodiments or examples.

[0014] The automated machine learning process achieved through any of the above-described embodiments of the system, method, or related system platform for automatically generating machine learning models simplifies the modeling process and reduces reliance on professionals such as algorithm developers. This enables non-machine learning experts, such as sequencing platform users without relevant algorithm development background or experience, to develop quality score models themselves to evaluate or compare the accuracy or reliability of base detection results, thereby better meeting the various application detection needs or requirements of users based on sequencing.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] The above and / or additional aspects and advantages of the embodiments of this application will become apparent and readily understood from the description of the embodiments in conjunction with the following drawings, wherein:

[0017] Figure 1 This is a schematic diagram of the structure of the system for automatically generating machine learning models according to embodiments of this application;

[0018] Figure 2 This is a schematic diagram illustrating how the entire image is converted into a group of blocks and the group of blocks is written as a matrix as input data in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of the structure of the system for automatically generating machine learning models according to embodiments of this application;

[0020] Figure 4 This is a schematic diagram of the structure of the system for automatically generating machine learning models according to embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the structure of the system for automatically generating machine learning models according to embodiments of this application;

[0022] Figure 6 This is a schematic diagram of the structure of the system for automatically generating machine learning models according to embodiments of this application;

[0023] Figure 7 This is a schematic diagram of the structure of the automatic generation system for base recognition model according to an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of the structure of the automatic generation system for base recognition model according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the structure of the automatic generation system for base recognition model according to an embodiment of this application;

[0026] Figure 10 This is a schematic diagram of the structure of the automatic generation system for base recognition model according to an embodiment of this application;

[0027] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0029] In this document, unless otherwise stated, the singular forms “a,” “an,” etc., are used to include the plural referents (one or more). “A group” or “a plurality” refers to two or more.

[0030] In this document, terms such as "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number or order of the indicated technical features. In the description of this application, "multiple" means two or more, unless otherwise explicitly stated.

[0031] Unless otherwise stated, the terms "connected" and "linked" as used herein should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components; they can refer to a connection through physical adsorption or other forces, or a chemical connection through chemical bonds, such as incorporation via polymerization reactions. Those skilled in the art will understand the specific meaning of these terms in the relevant examples based on the specific embodiments described, including the context and conventional understanding.

[0032] In this article, "sequencing" refers to nucleic acid sequencing, also known as "nucleic acid sequencing" or "gene sequencing," which involves determining the base sequence of the primary structure of a nucleic acid molecule. This can be achieved using sequencing by synthesis (SBS), sequencing by ligation (SBL), or sequencing by hybridization (SBH). Unless otherwise specified, sequencing by synthesis, as used in this article, includes not only the commonly understood SBS (typically ILLUMINA / Solexa technology), which uses polymerases to catalyze the incorporation of nucleotides into the nucleic acid molecule to be tested (polymerization reaction) and detects the corresponding reaction signals to identify the type of incorporated nucleotides, but also sequencing similar to SBS that uses polymerases or non-polymerases to controllably introduce or ligate nucleotides into the nucleic acid molecule to be tested, and directly or indirectly, simultaneously or sequentially detects the corresponding signals to determine the type of one or more ligated nucleotides. Examples include SBL, SBH, SBB (Sequencing by Binding), or SBE (Sequencing by Expansion), which use surface fluorescence imaging detection for sequencing.

[0033] In the process of sequencing based on solid-phase surface signal detection, sequencing can also be described as measuring the intensity value corresponding to the location of one or more nucleic acid molecules. This intensity value, sometimes simply referred to as intensity, can be the intensity of any signal, such as electrical or electromagnetic radiation like visible light. A single base can have one intensity value, or multiple bases can have one or fewer intensity values. Furthermore, the intensity value can be specific to a location on the solid-phase surface or to multiple locations within a nucleic acid molecule. The intensity value can be converted or mapped to a predetermined numerical value (e.g., an integer in a binary or decimal number system), a continuous numerical value, or a range.

[0034] Sequencing can be performed using a sequencing platform. According to the embodiments of this application, the sequencing platforms that can be selected include, but are not limited to, Illumina's HiSeq, MiSeq, Nextseq and Novaseq series sequencing platforms, Thermo Fisher / Life Technologies' Ion Torrent platform, BGISEQ and MGISEQ / DNBSEQ platforms from BGI Genomics, Element Bioscience's AVITI and single-molecule sequencing platforms; the sequencing method can be selected as single-end sequencing, paired-end sequencing, or sequencing methods supported by the selected automated sequencing platform.

[0035] In this application's embodiments, "sequencing process" or "sequencing run" refers to the batch determination of intensity values ​​corresponding to the locations of one or more nucleic acids. For example, in a sequencing scenario involving multiple cycles (also called wheels or sequencing rounds) of imaging nucleic acid molecules on a substrate, such as a solid substrate surface, after a specified biochemical reaction, a series of intensity values ​​are obtained during the same sequencing run. Generally, nucleic acid molecules from the first sequencing run will not participate in a second sequencing run independent of the first sequencing run; for example, they will not be included in an image acquired at a certain time point (a certain round) of the second sequencing run.

[0036] In some examples, sequencing-by-synthesis (SBS) is used to perform multiple rounds of sequencing to obtain sequencing sequences or reads. For instance, the nucleic acid molecule to be tested is brought into contact with a polymerase and a modified nucleotide and placed under suitable conditions for polymerization. The modified nucleotide is controllably incorporated into the nucleic acid molecule to be tested, or in other words, single-base extension is controllably achieved. The corresponding reaction signal is detected, and the type of nucleotide incorporated into the nucleic acid molecule in that reaction is determined based on this signal. This process of controllable single-base extension and corresponding signal detection is repeated multiple times to detect the type of nucleotide or base incorporated into the nucleic acid molecule to be tested in multiple or multiple rounds of reactions, thereby reading a portion of the sequence of the nucleic acid molecule to be tested.

[0037] The nucleic acid molecule to be tested, also known as the nucleic acid template or template, can be an unamplified single molecule or an amplified molecular cluster or long chain containing multiple identical polynucleotide molecules, such as fluorescent clusters or DNA nanospheres (DNBs) formed by bridge amplification or rolling circle amplification used by current mainstream commercial sequencing platforms. The nucleic acid molecule to be tested can be single-stranded, double-stranded, and / or a triple-stranded or more-stranded complex hybridized with probes or primers.

[0038] The corresponding reaction signals can be, for example, fluorescence signals, which can be represented as image data (e.g., color or grayscale images) formed by collecting these fluorescence signals. These image data are then processed and analyzed to detect the nucleotides incorporated into the nucleic acid molecule to be tested in each or each round of reactions, so as to determine a portion of the base sequence of the nucleic acid molecule to be tested.

[0039] Specifically, in some examples, sequencing is achieved based on surface fluorescence imaging detection. The nucleic acid molecule to be tested is attached to a solid surface. For example, the nucleotide can be modified to have or be able to bind a fluorescent label, as well as a removable inhibitor group that prevents other nucleotides from polymerizing and attaching to the next position of the nucleic acid molecule to be tested (this type of modified nucleotide is also called a reversible terminator). After each polymerization reaction or single base extension reaction, the fluorescent label is excited to emit light, and these emission signals are collected to obtain an image of the nucleic acid molecule to be tested that has undergone a single base extension reaction at a specified surface position. Then, the inhibitor group and fluorescent label are removed to perform the next polymerization reaction and signal acquisition (photographing). This process of polymerization reaction-photographing-removal is repeated multiple times or multiple rounds to obtain image information related to the nucleotides attached to the nucleic acid molecule to be tested in each single base extension reaction.

[0040] Understandably, when a nucleic acid molecule to be tested undergoes a polymerization reaction at a designated location on the surface, it emits fluorescence. This fluorescence typically appears as a bright spot or bright patch with a higher intensity than the background signal at the corresponding location in the image acquired in that round of reflection. Therefore, based on the information contained in these image sets corresponding to specific chemical characteristics (the nucleic acid molecule to be tested undergoing a polymerization reaction), such as intensity and / or morphology, it is possible to determine whether a nucleic acid molecule to be tested exists at the designated location and whether it has undergone a polymerization reaction. Furthermore, by combining the pre-defined correspondence between the fluorescence emission signal and the nucleotide type, the type of nucleotide that has undergone a biochemical reaction and is linked to the nucleic acid molecule to be tested can be detected. This allows for the determination of at least a portion of the sequence of the nucleic acid molecule to be tested, obtaining what is known as a read.

[0041] The set of locations corresponding to chemical features (nucleic acid molecules to be tested) on a surface is determined based on an image, i.e., template acquisition. This can be determined by identifying features on the detection image that correspond to the chemical features, such as bright spots (locations), or by identifying other features on the image that have a specific spatial relationship with the chemical features. For example, for a regular array surface containing labeled regions (also often called tracer regions), the spatial relationship between the labeled regions and the reaction regions (the areas where the nucleic acid molecules to be tested are located) is generally known beforehand, and the signals from the labeled regions are clearly distinguishable from the signals from the reaction regions, or the signals from the labeled regions appear as detectable features on the image. By identifying the signals from the labeled regions and determining the location of the labeled regions on the image, the location of each amplicon (nucleic acid molecule to be tested) in the reaction region can be determined.

[0042] It should be noted that the term "nucleotide" as used herein includes ribonucleic acid or deoxyribonucleic acid, including natural nucleotides or their derivatives or modifications thereof (also called modified nucleotides or altered nucleotides, etc.). In this document, the term "nucleotide" is sometimes used to refer to the bases contained within it, which will be readily understood by those skilled in the art based on conventional understanding and / or context. Furthermore, the terms "nucleic acid sequence," "nucleotide sequence," "oligonucleotide sequence," or "base sequence" as used herein are interchangeable unless otherwise specified, and are sometimes simply referred to as "sequence." Additionally, sometimes one or more nucleotides or bases are also referred to as a sequence, which will be readily understood by those skilled in the art based on conventional understanding and / or context.

[0043] In some embodiments, a single round of sequencing may include one base extension reaction (one repeat). For example, four different nucleotides (dATP, dTTP, dGTP, and dCTP) may be placed in the same polymerization reaction system with multiple nucleic acid molecules to be tested, so that each nucleotide can be excited to emit a signal that is distinguishable from other types of nucleotides. Thus, the type of nucleotide incorporated or introduced at a position in any nucleic acid molecule to be tested can be determined by the information obtained from one repeated reaction. For example, the four nucleotides may be labeled with four different fluorescent bands for four-color or four-channel single-molecule sequencing or high-throughput sequencing; or, for example, the four nucleotides may be labeled with three different fluorescent bands and unlabeled (cold nucleotides) for two-color or three-color high-throughput sequencing. One round of sequencing includes one repeat, and one round of sequencing can detect the type of base at a position on any nucleic acid template.

[0044] The term "amplifier" refers to the amplified nucleic acid molecule to be tested. An amplifier is a cluster, chain, or group containing multiple identical polynucleotide sequences. The term "fluorescent cluster, group, or sphere" refers to an amplifier that can be excited to emit light, such as fluorescence, after undergoing a specified biochemical reaction. During sequencing, amplifiers can emit light and be detected by imaging. By processing the signals acquired during imaging and / or the corresponding image information, bases incorporated into or linked to the nucleic acid molecule to be tested after the specified biochemical reaction can be detected, thereby determining at least a portion of the sequence of the nucleic acid molecule to be tested.

[0045] In the embodiments of this application, unless otherwise stated, the terms amplicon, fluorescent cluster or sphere, nucleic acid molecule cluster, clone cluster, nucleic acid molecule to be tested, template, nucleic acid template, template spot, bright spot corresponding to chemical features or fluorescent clusters on the surface, etc., are generally used interchangeably. The terms signal intensity, intensity, brightness, and fluorescence brightness, etc., are also generally used interchangeably. The terms surface, chip, array, etc., are generally used interchangeably in the embodiments of this application.

[0046] The machine learning model referred to in the embodiments of this application, also simply as a model, refers to a determination technique that predicts the type of an output base or base combination as correct or incorrect, or as a quality score, based on known results (training data). In some embodiments, it also includes a determination technique for predicting the type of an output base or base combination. The quality score characterizes the accuracy, error rate, or reliability of the predicted base or base combination result. The known result can be a sequence or base that is considered or assumed to be accurate, that is, it is assumed that the sequence or base is correct. In some embodiments, the assumed accurate sequence or base is the result predicted by the expected model, sometimes also referred to as the corresponding benchmark true base. Supervised learning of the model can be performed using training data.

[0047] In the machine learning embodiments of this application, the concepts of label or target value are generally synonymous, referring to the result that the model aims to predict. The bases or base sequences involved in the labels or target values ​​in machine learning are assumed to be accurate sequences (bases or base combinations), considered accurate sequences, and sometimes referred to as the corresponding true bases. The corresponding bases in the labels included in the training data may be inaccurate, but the model assumes they are accurate during training. The assumed sequence can be determined in various ways, as described in the embodiments of this application: using a reference sequence, such as the bases or base combinations corresponding to the positions of sequencing results aligned to the reference sequence; using sequences determined or verified by generally considered high-accuracy detection methods or means, such as Sanger sequencing and / or DNA synthesis; using consistent results from multiple detection methods or means; or using sequences or base combinations with a high probability of accuracy, such as bases with a quality fraction reaching or exceeding Q40, or Q40 reaching or exceeding 80, 85, or 90. Here, Q40 represents an error rate or probability of error of one in ten thousand. If the training model outputs predictions with high accuracy or higher confidence, which is not the desired goal, the measurement results with low accuracy can be used as the assumed sequence and as training data. However, it is understandable that the accuracy of the prediction results of the model learned from this is generally not high.

[0048] This application involves learning a quality scoring algorithm, quality scoring model, or related model. Based on the trained quality scoring model, the accuracy or reliability of the base determination results is scored to quantitatively assess or evaluate the accuracy or reliability of such base determination results. This can be used to compare the sequencing quality of different sequencing methods or sequencing platforms, and to determine whether the accuracy of the bases or sequences identified in one or more sequencing runs or one or more sequencing runs meets expectations, and whether to continue the sequencing run, etc.

[0049] The base determination referred to in the relevant embodiments, also known as base identification, base category identification, base detection, base category detection, base readout, or base interpretation, refers to the determination of a base or combination of bases at a specified or unspecified position in a nucleic acid molecule. Base determination can be performed independently or as part of a specified base such as A or T, or a specified base combination such as ACT or GTCA. Base determination can be performed for the same genomic location (e.g., multiple bases detected at that location have similar scores or probabilities) or for different locations. The base type at a specified or unspecified position can be determined using the score output from a machine learning model. For example, a score can be provided or assigned to each identified base or combination of bases, and base determination based on such scores can be part of the model's prediction. Furthermore, some models can provide scores, which can be used in subsequent processes. The score or value can reflect probability or likelihood. The probability score for each base or base combination is usually summed to a fixed value, such as 1; while the score reflecting likelihood generally does not need to be summed to a fixed value, although it can be summed to a fixed value, for example, by limiting the likelihood score for each to between 0 and 1, so that the likelihood score can also be summed to 1. In the embodiments of this application, the numerical values ​​or scores that reflect the probability or likelihood of each base or base combination are provided or assigned are sometimes collectively referred to as base identification probability. In related embodiments, the base category in the base identification result is often simply referred to as a base.

[0050] Please refer to Figure 1Some embodiments provide a system 100 for automatically generating machine learning models. The machine learning models generated by this system 100 are used to output quality scores to evaluate the accuracy of base identification results. The system 100 includes: an input layer 120 for receiving user input from a user, including sequencing results and optionally one or more tasks. The sequencing results are obtained by sequencing nucleic acid molecules linked to multiple locations on a surface using surface imaging sequencing technology. The sequencing results include base identification results and corresponding base identification probabilities. A machine learning layer 140 includes a training data generation module 142 and a training module 144. The training data generation module 142 processes the user input from the input layer 120 to generate training data. The training data includes multiple pairs of associated features and labels. The features describe the strength of a nucleic acid molecule at one or more locations and / or the recognition probability of one or more bases. The labels include two categories: base recognition results matching and not matching the corresponding ground truth bases. The training module 144 is used to train a specified model based on a portion of the training data to obtain a quality score model. This includes iteratively training the specified model using the training set and evaluating the performance of the specified model after each iteration using the validation set to obtain the quality score model. The training set and validation set are each independent, non-overlapping portions of the training data. The output layer 160 outputs a quality score model that meets preset requirements. In related embodiments, the automatically generated machine learning system 100 is also referred to as an automatic training tool.

[0051] In related technologies, the quality score or quality rating (Q-value) of base identification / classification results, regardless of the calculation method, rules, or custom rules used by different sequencing platforms from various companies, can generally be considered another quantitative representation of the probability of a correct or incorrect base detection result. It is an indicator of the accuracy or reliability of sequencing results. A higher Q-value indicates higher accuracy and reliability of the sequencing results. The Q-value is usually calculated based on the Phred algorithm, and the formula can be written as Q = -10 * log0 10The Q value (err) refers to the probability of an error in base identification, i.e., the error rate or probability of the base detection result. For example, common Q values ​​like Q20, Q30, and Q40 represent error rates of 1 percent (99% accuracy), 1 / 1000 (99.9% accuracy), and 1 / 10000 (99.99% accuracy), respectively. The Q value provides a direct understanding of the accuracy and reliability of sequencing data. Furthermore, in various practical applications that utilize sequencing data or results for detection, such as non-invasive prenatal screening for genetic variations, liquid biopsy for early tumor screening, and pathogen detection, Q values ​​are typically used to process sequencing data or set quality standards to ensure high-quality sequencing data or meet specific requirements, thereby guaranteeing the accuracy and reliability of the application's detection results.

[0052] This application does not limit the method by which the sequencing results received by the user input in the input layer 120 are obtained. The sequencing results are obtained by sequencing and detecting nucleic acid molecules connected to multiple locations on the surface through surface imaging sequencing technology, including base recognition results and corresponding base recognition probabilities. In some embodiments, the sequencing data comes from a sequencing platform that performs sequencing based on surface imaging detection. Most of the current mainstream high-throughput sequencing platforms, such as sequencers from companies like ILLUMINA, BGI Genomics, and Element Biosciences, are platforms that determine the sequence of nucleic acid molecules on the surface based on surface imaging (e.g., surface fluorescence imaging). This is achieved by converting relevant biochemical reaction signals into electrical signals, such as image data, and then determining the base arrangement order (base recognition result) of the nucleic acid molecule to be tested based on the processing and analysis of the image data. In addition, much of the sequencing data disclosed or provided by sequencing platform manufacturers or user institutions is also derived from sequencing platforms based on this principle. Sequencing results can be data from one or more nucleic acid samples, obtained through one or more library constructions, one or more sequencing runs on one or more sequencing platforms, and optionally include intermediate data, such as detected sequences, detected bases, the probability / attribution of each base, and optionally related image data or other forms or formats of data. Sequencing results can be obtained by the user themselves, provided by others, or available for download or access from relevant data platforms. They can also be a mixture or combination of sequencing data generated by the user and others from one or more runs on one or more sequencing platforms.

[0053] In some embodiments, the sequencing results provided by the user input can be in the form of access links or paths to reduce storage requirements and facilitate system 100 in receiving and connecting to read the data. The model output by the output layer 160 can also be in the form of access paths or links, for example, it can be retained or saved in a specified path for easy access and execution.

[0054] In some examples, the so-called user input or sequencing results also include alignment results of the sequence or base detection in the sequencing results to a reference sequence, such as the bases or base combinations corresponding to specific positions in the reference sequence that are aligned. The reference sequence is a known sequence, which can be a reference genome, a portion of a reference genome, or a known sequence formed by recombination and reorganization of one or more reference genomes. In some related examples, the bases or base combinations on the reference sequence corresponding to the alignment position are used as the corresponding ground truth bases in the labels of the training data.

[0055] In some embodiments, the sequencing results include at least 10 million sequence reads and / or at least 1 billion base pairs. This generates sufficient training data to potentially generate a model with better predictive performance based on the system 100.

[0056] The training data generation module 142 is used to process user input from the input layer 120 in order to generate training data, which includes multiple pairs of associated features and labels, in order to supervise the training of a specified model based on the training data.

[0057] Methods or sequencing platforms based on surface imaging detection involve the use of base recognition algorithms, software, or tools to identify bases or their sequence at a specific location in a nucleic acid molecule. These algorithms, software, or tools typically involve comparing the probability of identifying a location as one of the four bases (AT, C, G) or using this probability as a basis for determining the identification result. For example, based on analysis of signal intensity at locations corresponding to biochemical features in an image, if the probability of identifying a location as A is 10%, as G is 10%, as C is 10%, and as T is 70%, then the base at that location is very likely to be identified as T. This is especially true if the relative probabilities of these four bases at that location represent the final result (the signals corresponding to each base have already been corrected or will not be adjusted further). For example, based on the processed image information, including crosstalk correction and / or phase correction of relevant chemical features, if the probability of a certain position being identified as A is 22%, as G is 28%, as C is 24%, and as T is 26%, then given that there is no significant difference or the difference is less than a preset level between any bases identified at that position, the base at that position is very likely to be identified as N or assigned as N, where N is selected from one of A, T, C, and G, or N refers to an uncertain one of A, T, C, G, and G.

[0058] In some embodiments, the referred features describe the probability of identifying one or more bases. For example, the probability values ​​of all base categories, the probability value of the base category with the highest probability, and the probability values ​​of the base categories with the highest and second-highest probabilities. The base category referred to here can be a base category formed by a single A, T, C, or G base in each base identification result, or a base category formed by a single base combined with two or more bases, such as AT, ACG, etc.

[0059] Specifically, for example, the feature can be the probability value of identifying each base class, including base combination classes (probability of identifying all types of bases), or the probability value of the base class with the highest probability, the probability values ​​of the base classes with the highest and second highest probabilities, the probability values ​​of the base classes that are ranked in descending order of numerical value, the probability values ​​of the base classes that are ranked in descending order of numerical value, or any one or a combination of the above, or a processed or modified probability value, etc. The probability or probability value referred to in the relevant embodiments can be an absolute value, a relative value, a ratio, a modified value after normalization or standardization, or a value amplified or reduced based on a certain rule. In a specific example, the feature describes the probability of each base identification result corresponding to multiple base classes.

[0060] In other embodiments, the user input or the sequencing result input by the user also includes intermediate sequencing data or parameters such as sequencing images, including processed sequencing images or a portion of information from sequencing images, whereby the so-called features describe the intensity corresponding to one or more nucleic acid molecules at one or more locations. Alternatively, the so-called features may include both a description of the intensity corresponding to one or more nucleic acid molecules at one or more locations and a description of the probability of at least one base class (base recognition probability).

[0061] Specifically, in some examples, intermediate sequencing data includes images of nucleic acid molecules at multiple locations on one or more surfaces collected from one or more sequencing runs. The term "image" or "sequencing image" includes images directly acquired during sequencing, images acquired after preprocessing, or images generated by reconstruction or recombination based on images acquired during sequencing. In this embodiment, these images are sometimes referred to as "raw images." Subsequently, the training data generation module 142 processes this image data, for example, extracting partial information from the images, such as the position and intensity of each nucleic acid molecule, as feature values.

[0062] In other examples, intermediate sequencing data includes a subset of data extracted from sequencing images, including the positions and intensity values ​​of multiple nucleic acid molecules on one or more surface regions. The sequencing image is an image of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more sequencing runs. Specifically, user input includes information obtained after image processing, containing information about the nucleic acid molecules or amplicones in the image that represent or reflect the target signal, i.e., the nucleic acid molecules or amplicones that have undergone a specified biochemical reaction, such as the position and intensity values ​​of the amplicon. The size of this input data is smaller than the size of its corresponding sequencing image. This size, also referred to as file size, dimensions, or data volume, reflects the storage space required or occupied by the data. Thus, compared to inputting a sequencing image, this input can be directly used as feature values ​​or directly as part of the training data, giving the system significant advantages in terms of computational cost and speed for generating models.

[0063] Specifically, intermediate sequencing data or parameters include the position and intensity information of amplicones on the image, and the size of this data is smaller than the size of its corresponding image. Therefore, from a certain perspective, the input data can be considered compressed image information or data after extracting or filtering image information. It can be represented in a simplified image form or a non-image form, preferably in a data form that is easy for a computer or processor to process. In a preferred example, it is represented in a non-image data form, such as a matrix, multidimensional matrix, array, or multidimensional array, recording or presenting only the relative position and intensity information of the amplicones on the image. This significantly reduces the file size of the input data compared to the original input image, lowering the requirements for computing power and / or storage, and facilitating rapid computation or processing by the computer to quickly obtain prediction results.

[0064] More specifically, in some examples, the size of the input data is less than or equal to half the size of its corresponding image. In one example, particularly for regular surfaces, the file size of the input data is less than or equal to one-third the file size of its corresponding image. In some tests, especially for high-density regular surfaces, the size of the input data is even less than one-quarter or one-fifth the file size of its corresponding image. Thus, compared to directly inputting image data, storage or space usage is significantly reduced, and running speed is significantly improved.

[0065] Specifically, for example, intermediate sequencing data can be represented as multiple matrices. Each matrix contains multiple submatrices, reflecting the position and intensity information of the amplicon (the nucleic acid molecule to be tested) on the images acquired in a single sequencing run. A single matrix contains multiple rows and columns, reflecting the information of the amplicon in all or a portion of a fluorescence channel image in a single sequencing run. Each element of the matrix corresponds to one amplicon, and the row and column positions of the matrix elements reflect the position information of the corresponding amplicon in the corresponding image. The values ​​of the matrix elements reflect the intensity information of the corresponding amplicon presented in the fluorescence channel image of that sequencing run. This matrix is ​​sometimes also called a fluorescence intensity matrix. In this way, compared to the input image, the amount of information or size of the input data is greatly reduced, which helps reduce the requirements for storage or computing power and improves the speed of computation. Furthermore, this input data can be directly used as feature values ​​or part of feature values, reducing subsequent calculations.

[0066] For example, a single intermediate sequencing data point can correspond to multiple amplicons in a region or field of view (FOV), and a single data point, such as one or a set of fluorescence intensity matrices, can correspond to the position and intensity information of amplicons that have undergone a specified reaction in an image acquired during a single round of sequencing in a field of view. Thus, compared to the original input image, the size of the input data is significantly reduced, lowering the requirements for computing power and / or storage.

[0067] Regarding the number of matrices and the dimensions of a set of matrices, it can be understood that a set of matrices corresponding to a set of images from a single round of multi-fluorescence channel, FOV sequencing, such as the four fluorescence intensity matrices of amplicons from four fluorescence channels in a single round of sequencing from four-color sequencing with one FOV, the three fluorescence intensity matrices of amplicons from three fluorescence channels in a single round of sequencing from three-color sequencing with one FOV, and the two fluorescence intensity matrices of amplicons from two fluorescence channels in a single round of sequencing from two-color high-throughput sequencing with one FOV, can all be referred to as a three-dimensional matrix. Based on conventional understanding, those skilled in the art, according to the specific embodiments described, will understand the specific meaning of the descriptions involving matrices, the number of matrices, or dimensions.

[0068] Furthermore, in some specific examples, the advantages of the input matrix compared to the original input image go beyond just the difference in image and matrix size. For instance, in a single round of sequencing for four-color fluorescence imaging, assuming each image is 2000*2000 pixels, the template point matrix (fluorescence brightness matrix) reflecting the position and intensity information of the amplicon in that image, written in matrix form, is 800*800 pixels. This represents a size reduction of 0.4^2 times the original. Moreover, the brightness information corresponding to these 800*800 template points in the input matrix can be the brightness information corresponding to sub-pixel level positioning, such as brightness obtained through bilinear interpolation or other interpolation methods. Therefore, in terms of information content, this 800*800*4 template brightness matrix reflects the information of the four images: a 2000*2000*4 matrix plus an 800*800 position matrix. In other words, the information content of the input matrix reflects the position and intensity information of the amplicon, determined directly or indirectly based on image information, including information from the processed image. In this way, the input data not only contains target information and its size is significantly reduced, but it is also presented in a matrix form that is easy for machines to run and compute, further facilitating rapid model training. Moreover, this input data can be directly used as feature values, or as part of the features or training data, to reduce subsequent computations.

[0069] In other specific examples, the intermediate sequencing data or sequencing results input by the user also include data in matrix format. A matrix reflects the information of amplicon amplicons in a block of a fluorescence channel image during one round of sequencing. The input data also includes a matrix reflecting the location information of each block in the image from which it originated. Thus, by transforming the matrix corresponding to a fluorescence channel image into a set of relatively small matrices, or in other words, by transforming a fluorescence channel image into a set of blocks, extracting the amplicon information from simplified blocks into matrix form, and processing these relatively small matrices in parallel, the processing speed of input data can be further improved under the same hardware and software computing environment, enabling faster model training.

[0070] Please refer to Figure 2 In a specific example, an image is converted into a set of block matrices. Block matrices of different sizes are all zero-padded to the same size and then used as input. Therefore, the entire image is converted into a total of 3*3, or 9 blocks. Since the length and width of the blocks may be different, after splitting into 9 blocks, the edges of all blocks smaller than b*y can be padded with 0 values ​​to the size of b*y. Finally, an additional position encoding layer is added according to the position number of the block in the whole image. For example, the block number in the lower right corner is 9, so a Pos(P) position encoding layer is filled with all values ​​of 9 to obtain the final input matrix.

[0071] The so-called labels include two categories: those whose base identification results match and those whose bases do not match the corresponding benchmark true bases. Specifically, as mentioned in the relevant embodiments, the base identification results can be base categories formed by a single base of A, T, C, and G, or A, T, C, G, and N; they can also be base categories formed by combinations of two or more bases, such as AT, ACG, etc.; or they can be base categories formed by a single base and combinations of two or more bases.

[0072] The corresponding true base is the base category that has been identified as correct. In some examples, the corresponding true base category comes from sequences of human and non-human samples that have been consistently verified and fully characterized by multiple sequencing instruments, sequencing chemistry principles, and sequencing protocols. Such a category can be considered correct or true, or, in other words, the accurate and reliable base category currently confirmed by the relevant detection methods. In other examples, the corresponding true base is the base or base combination at the corresponding position in the reference sequence when the sequencing result is aligned to the reference sequence.

[0073] In one example, the user input includes the corresponding benchmark true base. In other examples, the user input includes a reference sequence, such as providing access to or a link address to the reference sequence, so that the training data generation module 142 can process the user input to generate training data, including aligning the sequencing results to the reference sequence and using the bases or base combinations on the corresponding reference sequence as the corresponding benchmark true base categories.

[0074] In some embodiments, training a specified model in training module 144 further includes: obtaining the classification probabilities of base recognition results in the training set; dividing the base recognition results into groups a based on the classification probabilities and / or the base categories in the base recognition results, where a is a natural number not less than 5; for each group's classification probabilities and / or corresponding base recognition results, determining the error rate or accuracy rate of the base categories in the group's base recognition results by comparing the base categories in the group's base recognition results with the corresponding benchmark ground truth bases; and associating a predetermined quality score with the error rate or accuracy rate of the base categories.

[0075] Specifically, the base identification result can be used to obtain the so-called classification probability P value, denoted as P0, through the quality score model or algorithm of any of the above-mentioned related embodiments. P0 is a probability value between 0 and 1. Further, P0 can be divided into a segments, and the base categories in the base identification result will be correspondingly divided into these a segments or groups, where a is a natural number not less than 5. In some specific examples, P0 is divided into a segments in an equal manner. For example, P0 is divided into 5 segments, and the ranges of these 5 segments of P0 are [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), and [0.8, 1.0], respectively.

[0076] Furthermore, for each group, the base categories in the base identification results are compared with the corresponding benchmark true bases to determine the error rate or accuracy rate of the base categories in that group. This includes identifying or marking bases in the group that match the corresponding benchmark true bases as correct, and identifying or marking those that do not match as incorrect. In this way, the accuracy rate or error rate of the base categories in that group can be calculated.

[0077] In related examples, the Q value corresponding to the logarithm of the true error rate of a group or segment of bases is called the true Q value. For example, as mentioned in the above embodiment, the base categories in the base identification results are divided into segments 'a' based on the P0 value. Then, the total error rate of the bases in each segment can be calculated, and the corresponding Q value can be calculated based on this total error rate. This total error rate is called the true error rate, and the corresponding Q value is called the true Q value. Specifically, following the above example, all bases (base categories) with P0 in the range [0, 0.2) are identified. Using the total number of bases in this segment as the denominator and the number of bases in the base identification result that are different or inconsistent with the reference answer (benchmark true base) as the numerator, the total error rate of the base category corresponding to P0 in the segment [0, 0.2) is calculated, and the logarithm of this total error rate is taken to convert it into a Q value. In this way, the true Q value of the base category corresponding to this segment is determined. Furthermore, this true Q value can be assigned to each base in this segment, thus determining the true Q value or true error rate of each base category in this group or segment.

[0078] In other embodiments, training a specified model in training module 144 further includes: obtaining the classification probabilities of base recognition results in the training set; dividing the base recognition results into groups a based on the classification probabilities and / or the base categories in the base recognition results; for each group, determining the error rate of the base categories in the base recognition results of that group by comparing the base categories in that group with the corresponding benchmark true bases, where a is a natural number and the value of a is such that the logarithmic difference in the error rates of the base categories in the base recognition results of adjacent groups is less than 2; and associating a predetermined quality score with the error rate of the base categories.

[0079] In some preferred examples, the setting or adjustment of the value of 'a' follows these principles: After dividing P0 into 'a' segments or 'a' groups, the difference in the true quality values ​​(true Q values) of the base categories corresponding to adjacent segments should be small. Specifically, a difference of less than 2 in the true quality values ​​of the base categories of adjacent segments can be considered small; more specifically, a difference not greater than any value in the range [0.5, 2) can be considered "small," such as a difference less than or not greater than 0.5, 0.8, 1, 1.1, 1.2, 1.5, etc. More preferably, a difference of less than 1 in the true Q values ​​of base detection in adjacent segments is considered small. If this preset limit is exceeded, P0 can be re-subdivided or further subdivided, i.e., a larger value of 'a' can be used. Following this principle, multiple tests are performed, and the value of 'a' is usually much greater than 10. The inventors have synthesized the quality requirements of sequencing data for various testing applications and tested and verified sequencing data obtained from sequencing various samples, including human and non-human samples, using sequencing platforms based on different sequencing principles or from different vendors, to establish optimal principles or related settings or value rules. This facilitates the rapid and automatic generation of optimal models or models that meet user input tasks. For example, the system 100 can determine a suitable 'a' or a principle-compliant 'a' through iterative calculations.

[0080] Compared to the aforementioned examples where the bases (base categories) in the base identification results are directly determined by segmenting the classification probability P0, other specific examples first classify the base categories in the base identification results. This classification can be based on the current (round) base identification type, the previous n (rounds) base identification types, the next m (rounds) base identification types, or any two or all of these three combinations, where m and n are independent natural numbers. For example, classification can be based on the currently identified base categories such as A, C, G, and T; or based on the current (round) bases and the base categories identified in the previous or next round, such as CC, AA, AT, CG, etc.; or based on the current (round) bases and the base categories identified in the previous and next rounds, such as ATC, TAT, TTT, GCA, etc.; or based on any two or more of the aforementioned combinations, etc.

[0081] In examples where bases are first classified from the base identification results, the classified base categories are often directly referred to as base categories or bases. If this might cause ambiguity or confusion, the classified base categories are sometimes referred to as new base categories. The classification probability of the classified base category can be the classification probability of the current (round) base category, for example, still P0 in the aforementioned related embodiments, or a modified P0. For example, P0 can be modified or corrected based on the classification probability or base identification probability of non-current bases in the new base category, such as preceding and / or following bases. The mapping relationship between P0 and the Q value can also be modified. Specifically, for example, if the classification probability of non-current bases in the base category is relatively low, for example, all less than 0.8, it can be considered to some extent that the current base category is "dragged down" by one or more preceding or following relatively unreliable bases, thus affecting the classification probability of the base category. If the reliability of a base class decreases, its P0 value or the mapping relationship between P0 and Q value can be adjusted. For example, multiplying it by a coefficient less than 1, such as 0.9, to update P0, or subtracting 1 or 2 from the Q value corresponding to P0, or assigning it to the adjacent lower-level segment Q value, can correct the classification probability of that base class or the mapping relationship between the classification probability of that base class and Q value. Alternatively, bases with relatively low classification probabilities (relatively unreliable bases) in that base class can be removed to update the base class, and the updated base class can replace the original base class in subsequent analysis processes. Or, the base class can even be directly removed, so that the base classification does not participate in subsequent processes. This helps to establish or generate a more accurate correspondence.

[0082] Furthermore, the subsequent processing of this method, which first classifies the base categories in the base identification results, can be similar to the methods or approaches in the previous examples. For instance, it can involve segmenting the bases based on their classification probabilities P0, calculating the true error rate and / or true Q-value for each segment, and statistically analyzing or clarifying the number or distribution of base categories with each true error rate and / or true Q-value, in order to establish a correspondence between each classification probability and a predetermined quality score. The processes or methods and technical effects of the aforementioned related examples are also applicable to processing such new base categories and associated data, and will not be elaborated upon here.

[0083] The predetermined quality score refers to the predetermined Q-value segmentation requirements. This could be a parameter that the user can set and change themselves, such as a parameter that System 100 can automatically select or adjust based on the user's input task. Currently, Q-values ​​calculated using the Phred algorithm or output by various sequencing platforms commonly come in two forms: continuous integers and non-contiguous integers. Specifically, predetermined output Q-values ​​can be continuous integers, such as all integers in the range [0,40], [0,45], or [0,50]. Predetermined output Q-values ​​can also be non-contiguous integers, also known as Q-score binning, which divides continuous Q-values ​​into several bins. For example, Q-values ​​can form an arithmetic sequence or consist of non-contiguous integers without a clear pattern, such as 0, 10, 20, 30, 40, and 50, or 0, 10, 17, 21, 28, 35, 40, and 50. Both continuous and discontinuous Q-values ​​are widely used in current commercially available sequencing platforms, with discontinuous Q-values ​​being more prevalent. Discontinuous Q-values ​​can simplify data processing, improve robustness, and facilitate visualization, significantly enhancing the efficiency and stability of data analysis. However, discontinuous Q-values ​​may also lead to the loss of some information. In specific applications, the appropriate or desired presentation of Q-values ​​generally requires a trade-off between the advantages and disadvantages.

[0084] In a certain example, the predetermined quality score matches the actual Q value in the relevant example above. That is, the actual Q value meets the predetermined quality score value requirement. Thus, the established correspondence between the classification probability of each base category and the predetermined quality score constitutes the establishment of the so-called quality score model.

[0085] In other examples, the predetermined quality score is a set of scores that differ from the true Q-values. Typically, it is necessary to further update these correspondences or mappings based on the established correspondence between the classification probabilities of bases or groups of bases and the true Q-values, combined with the quantity (e.g., percentage or distribution) of each base representing the true Q-value / true error rate, and the new Q-value requirements. This is to establish or update the correspondence, i.e., to establish or update a quality score model that conforms to the user's input task or requirements.

[0086] Specifically, in some specific examples where the predetermined quality score differs from the true Q value and the true Q value includes the predetermined quality score, the so-called association between the predetermined quality score and the error rate of the base category includes, based on the error rate or correct rate of the base category in the base identification results of each group, the proportion or distribution of the number of base categories in each group, and the predetermined quality score requirement, merging the groups into new groups with fewer groups, adjusting the attribution of the base categories of adjacent new groups, and calculating the error rate of the base category of each new group, so that the error rate of the base category of the final new group corresponds to the predetermined quality score, thereby establishing a mapping relationship between the classification probability of the base category and the predetermined quality score.

[0087] Understandably, the so-called predetermined quality score, i.e., the predetermined Q-value requirement, is equivalent to predetermining the number of new segments and the error rate of each new segment. By merging one or more adjacent segments into fewer new segments, and adjusting the base assignments of adjacent new segments (equivalent to adjusting the proportion or distribution of base classes in each segment), and calculating the error rate of the base class in the new segment, the error rate of the segment calculated based on the P0 of the bases in each new segment is ultimately the predetermined error rate of that segment (the error rate of the new segment or the new Q-value). In this way, a mapping relationship is established between the classification probability of each base class and the predetermined quality score, thus establishing or generating the so-called quality score model.

[0088] Specifically, in a certain example, based on the statistics of the training data, including the true Q value (true error rate) of each base (base class) in each group or segment, the distribution or distribution trend of the number or proportion of bases in each group or segment, and the new quality score requirement (predetermined quality score), the bases of the original groups or segments are merged and regrouped or segmented, and the error rate of the bases in each segment after regrouping or segmentation (the true error rate of the bases in the segment after resegmentation) is made consistent with the predetermined quality score, so as to establish a new correspondence between each group of bases and the predetermined quality score, so as to update and generate the desired quality score model.

[0089] In System 100, the process of merging one or more adjacent segments into fewer new segments, allocating bases within these new segments, and calculating the error rate of the corresponding new segments can be achieved through iterative calculations. For example, if the intention is to divide segments of consecutive integers 0-50 (each integer representing a segment) into four new segments of 10, 20, 30, and 40, the initial division might result in a true Q value of 11 for the new segment 10 and 19 for the new segment 20. This indicates that the new segmentation is not reasonable or fails to meet the predetermined requirements. Therefore, the bases in the adjacent new segments are reallocated again. For instance, the lowest score (corresponding base) previously assigned to new segment 20 is reallocated to the previous new segment 10, thus lowering the true Q value of new segment 10 and raising the true Q value of new segment 20. This iterative update is repeated until the final true Q value / true error rate of the new segments matches the predetermined quality score.

[0090] Understandably, the calculation or determination of accuracy rate or precision rate in the relevant embodiments refers to the same meaning as the calculation or determination of error rate. Accuracy rate and error rate are corresponding and can be converted into each other. In addition, accuracy rate and correct probability or precision rate or accurate probability in the relevant embodiments generally refer to the same meaning unless otherwise stated; the same applies to the corresponding error rate, error probability, and probability of error, etc., which can be substituted unless otherwise stated.

[0091] In other specific examples, the process further includes merging and regrouping groups based on the error rate of base categories in the base identification results of each group and the proportion or distribution of base categories in each group. This is done so that the error rate of the base categories in the new groups corresponds to a first quality score, establishing a mapping relationship between the classification probability of each base category and the first quality score. Then, based on the error rate of the base categories in the new groups and the proportion or distribution of base categories in each new group, the new groups are merged and regrouped so that the error rate of the base categories in the final groups corresponds to the predetermined quality score, establishing a mapping relationship between the classification probability of each base category and the predetermined quality score. The values ​​in the first quality score include all possible values ​​of the predetermined quality score. In this way, two mapping relationships can be directly generated, outputting two corresponding quality score models or a cascaded quality score model for user comparison and selection.

[0092] The first quality score is the specified quality score, which can be called the standard quality score. In one example, the standard quality score comes from user input, such as parameters that the user can set, adjust, or input themselves. In another example, the standard quality score or the corresponding mapping relationship is an optional parameter built into the system 100, or a mapping relationship that can be established. This parameter or relationship is used by the system 100 to determine whether to select or learn based on the task input by the user.

[0093] In a specific example, the first quality score is a continuous integer, such as a continuous integer Q-value that adapts to or covers the Q-values ​​of most current sequencing platforms, such as [0,45] or [0,50]. The predetermined quality score is a non-continuous integer, such as one that meets or adapts to the user-input task or specified application detection requirements. If the predetermined Q-values ​​are the same, but the upstream base identification algorithm or machine learning base identification model has changed, or it is unknown whether the upstream base identification algorithm or model is the same, then in most cases only the first mapping relationship needs to be updated and adjusted—that is, the mapping relationship between the classification probability of the preceding base category and the standard quality score—without adjusting subsequent mapping relationships. In this way, a quality score model can be quickly generated, facilitating the conversion and evaluation of the actual quality of sequencing data from different sequencing platforms that exhibit the same or different Q-values.

[0094] Furthermore, it can quickly and conveniently meet some special Q-value segmentation requirements. Specifically, the first mapping relationship (the preceding mapping relationship) maps all bases to standard segmentation intervals (standard quality scores) in any scenario. Therefore, such example schemes can provide standard intermediate results when verifying Q-value algorithm performance and troubleshooting algorithm problems, which is helpful for performance verification and problem troubleshooting. For example, the performance based on the preceding mapping relationship can evaluate whether the mapping relationship from P0 to consecutive integer Q values ​​of [0,50] is reasonable, while the performance based on the following mapping relationship can evaluate whether the segmentation mapping from consecutive integer Q values ​​of [0,50] to non-consecutive integer Q values ​​is reasonable, etc.

[0095] Please refer to the relevant examples. Figure 3 The user input also includes a reference sequence, and the training data generation module 142 further includes an alignment submodule 1422, which is used to align the sequencing results to the reference sequence to generate alignment results. Based on the alignment results from the alignment submodule, the labels associated with the features are determined, for example, using the corresponding bases or base combinations on the reference sequence as the corresponding ground truth bases, in order to generate training data.

[0096] Furthermore, in a specific example, the alignment submodule 1422 is connected to the sequencing system or sequencing platform that generates the sequencing results, and is connected to the base recognition software provided or carried by the sequencing system or sequencing platform itself, so as to align the sequencing results to the reference sequence to determine the corresponding bases on the reference sequence.

[0097] Furthermore, during the training data generation module 142's construction of training data, sequences with poor sequencing quality or unreliable alignment positions or relatively low reliability can be removed to improve the reliability of the generated training data. Training a model based on highly reliable training data is beneficial for obtaining a more accurate and reliable prediction model. For example, in a certain example, for sequences that failed to align successfully with the reference sequence or whose alignment results are unreliable or have low reliability, these sequences or bases can be reassigned as "N" or marked as "N," and the bases or sequences marked as "N" can be excluded from training data or subsequent model training processes. This can improve the reliability of the generated training data or the model trained based on such training data.

[0098] In some implementations, please refer to Figure 4 The machine learning layer 140 also includes an evaluation module 146, which is connected to the training data generation module 142, the training module 144, and the output layer 160. The evaluation module 146 is used to evaluate the performance of the quality score model using a test set before outputting the quality score model in any relevant example, so as to determine that the quality score model meets preset requirements. The test set is a portion of the training data that does not overlap with the training set and the validation set. Meeting the preset requirements means that at least one of the following metrics of accuracy, precision, recall, and speed of the classification prediction results of the quality score model on the test set meets a preset standard or is able to perform the task input by the user.

[0099] In some embodiments, the specified model is selected from at least one of logistic regression, Naive Bayes, support vector machines, and tree models such as LightGBM. The specified model can be a single model, a cascaded model containing multiple single models, or a composite model.

[0100] In some embodiments, please refer to Figure 5 The machine learning layer 140 also includes a selection module 148 for determining a specified model based on user input. This can be done either by the user directly selecting a specified model through user input, or by the system 100 automatically selecting a specified model based on user input.

[0101] Additionally, in some related embodiments, the user input also includes sequencing data (intermediate sequencing data), which includes images of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection during one or more sequencing runs; or the sequencing data includes the positions and intensity values ​​of multiple nucleic acid molecules in the one or more surface regions in which they reside, and the sequencing images are images of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection during one or more sequencing runs.

[0102] The sequencing data is processed using a base recognition model to obtain the so-called sequencing results. The base recognition model or algorithm can be the base recognition software provided with the sequencing platform, or it can be a machine learning base recognition model automatically generated by the system. In some embodiments, please refer to... Figure 6 A or Figure 6 B. System 100 also includes an automatic base recognition model generation system 110 connected to or placed in the input layer 120, the automatic base recognition model generation system 110 being used to generate and output a base recognition model.

[0103] Based on some specific examples, input layer 120 is the first input layer 120, machine learning layer 140 is the first machine learning layer 140, and output layer 160 is the first output layer 160. Please refer to... Figure 7 The base recognition model automatic generation system 110 includes: a second input layer 130 for receiving user input from a user, including so-called sequencing data and optionally one or more tasks, wherein the sequencing data is obtained by detecting nucleic acid molecules connected to multiple locations on a surface using surface imaging sequencing technology; a second machine learning layer 150, including a second training data generation module 152 and a second training module 154, wherein the second training data generation module 152 processes the user input from the second input layer 130 to generate second training data, wherein the second training data includes multiple pairs of associated second features and second labels, wherein the second features describe the strength corresponding to nucleic acid molecules at one or more locations, and the second labels include a selection of bases A, T, C, and G; the second training module 154 is used to train a second specified model based on at least a portion of the second training data, including performing multiple iterations of training on the second specified model using a second training set and evaluating the performance of the second specified model after each iteration of training using a second validation set, in order to obtain a base recognition model, wherein the second training set and the second validation set are each independently non-overlapping portions of the second training data; and a second output layer 170 for outputting the base recognition model. The second output layer 170 is connected to the input layer 120.

[0104] The acquisition methods and technical features of sequencing data or intermediate sequencing data in the aforementioned related embodiments are also applicable to the automatic generation system 110 for such base recognition models, and will not be repeated here.

[0105] In some examples, the so-called second label also includes combinations of any two, three, or four bases selected from A, T, C, and G. The second label is a presumed correct base identification result, and the methods and technical characteristics for obtaining the corresponding benchmark true bases in the aforementioned related embodiments are also applicable to the automatic base identification model generation system 110.

[0106] In some examples, please refer to Figure 8 The user input from the second input layer 130 also includes a reference sequence. The second training data generation module 152 further includes a second alignment submodule 1522. The second alignment submodule 1522 is used to align the sequence readout or base readout from the sequencing results to the reference sequence to generate a second alignment result; based on the second alignment result, it determines the second label associated with the second feature to generate second training data. The description of the technical features and characteristics of the first alignment submodule 1422 in the related embodiments also applies to the second alignment submodule 1522. In a specific example, the first alignment submodule 1422 and the second alignment submodule 1522 are the same module.

[0107] In some examples, please refer to Figure 9 The second machine learning layer 150 further includes a second evaluation module 156 connecting the second training data generation module 152, the second training module 154, and the second output layer 170. This module is used to evaluate the performance of the base recognition model using a second test set before outputting the base recognition model, so as to determine whether the base recognition model meets preset requirements. The second test set is a portion of the second training data that does not overlap with the second training set and the second validation set. Meeting the preset requirements means that at least one of the accuracy, precision, recall, and speed of the base recognition model's classification prediction results on the second test set meets a preset standard or is able to perform the user input task.

[0108] In some examples, the second specified model is selected from at least one of logistic regression, support vector machines, tree models, and neural networks. The second specified model is a single model or a composite or cascaded model containing multiple models. In a specific example, the second specified model includes a neural network model, such as a neural network model for semantic segmentation like a convolutional neural network (CNN), more specifically, at least one of U-Net, DeepLab, SegFormer, PSPNet, FCN, GCN, and their respective variants. In a specific example, the automatic base recognition model generation system 110 trains the second specified model using the scheme disclosed in CN119479831A, the entire text of which is incorporated herein by reference.

[0109] In some examples, please refer to Figure 10The second machine learning layer 150 also includes a second selection module 158, used to determine the type of the second specified model based on the corresponding user input. This can be either a direct selection or specification of a model by the user through user input, or an automatic selection of the second specified model by the task system 100 or the base recognition model automatic generation system 110 based on user input. The descriptions of the technical features and characteristics of the selection module 148 in the relevant examples also apply to this second selection module 158. In a specific example, the selection module 148 (the first selection module) and the second selection module 158 are the same module.

[0110] Some embodiments of this application also provide a method for developing a quality score model, including accessing user input using a system 100 for automatically generating machine learning models in any of the above embodiments, and running the system 100 using at least one hardware processor to present the best quality score model to the user. The user input includes sequencing results and optionally one or more tasks. The sequencing results are obtained by detecting nucleic acid molecules linked to multiple locations on a surface using surface imaging sequencing technology. The sequencing results include base recognition results and base recognition probabilities.

[0111] It is understood that the descriptions of the technical features, additional technical features, and related technical effects of the system 100 for automatically generating machine learning models in the above-described embodiments are also applicable to the method for developing a quality score model or the method for developing a quality score model and an associated base recognition model in this embodiment, and will not be repeated here.

[0112] For example, in some embodiments, the sequencing results include readouts of no less than 10 million sequences and / or no less than 1 billion base pairs.

[0113] In some embodiments, the user input is received through a model generation framework or interface; and / or, the best quality score model or a quality score model that meets the task requirements input by the user is presented to the user through a model generation framework or interface.

[0114] In some embodiments, the system is run using two or more operating systems, and interaction between the different operating systems is achieved using at least one of file sharing, remote access, network communication, virtual machines, containers, and virtualization technologies or tools, in order to quickly train and generate relevant models.

[0115] Specifically, in some examples, a portion of system 100 is run using Windows and Linux respectively, and interaction between the different operating systems is achieved using at least one of WSL and / or SSH.

[0116] SSH (Secure Shell) is a type of "remote access" method (used to operate or access another operating system from one operating system). It is suitable for remote management of Linux and macOS and can be accessed from Windows via tools such as PuTTY and Termius. WSL (Windows Subsystem for Linux) is a type of "virtual file system and bridging tool" (bridging files or processes of different operating systems through virtualization technology). It enables running a Linux environment on Windows and allows for file and tool sharing.

[0117] In some embodiments, system 100 utilizes multi-threaded parallel computing to improve the speed of model generation. Specifically, in one example, system 100, such as training data generation modules 142 and / or 152 in its machine learning layers 140 and / or 150, runs interactively on multiple operating systems, with multi-threaded parallel mechanisms configured on each operating system.

[0118] Some embodiments of this application also provide a quality score model generated using the system 100 for automatically generating machine learning models in any of the above embodiments.

[0119] Other embodiments of this application also provide a computer-readable storage medium storing one or more instructions for controlling a processor to execute the method for developing a quality score model in any of the above embodiments. The computer-readable storage medium includes, but is not limited to, random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0120] A sequenced list of executable instructions for implementing logical functions may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). In related embodiments, the computer-readable storage medium may be any apparatus that can contain, store, communicate, propagate, or transmit programs for instruction execution, or be used in conjunction with such a system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable storage media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory. The various computer-readable storage media described in the related examples can represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" as used herein can include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0121] Embodiments of this application also provide a system for automatically generating machine learning models, also referred to as a computer program product or computing device, the system including a memory storing a program and one or more processors coupled to the memory, the processors running the program to implement the method for developing a quality score model in any of the related embodiments.

[0122] When using a software implementation method, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more instructions or programs. When such computer programs or instructions are loaded and executed on a computer, the processes or functions in the relevant embodiments of this application are implemented, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0123] The term "computer program product" or "computing device" is intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. A computing device can also refer to various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components, connections, associations, and functions of the components shown in the embodiments of this application are merely examples and are not intended to limit the description and / or related implementations of the embodiments of this application.

[0124] Please refer to Figure 11 The system 100 or electronic device 500 that automatically generates machine learning models includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 502 or a computer program loaded from storage unit 508 into RAM (Random Access Memory) 503. The RAM 503 may also store various programs and data required for the operation of the device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.

[0125] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0126] The processor or computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. Computing unit 501 executes the modules, methods, or processes described in the relevant embodiments, such as implementing the functions of machine learning layers 140 and / or 150, etc.

[0127] For example, in some examples, the method for developing a quality score model can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the methods in the aforementioned related embodiments can be performed. In other embodiments, computing unit 501 can be configured by any other suitable means (e.g., by means of firmware) to perform the methods for developing quality score models and / or base recognition models using automated training tools in any of the foregoing related embodiments.

[0128] It should be understood that the various parts of the system or method in the embodiments of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described related embodiments or implementations, multiple layers, modules, sub-modules, or steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, it can be implemented using any one or a combination of the following related prior art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0129] Those skilled in the art will understand that all or part of the steps of the system for automatically generating models in any of the above embodiments or the methods for implementing any of the above related embodiments can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium and, when executed, includes one or a combination of steps for implementing the example methods.

[0130] Furthermore, the functional units in various implementation methods or embodiments can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0131] Publicly available schemes for building Q-value models based on machine learning include CN118471340A, CN117912550A, CN117976042A, and CN118116469A, while publicly available schemes for constructing base recognition models based on machine learning include CN118429967A, CN118429965A, and CN118969086A. The system 100 for automatically generating machine learning models in this application is applicable to the automated development of any publicly available machine learning-based Q-value algorithm and / or base recognition model. It automates the machine learning model development process, reducing reliance on specialized knowledge and experience in algorithm development.

[0132] The following describes, exemplarily, the establishment, deployment, and use of a system (automatic development tool) for automatically generating machine learning models, using one or more publicly available solutions or related optimizations and improvements to develop Q-value algorithms and / or base recognition models based on machine learning. The following example begins with the automated generation of a base recognition model. Based on the base recognition results predicted by the generated model and related parameters (e.g., base recognition probabilities), training data is constructed to automatically develop the corresponding quality score model. Based on the foregoing embodiments, it is understood that this application does not limit the manner in which the sequencing results / base recognition results on which the automated development of the quality score model is based are obtained, and they may not originate from the prediction results of the base recognition model generated by this automatic training tool.

[0133] The following further examples describe the specific operating methods or implementation processes that may be included in some of the above embodiments or examples. However, this should not be construed as limiting the scope of the subject matter of this application to the following further examples or embodiments.

[0134] Establishment and Evaluation of Machine Learning-Based Base Recognition Model and Corresponding Q-Value Algorithm (Part 1) Establishment of Machine Learning Base Recognition Model

[0135] The main steps include:

[0136] 1. Determine the application scenario of the base recognition model to be built, and determine the corresponding platform, instrument, biochemical, and fluid dynamics versions of the model. Here, we take a multi-fluorescence channel based on surface imaging, such as the commonly known two-color or four-color fluorescence high-throughput sequencing platform, as an example.

[0137] 2. Under the defined operating environment and conditions in 1, sequence a standard library sample containing a known reference genome and save the original sequencing data, such as the original fluorescence image (sequencing image).

[0138] 3. Establishment of training data for the base recognition model (hereinafter referred to as training data I):

[0139] (1) Use existing basic base recognition software, such as the base recognition software built into various sequencing platforms (which may involve machine learning or not), to analyze the sequencing data and identify bases, and output the base recognition result file (fastq file). When using basic base recognition software for base recognition, retain the cluster location information during the base recognition process, as well as the fluorescence brightness information (intensity information) of each fluorescence channel in each round of each cluster (amplifier), for subsequent model feature value extraction.

[0140] (2) Align the fastq file with the reference genome to obtain the alignment result file (sam or bam file).

[0141] (3) Based on the brightness information of each cluster in each round, the brightness information of the first N rounds and the last M rounds (N and M are independent integers) and other optional information such as the brightest base in the first N rounds and the position information of the cluster are selected to create feature values.

[0142] (4) Create target values ​​based on the information after the sequence alignment of the cluster. For example, the target value is the reference base corresponding to each base after alignment, which is sometimes called the standard answer.

[0143] (5) Map the feature values ​​to the target values ​​one-to-one to complete the creation of training data I. When creating training data I, only sequences with high alignment confidence can be retained. Furthermore, it is preferable to exclude polymorphic sites from the scope of training data I. In short, when selecting training data, ensuring the accuracy of the "target value" as much as possible is beneficial for training a model with more accurate prediction results. In one example, for data with a high probability of error in the target value, either it should not be included in training data I, or the target value should be labeled as N, where N represents one of A, G, C, and T; in a scenario where deep learning, such as neural networks, is used for model training, all feature values ​​and target values ​​corresponding to clusters are written into training data I.

[0144] 4. Use a portion of the training data I to train the base recognition model. For example, in this example, the training data I is divided into three parts: one part is used for the development of the base recognition model (this part of the training data I is abbreviated as I.1), one part is used for the development of the corresponding Q-value model (this part of the training data I is abbreviated as I.2), and one part is used for validation (this part of the training data I is abbreviated as I.3).

[0145] 5. Establish a base recognition model: Randomly divide the training data I.1 into training and test sets, and establish a base recognition model (abbreviated as Model B0) based on the dataset. The "base recognition model" refers to the correspondence between "feature values" and "target values" mentioned earlier. Any machine learning or deep learning algorithm can be used to build the model. In this example, the model algorithm disclosed in CN119479831A is used, and its entirety is incorporated herein by reference.

[0146] In paired-end (PE) sequencing, in some best-case scenarios, the biochemical characteristics of R1 (read1) and R2 (read2) often differ, so it is better to extract features and train the model separately for them. Similarly, in some scenarios involving double-sided chip sequencing, the biochemical and optical characteristics of the two surfaces may differ, so it is better to build models independently based on the detection or readout of each surface, which generally helps to obtain models with better prediction performance. Here, we take the establishment of one model as an example; the other model is similar and will not be repeated below. (II) Establishment of the Q-value algorithm corresponding to the base recognition model B0

[0147] 1. Based on training data I.2, establish training data for the Q-value algorithm (hereinafter referred to as training data II).

[0148] (1) Using the base recognition model B0 established above, base recognition is completed based on the feature values ​​of training data I.2, and the base recognition probability corresponding to the model is obtained, that is, the normalized recognition probability of A, G, C and T output by each feature data after model prediction, and the sum of the four base recognition probabilities of each data is 1; and the base recognition result predicted by the model, which is usually the base corresponding to the maximum base recognition probability in each data.

[0149] (2) Compare the base identification results of the above model with the target value of training data I.2. If they match, mark them as 1 or correct; if they do not match, mark them as 0 or incorrect. This correct / incorrect information is used as the target value of training data II.

[0150] (3) The feature value of training data II can be exactly the same as that of training data I.2; it can also be the base recognition probability corresponding to each data point; it can also be one or more of the four base recognition probabilities, such as the feature value being the largest base recognition probability, or the largest and second largest base recognition probabilities, etc.; it can also be the derived calculation result based on the four base recognition probabilities, such as the ratio of the largest and second largest base recognition probabilities; or it can be any combination of any part of the above-mentioned parameters.

[0151] 2. A Q-value algorithm is established based on training data II (hereinafter referred to as the basic scheme).

[0152] a) Train the machine learning classification algorithm based on the training data II. Any suitable machine learning model can be selected, such as logistic regression, support vector machine, tree model, etc. The obtained model is denoted as model Q0.

[0153] b) Apply the Q0 model to training data II, that is, use the Q0 model to obtain the model prediction result based on the feature values ​​of training data II. Here, the final model classification result does not have to be output; only the classification probability P value given by the model during prediction can be output, denoted as P0.

[0154] c) Typically, P0 is a probability value between 0 and 1. P0 is divided into 'a' segments, and the bases in training data II are correspondingly divided into 'a' segments. The principle for choosing 'a' is that after dividing P0 into segments 'a', the difference in the true Q-values ​​of the bases corresponding to adjacent segments is not significant. In the actual operation of the example, it is generally required that the difference in the true Q-values ​​of adjacent segments is less than 1.

[0155] Here's some explanation of the "true Q-value". The Q-value is actually another way of representing the error rate. The relationship between Q and the error rate can be written as Q = -10 * log10(err). Here, Q refers to the sequencing quality score of the bases, i.e., the Q-value; err refers to the error probability or error rate of that base. The "true Q-value" refers to the Q-value calculated based on the true error rate (err) of a set of bases. For example, as mentioned above, if the bases are divided into segments 'a' based on the P0 value, then the total error rate of the bases in each segment can be calculated. For example, the error rate of that segment can be calculated by using the number of inconsistent bases as the numerator and the total number of bases in that segment as the denominator, and the corresponding Q-value can be calculated based on this error rate. This Q-value is the "true Q-value" defined here.

[0156] d) Write down the actual Q values ​​corresponding to the bases in segment a.

[0157] e) Define the segmentation requirements for the Q value based on the algorithm development needs. For example, some scenarios require Q to take the value of all integers in the range [0, 45], while others require Q to take the value of 5 specific values ​​in [0, 45], etc.

[0158] f) Based on the above requirements for Q values, segment the Q values. In the above steps, based on the statistics of training data II, the true Q value (true error rate) corresponding to each base in each of the a segments is known; at the same time, the distribution trend of the number of bases in different segments is also known. Therefore, based on the error rate and base percentage corresponding to each of these a segments, and the error rate requirement in the target Q value segment (i.e., the Q value mentioned above), the bases in the above a segments can be merged and segmented again. After merging the segments, the total error rate corresponding to the bases in each segment is consistent with the pre-required Q value.

[0159] g) Record the correspondence between each final P0 value and the Q value (segmented Q value), which is the Q-value algorithm corresponding to the machine learning algorithm.

[0160] 3. A Q-value algorithm is established based on training data II (hereinafter referred to as Supplementary Scheme 1).

[0161] a) The first few steps are consistent with a) and b) of the basic scheme.

[0162] b) Corresponding to step c) of the basic scheme, when segmenting bases according to the P0 value, the category of base identification is also considered to distinguish the bases. For example, in the basic scheme, all bases are placed together and divided into segment a according to the P0 value. In supplementary scheme 1, bases are first classified according to the final identification type. For example, they can be classified according to the current base identification category A, C, G, and T; or they can be classified according to the current base and the categories of the bases before and after it, such as ATC, TAT, etc. In short, bases can be classified according to the current base identification type, the identification types of the previous n bases, the identification types of the next m bases, or a combination of any two or three of the above, where n and m are independent integers.

[0163] c) After completing the above classification, perform the operations on each type of base according to d)-g) of the basic scheme.

[0164] d) Record the correspondence between the P0 values ​​and Q values ​​(segmented Q values) corresponding to each base classification. This correspondence is the so-called corresponding Q value algorithm.

[0165] Compared to the basic scheme, Supplementary Scheme 1 takes into account the potential P0 value preference between different base combinations and corrects for it when mapping Q values. The preference of P0 value relative to base combinations mainly comes from biochemical preference characteristics, that is, bases with the same brightness signal characteristics but different base combinations may have significantly different error rates.

[0166] 4. Establish a Q-value algorithm based on training data II (hereinafter referred to as Supplementary Scheme 2)

[0167] Both the basic scheme and supplementary scheme 1 can be combined with supplementary scheme 2. One of the characteristics of supplementary scheme 2 is that the segmentation of the Q value is divided into two steps.

[0168] Specifically, in steps e)-f) of the basic scheme or the corresponding steps of supplementary scheme 1, the bases segmented according to the P0 value are directly mapped to the final Q value segment. In supplementary scheme 2, however, all bases are first mapped to the Q value segment containing all integer values ​​in the range [0,50].

[0169] Specifically, the basic approach in the example is to directly map the bases segmented according to the P0 value to, for example, a Q-value segment of all integer values ​​in the range [0, 50]. Supplementary approach 1 first maps the bases in different groups to Q-value segments of all integer values ​​in the range [0, 50], and then merges all base categories according to the Q-value or segmentation requirements. Supplementary approach 2, in short, obtains the mapping relationship between the P0 values ​​corresponding to different categories of bases and, for example, a Q-value segment of all integer values ​​in the range [0, 50], which is referred to here as mapping relationship 2.1.

[0170] After obtaining mapping relation 2.1, the Q-value segments of all integer values ​​in the range [0,50] are then mapped to the final required Q-value segments, which is referred to here as mapping relation 2.2. Mapping relation 2.1 and mapping relation 2.2 together constitute the corresponding Q-value algorithm.

[0171] One advantage of Supplementary Scheme 2 is that mapping relationship 2.1 is implemented using the same logic in any scenario, which is beneficial for the implementation of automatic modeling software. If different scenarios have different Q-value segmentation requirements, then mapping relationship 2.2 can generally be adjusted directly. If the Q-value segmentation requirements are the same, but the upstream machine learning base recognition model has changed, then in most cases only mapping relationship 2.1 needs to be adjusted, and mapping relationship 2.2 usually does not need to be adjusted.

[0172] Since mapping relationship 2.1 maps all bases to standard segmented intervals in any scenario, scheme 2 can provide standard intermediate results when verifying the performance of the Q-value algorithm and troubleshooting algorithm problems, which is helpful for performance verification and problem troubleshooting. For example, the performance based on mapping relationship 2.1 can be used to evaluate whether the mapping relationship from P0 to integer Q values ​​of [0,50] is reasonable, while the performance based on mapping relationship 2.2 can be used to evaluate whether the segmented mapping from integer Q values ​​of [0,50] to large segmented Q values ​​is reasonable.

[0173] The advantage of Supplementary Scheme 2 is that it can easily meet some special Q-value segmentation requirements. The bases corresponding to the subdivided Q-value segments (e.g., the bases corresponding to the integer Q-values ​​[0, 50], or the mapping relationship between the bases directly segmented according to P0 in the basic scheme and the final target Q-value segmentation) are often not unique.

[0174] First, for example, during product development, the final target Q value is often given in the form of intervals. For instance, it might be required to "determine a value as the Q value from the intervals (0,15), (15,20), (20,30), (30,40), and (40,50)," but the specific value for each interval is often not predetermined. It can be determined during development based on the specific base classification, thus there can be many Q value segments that meet the requirements. Second, even if the specific final Q value is defined, there is more than one way to divide the bases to meet the requirements. In this case, the mapping relationship 2.1 of Supplementary Scheme 2 maps bases to standard Q value segments, which is more conducive to meeting special Q value segmentation requirements, such as "the Q35 segment cannot contain bases with a Q value lower than 30," and similar base division requirements.

[0175] (III) Model Performance Evaluation

[0176] a) Test the currently established base recognition model B0 and the corresponding Q-value algorithm on the validation set I.3.

[0177] b) Evaluation of base recognition model B0

[0178] Based on the brightness information of the current round in the feature values ​​of I.3, the base identification result under the traditional algorithm (not based on machine learning model prediction) can be obtained based on the brightest base. This result is denoted as base identification result 1 of I.3. Based on the feature values ​​of I.3, the base identification model B0 is used for prediction, and the result is denoted as base identification result 2. Base identification result 1 and base identification result 2 are compared with the target value of I.3 respectively, and the error rate is calculated for each (i.e., the number of base identification results that differ from the target value divided by the total number of entries in the I.3 data). The reduction in the error rate of base identification result 2 compared to base identification result 1 is the improvement in base identification accuracy brought about by base identification model B0.

[0179] c) Performance evaluation of the Q-value algorithm

[0180] Compare the base identification result 2 with the target value of I.3 to obtain the correctness result of each data entry under the base identification model B0, which is recorded as the base correctness result 2.

[0181] For the basic scheme, based on the base error result 2, the error rate corresponding to each Q-value segment is output, and the actual Q-value is calculated based on the error rate. The difference between the Q-value assigned by the algorithm and the actual Q-value is compared to determine whether the Q-value algorithm meets expectations.

[0182] For supplementary scheme 1, in addition to comparing the actual Q values ​​corresponding to the bases in different Q value segments, it is also necessary to compare the actual Q values ​​corresponding to the bases in different Q value segments for different base recognition types, so as to evaluate whether the total base Q value assignment is reasonable and whether the Q value assignment for specific types of bases is reasonable / meets expectations.

[0183] For supplementary scheme 2, in addition to comparing the actual Q values ​​corresponding to the bases in the final Q value segment, it is also necessary to compare the actual Q values ​​calculated for the bases corresponding to the [0,50] integer Q values ​​generated by mapping relationship 2.1.

[0184] In addition, before developing the base recognition model and Q-value algorithm, it is generally necessary to pre-set the pass indicators and qualified thresholds for the performance of the base recognition model and Q-value algorithm. For example, at least one of the indicators of accuracy, precision, recall and speed of the base recognition model for the classification prediction results of the validation set I.3 must meet the preset standard or be able to achieve the user input task.

[0185] When the performance of the base recognition model or Q-value algorithm fails to meet the threshold requirements, a mechanism for algorithm redevelopment can be triggered, or a manual intervention alert can be activated. For example, users can be prompted to add user input, such as sequencing data, to generate more training data for better model training.

[0186] Finally, the output model is a base recognition model and / or Q-value algorithm that has passed or met the preset indicators or expected requirements after performance evaluation, so as to complete machine learning modeling.

[0187] Model development process automation

[0188] 1. The above-described data processing method and model development logic can be implemented using any programming language on Linux or Windows systems.

[0189] 2. When using a Windows system to complete the main workflow of machine learning model training, there may be a need for system interaction between Windows and Linux. For example, since most mainstream bioinformatics analysis software is developed on Linux systems, even if there are Windows versions, their running speed is often much slower than the Linux versions. The workflow of building a base recognition model in the example above includes bioinformatics analysis steps such as alignment and error rate statistics. If the main workflow of model development is carried out on Windows, then completing this part of the bioinformatics process on Linux will help improve the efficiency of automated training tools.

[0190] Here are two examples of how to achieve interaction between Windows and Linux systems.

[0191] (1)WSL2

[0192] Installing a Linux subsystem on Windows via WSL (Windows Subsystem for Linux) enables interaction between Windows and Linux. Advantages: The Linux subsystem boots quickly, consumes few resources, and has almost no impact on Windows performance; WSL2 has a complete Linux kernel and supports the installation of most Linux software; WSL2 supports container technologies such as Docker / Singularity, making software deployment convenient and fast. Disadvantages: File access between Windows and Linux is via network output, resulting in slower read and write speeds compared to the native system.

[0193] (2) OpenSSH

[0194] Implementation: Install OpenSSH software on Windows and Linux, and interact via the SSH protocol. Advantages: Based on the SSH protocol, security is greatly improved, and the connection is stable; easy integration, by configuring passwordless SSH access, remote connections, task submissions, task status checks, and other operations can be performed in Python scripts; independent Windows and Linux nodes can maximize the computing power of each node. Disadvantages: Installation and deployment are relatively complex, requiring separate deployment of OpenSSH software on Windows and Linux, and configuration of passwordless access.

[0195] 3. During the model training phase, parallel computing can be used to improve the running speed of training tools.

[0196] For example, in the process of base identification using basic or traditional non-machine learning-based base identification software as described in (I) 3(1) above, multiple threads can be called simultaneously for analysis via code. When developing automatic training tools, it is generally necessary to analyze the computing power and memory occupied by the basic version of the base identification software in advance and write the calculation formula for the maximum number of threads corresponding to different computing power and memory. When using the automatic modeling tool (automatic training tool), the tool will automatically read the computing power and memory of the system used for modeling and automatically calculate the maximum number of threads that can run in parallel according to the maximum number of threads calculation formula.

[0197] Regarding the calculation of the maximum or optimal number of threads corresponding to specific computing power and memory, those skilled in the art can reasonably determine the number by comprehensively considering hardware specifications (such as the number of CPU cores, GPU hardware characteristics such as the number of multiprocessors, thread block size, etc., memory capacity and bandwidth) and task type (such as CPU-intensive or I / O-intensive). For example, the number of threads can be calculated based on processors such as CPU and / or GPU, memory and / or task type, or by comprehensively considering the number of threads, in order to reasonably calculate and optimize the number of threads.

[0198] (1) As mentioned in (a) 3 (2), whether it is performed on a Windows system or a Linux system, a multi-threaded parallel mechanism can be set. Similarly, the calculation formula for the optimal number of parallel threads is pre-entered, and the number of parallel threads is automatically set during actual operation.

[0199] (2) The training data production in (a) 3 (3)-(5) above can also be set up for multi-threaded parallel computing in the same way.

[0200] (3) The machine learning training steps of the aforementioned (I) 5-base recognition model can also be set up for parallel computation. Furthermore, in scenarios where models are trained separately using R1 and R2 data from paired-end (PE) sequencing, both R1 and R2 models can be trained in parallel if computing power and memory allow. Similarly, in scenarios where models are trained separately using multi-faceted data such as bifacial microarray sequencing, parallel computation is set up for each model training.

[0201] 4. The feature value form of the model can be determined before training to improve the data extraction speed. And when using the traditional or basic version algorithm for base identification in step (I) 3(1), the feature value can be directly calculated and output using the cached data in the base identification process, without retaining other unnecessary intermediate parameters, thereby improving the efficiency of data calculation and reducing the data writing time.

[0202] Use automated training tools

[0203] The following example illustrates how an automated training tool can output a model file based on user input, such as sequencing images, in a specific scenario.

[0204] Suppose a user needs to use a machine learning model in a specific application scenario. This scenario might require special sample features, unique biochemical and instrument configurations, and no suitable machine learning model has ever been built for this scenario. Under such circumstances, the user can use the automated model generation system (automated training process) to build the corresponding or suitable model themselves.

[0205] 1. Generally, users first define the sequencing conditions required for the model they need to build, such as biochemical parameters, instrument configuration, and library characteristics. Then, they perform sequencing under these defined conditions to obtain sequencing data for generating training data. For example, they might use a library with a known reference sequence. If the user doesn't specify a particular library for this sequencing scenario, a larger genome library, such as a human genome sample, can be used as the sequencing sample. Larger genome samples allow for more diverse sequence feature information in the training data, which typically helps improve the model's generalization ability. If the "special sequencing scenario" requires specific library features, such as for libraries with extreme base imbalances, then it's preferable to use a library with features identical or similar to the target sequencing library to help the machine learning model learn the sequencing preferences introduced by the library features.

[0206] 2. During the sequencing process described above, the user invokes the machine learning automatic training tool. Currently, mainstream commercially available sequencing platforms or common sequencing procedures often perform data processing, including base identification, during sequencing. The software included with the sequencer for this data processing is what is referred to as the traditional or basic version of the base identification algorithm. This traditional or basic version of the base identification algorithm or tool may be a machine learning model or software that does not involve machine learning. Here, the user can directly use the automatic training tool during the sequencing process. The automatic training tool will automatically invoke the traditional or basic version of the base identification algorithm during the sequencing steps. While invoking the traditional or basic version of the algorithm to complete base identification, it can simultaneously generate the feature values ​​needed by the machine learning model based on intermediate sequencing parameters such as the brightness of the original image and contextual base identification information, and store them in a specified path. Understandably, in a regular sequencing process (without invoking the automatic training tool), the intermediate sequencing parameters mentioned here are often parameter data that the sequencer needs or usually generates during base identification but generally does not need to be output or output to the user.

[0207] 3. The automated training process can automatically identify the sequencing progress. For example, after sequencing is complete, the automated training tool can automatically call the alignment tool, align the FASTQ file obtained from sequencing with a reference genome (e.g., user-inputted data), and output the alignment results to a specified path.

[0208] 4. After alignment is complete, the automatic training tool will begin automatically generating training data based on the stored feature values ​​and alignment results. The feature values ​​for the training data have already been output during the sequencing process; the target values ​​will be extracted based on the alignment results. See section (I) 3 of the aforementioned example for details. The training data will be automatically stored in the specified path.

[0209] 5. After the training dataset is created, the automatic training tool can automatically divide the training data, automatically complete the modeling of the machine learning model, and automatically complete the development of the Q-value algorithm, following the process described in (I) 4 and 5 and (II) above.

[0210] 6. After completing the machine learning model modeling and Q-value algorithm development, the automatic training tool will automatically save the model and algorithm files in the specified path. The machine learning and Q-value algorithm model files can be used as configuration files for the machine learning base recognition tool. As long as the configuration file is stored in the specific folder called by the base recognition tool, the next time the machine learning base recognition tool is used for sequencing, it will automatically use that configuration for base recognition and Q-value calculation. Therefore, after completing the modeling, users can directly copy the configuration file to the corresponding configuration file path of the base recognition tool on the sequencer, and the model can be used directly for the next sequencing run.

[0211] In this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "certain examples," "specific examples," or "exemplary practice," etc., refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0212] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A system for automatically generating machine learning models, wherein the generated machine learning models are used to output quality scores to evaluate the accuracy of base identification results, characterized in that, include: The input layer is used to receive user input from the user, including sequencing results and optionally one or more tasks. The sequencing results are obtained by sequencing and detecting nucleic acid molecules connected to multiple locations on the surface using surface imaging sequencing technology. The sequencing results include base recognition results and corresponding base recognition probabilities. The machine learning layer includes a training data generation module and a training module. The training data generation module is used to process user input from the input layer to generate training data. The training data includes multiple pairs of associated features and labels. The feature description (i) is one or more base recognition probabilities corresponding to the base recognition result, or the description (ii) is one or more base recognition probabilities corresponding to the base recognition result and the intensity corresponding to one or more nucleic acid molecules at one or more positions. The labels include two categories: base recognition result and corresponding benchmark true base, which are consistent and inconsistent. The base recognition probability comes from the output of the base recognition model. The training module is used to train a specified model based on a portion of the training data in order to obtain a quality score model. This includes iteratively training the specified model using the training set and evaluating the performance of the specified model after each iteration using the validation set in order to obtain a quality score model. The training set and the validation set are each independent and non-overlapping portions of the training data. as well as The output layer is used to output a quality score model that meets preset requirements.

2. The system according to claim 1, characterized in that, The sequencing results include at least 10 million sequence reads and / or at least 1 billion base pairs reads.

3. The system according to claim 1 or 2, characterized in that, Training the specified model in the training module further includes, Obtain the classification probability of the base identification results in the training set; Based on the classification probability and / or the base category in the base identification result, the base identification result is divided into group a, where a is a natural number not less than 5; For each group classification probability and / or corresponding base identification result, the error rate or correct rate of the base category in the base identification result of the group is determined by comparing the base category in the base identification result of the group with the corresponding benchmark true base. as well as The predetermined quality score is correlated with the error rate or correctness rate of the base category.

4. The system according to claim 1 or 2, characterized in that, Training the specified model in the training module further includes, Obtain the classification probability of the base identification results in the training set; Based on the classification probability and / or the base category in the base identification result, the base identification result is divided into group a. For each group, the error rate of the base category in the base identification result of that group is determined by comparing the base category in that group with the corresponding benchmark true base. a is a natural number and the value of a can ensure that the logarithmic difference of the error rate of the base category in the base identification result of adjacent groups is less than 2. as well as The predetermined quality score is correlated with the error rate of the base category.

5. The system according to claim 4, characterized in that, It also includes, Based on the error rate or accuracy of base categories in the base identification results of each group, the proportion or distribution of base categories in each group, and the predetermined quality score, the groups are merged into new groups with fewer groups, the attribution of base categories in adjacent new groups is adjusted, and the error rate of base categories in each new group is calculated so that the error rate of base categories in the final new group corresponds to the predetermined quality score, thereby establishing a mapping relationship between the classification probability of the base category and the predetermined quality score.

6. The system according to claim 4, characterized in that, It also includes, Based on the error rate of base categories in the base identification results of each group and the proportion or distribution of the number of base categories in each group, the groups are merged and then regrouped so that the error rate of the base categories in the new group corresponds to the first quality score, so as to establish the mapping relationship between the classification probability of each base category and the first quality score. Based on the error rate of the base categories in the new group and the proportion or distribution of the number of base categories in each new group, the new groups are merged and then regrouped so that the error rate of the base categories in the final group corresponds to the predetermined quality score, so as to establish a mapping relationship between the classification probability of each base category and the predetermined quality score, wherein the value in the first quality score includes all values ​​of the predetermined quality score.

7. The system according to claim 6, characterized in that, The first quality score is a continuous integer, while the predetermined quality score is a non-continuous integer.

8. The system according to claim 6, characterized in that, The user input also includes the first quality score.

9. The system according to claim 4, characterized in that, The user input also includes the corresponding benchmark true base.

10. The system according to claim 4, characterized in that, The user input also includes a reference sequence, and the training data generation module also includes an alignment submodule. The alignment submodule is used to align the sequencing results to the reference sequence in order to generate alignment results; and to determine the corresponding baseline true base based on the alignment results from the alignment submodule.

11. The system according to claim 9, characterized in that, The user input also includes a reference sequence, and the training data generation module also includes an alignment submodule. The alignment submodule is used to align the sequencing results to the reference sequence in order to generate alignment results; and to determine the corresponding baseline true base based on the alignment results from the alignment submodule.

12. The system according to claim 1, characterized in that, The specified model is selected from at least one of logistic regression, support vector machine, and tree model.

13. The system according to any one of claims 1, 2, 5-12, characterized in that, The machine learning layer further includes an evaluation module, which connects the training module and the result output layer. This evaluation module evaluates the performance of the quality score model using a test set before outputting the quality score model, in order to determine whether the quality score model meets preset requirements. The test set is a portion of the training data that does not overlap with the training set and the validation set. Meeting the preset requirements means that at least one of the following indicators of the classification prediction results of the quality score model on the test set—accuracy, precision, recall, and speed—meets the preset standard, or that the model is able to perform the user-input task.

14. The system according to claim 3, characterized in that, The machine learning layer further includes an evaluation module, which connects the training module and the result output layer. This evaluation module evaluates the performance of the quality score model using a test set before outputting the quality score model, in order to determine whether the quality score model meets preset requirements. The test set is a portion of the training data that does not overlap with the training set and the validation set. Meeting the preset requirements means that at least one of the following indicators of the classification prediction results of the quality score model on the test set—accuracy, precision, recall, and speed—meets the preset standard, or that the model is able to perform the user-input task.

15. The system according to claim 4, characterized in that, The machine learning layer further includes an evaluation module, which connects the training module and the result output layer. This evaluation module evaluates the performance of the quality score model using a test set before outputting the quality score model, in order to determine whether the quality score model meets preset requirements. The test set is a portion of the training data that does not overlap with the training set and the validation set. Meeting the preset requirements means that at least one of the following indicators of the classification prediction results of the quality score model on the test set—accuracy, precision, recall, and speed—meets the preset standard, or that the model is able to perform the user-input task.

16. The system according to any one of claims 1, 2, 5-12, 14 and 15, characterized in that, The user input also includes sequencing data, which includes images of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection during one or more sequencing runs.

17. The system according to any one of claims 1, 2, 5-12, 14 and 15, characterized in that, The user input also includes sequencing data, which is a subset of data extracted from the sequencing image. The sequencing data includes the position and intensity values ​​of multiple nucleic acid molecules in one or more surface regions. The sequencing image is an image of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection in one or more sequencing runs.

18. The system according to claim 4, characterized in that, The user input also includes sequencing data, which includes images of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection during one or more sequencing runs.

19. The system according to claim 4, characterized in that, The user input also includes sequencing data, which is a subset of data extracted from the sequencing image. The sequencing data includes the position and intensity values ​​of multiple nucleic acid molecules in one or more surface regions. The sequencing image is an image of nucleic acid molecules at multiple locations on one or more surfaces obtained from one or more rounds of detection in one or more sequencing runs.

20. The system according to claim 16, characterized in that, The sequencing data is processed using the base recognition model to obtain the sequencing results.

21. The system according to claim 17, characterized in that, The sequencing data is processed using the base recognition model to obtain the sequencing results.

22. The system according to claim 18 or 19, characterized in that, The sequencing data is processed using the base recognition model to obtain the sequencing results.

23. The system according to claim 20 or 21, characterized in that, The system also includes an automatic base recognition model generation system connected to the input layer, which is used to generate and output the base recognition model.

24. The system according to claim 22, characterized in that, The system also includes an automatic base recognition model generation system connected to the input layer, which is used to generate and output the base recognition model.

25. The system according to claim 23, characterized in that, The input layer is a first input layer, the machine learning layer is a first machine learning layer, the output layer is a first output layer, and the automatic base recognition generation system includes: The second input layer is used to receive user input from the user, including the sequencing data and optionally one or more tasks, wherein the sequencing data is obtained by detecting nucleic acid molecules connected to multiple locations on the surface using surface imaging sequencing technology; The second machine learning layer includes a second training data generation module and a second training module. The second training data generation module processes user input from the second input layer to generate second training data. The second training data includes multiple pairs of associated second features and second labels. The second features describe the intensity corresponding to one or more nucleic acid molecules at different locations. The second labels include bases selected from A, T, C, and G. The second training module is used to train a second specified model based on at least a portion of the second training data, including iteratively training the second specified model using the second training set and evaluating the performance of the second specified model after each iteration using the second validation set, in order to obtain a base recognition model, wherein the second training set and the second validation set are each independent, non-overlapping portions of the second training data; and The second output layer is used to output the base recognition model.

26. The system according to claim 24, characterized in that, The input layer is a first input layer, the machine learning layer is a first machine learning layer, the output layer is a first output layer, and the base recognition automatic generation system includes: The second input layer is used to receive user input from the user, including the sequencing data and optionally one or more tasks, wherein the sequencing data is obtained by detecting nucleic acid molecules connected to multiple locations on the surface using surface imaging sequencing technology; The second machine learning layer includes a second training data generation module and a second training module. The second training data generation module processes user input from the second input layer to generate second training data. The second training data includes multiple pairs of associated second features and second labels. The second features describe the intensity corresponding to one or more nucleic acid molecules at different locations. The second labels include bases selected from A, T, C, and G. The second training module is used to train a second specified model based on at least a portion of the second training data, including iteratively training the second specified model using the second training set and evaluating the performance of the second specified model after each iteration using the second validation set, in order to obtain a base recognition model, wherein the second training set and the second validation set are each independent, non-overlapping portions of the second training data; and The second output layer is used to output the base recognition model.

27. The system according to claim 25 or 26, characterized in that, The second tag also includes combinations of any two, three, or four bases selected from A, T, C, and G.

28. The system according to claim 25 or 26, characterized in that, The user input from the second input layer also includes a reference sequence, and the second training data generation module also includes a second alignment submodule. The second alignment submodule is used to align the sequence readout or base readout corresponding to the sequencing data to the reference sequence in order to generate a second alignment result. Based on the second comparison result, a second label associated with the second feature is determined in order to generate the second training data.

29. The system according to claim 25 or 26, characterized in that, The second machine learning layer further includes a second evaluation module connecting the second training module and the second output layer, used to evaluate the performance of the base recognition model using a second test set before outputting the base recognition model, so as to determine whether the base recognition model meets preset requirements, wherein... The second test set is a portion of the second training data that does not overlap with the second training set and the second validation set. The condition of meeting the preset requirement is that at least one of the following indicators of the base recognition model’s classification prediction results for the second test set meets the preset standard or is able to perform the user input task.

30. The system according to claim 25 or 26, characterized in that, The second specified model is selected from at least one of logistic regression, support vector machine, tree model and neural network.

31. The system according to any one of claims 1, 2, 5-12, 14, 15, 18-21, 24, characterized in that, The machine learning layer also includes a selection module for determining the specified model, the first specified model, or the second specified model based on the corresponding user input.

32. The system according to claim 25 or 26, characterized in that, The first machine learning layer or the second machine learning layer further includes a selection module for determining the specified model or the first specified model or the second specified model based on the corresponding user input.

33. A method for developing a quality scoring model, characterized in that, The system includes accessing user input using a system for automatically generating machine learning models according to any one of claims 1-32, and running the system using at least one hardware processor to present the best quality score model to the user. The user input includes sequencing results and optionally one or more tasks, the sequencing results being obtained by detecting nucleic acid molecules linked to multiple locations on a surface using surface imaging sequencing technology, the sequencing results including base recognition results and base recognition probabilities.

34. The method according to claim 33, characterized in that, The sequencing results include at least 10 million sequence reads and / or at least 1 billion base pairs reads.

35. The method according to claim 33 or 34, characterized in that, Receive the user input through a model generation framework or interface; and / or present the best quality score model to the user through the model generation framework or interface.

36. The method according to claim 33 or 34, characterized in that, The system is run using two or more operating systems, and at least one of file sharing, remote access, network communication, virtual machines, containers, and virtualization technologies or tools is used to enable interaction between the different operating systems.

37. The method according to claim 36, characterized in that, The system utilizes portions of both Windows and Linux, and enables interaction between the different operating systems using at least one of WSL and / or SSH.

38. The method according to any one of claims 33, 34, and 37, characterized in that, The system is run using multi-threaded parallel computing.

39. A quality scoring model, characterized in that, The system for automatically generating machine learning models is used according to any one of claims 1-32.

40. A computer-readable storage medium storing a plurality of instructions for controlling a processor to execute the method of any one of claims 33-38.

41. A system for automatically generating machine learning models, characterized in that, The system includes a memory storing a program and one or more processors coupled to the memory, the processors running the program to implement the method of any one of claims 33-38.

Images