A centromere and hierarchical transformer-based tumor detection method and system

Abstract

The application discloses a tumor detection method and system based on centromere and layered Transformer, relates to the technical field of tumor detection, and comprises the following steps: for cfDNA liquid biopsy, the problems of difficult modeling of centromere region signal and high calculation cost of long input are solved. Multidimensional features of a sample centromere region are extracted and standardized, and a structured sample input is constructed by aggregation according to the order of chromosomes and bins. Local fragments are generated by segmentation, local context is learned by a first-layer Transformer, a second-layer aggregation module models the global relationship across chromosomes, and a sample-level representation is obtained. Finally, a tumor positive probability and a judgment result are output by a classification module. The system is correspondingly divided into a collection module, a pretreatment module, a segmentation module, a layered processing module and an output module. The scheme considers local and global features, retains structural information, reduces calculation cost, improves tumor detection performance and is suitable for liquid biopsy early screening scenes.

Description

Technical Field

[0001] This invention relates to the field of tumor detection technology, and more specifically to a tumor detection method and system based on centromeres and hierarchical Transformers. Background Technology

[0002] Early cancer screening and detection are important directions in precision medicine. Current liquid biopsy protocols typically identify risks based on information such as mutations, methylation, fragment length, and copy number of circulating cell-free DNA (cfDNA). However, the utilization of centromere-related sequence features in cfDNA remains relatively limited. Centromere regions are characterized by high repetition, wide cross-chromosomal distribution, and complex local patterns. Using traditional shallow models or simple flattened feature input methods often fails to simultaneously capture both the internal patterns of local fragments and global information from multiple regions, thus limiting tumor detection performance.

[0003] For data objects with multiple regions, long inputs, and obvious hierarchical structures, directly modeling the entire dataset incurs significant computational overhead; while simply truncating or randomly sampling can easily disrupt the original structural relationships. Furthermore, traditional single-layer models struggle to simultaneously handle local dependencies and sample-level global relationships.

[0004] Therefore, how to propose a tumor detection method and system based on centromeres and hierarchical Transformers to overcome the shortcomings of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a tumor detection method and system based on centromeres and hierarchical Transformers, proposing a technical solution that is oriented towards centromere region sequence features, suitable for structured input, and capable of achieving efficient tumor detection. It solves the problems in existing technologies where it is difficult to simultaneously utilize local fragment information and sample-level global information for cfDNA centromere region-related signals, structural relationships are easily lost, and modeling costs are high in long input scenarios, thereby achieving efficient and automated tumor detection of tested samples. To achieve the above objectives, the present invention adopts the following technical solution: A tumor detection method based on centromeres and hierarchical Transformers, comprising: Acquire data from the tested samples and extract relevant features of the centromere region; The centromere region features from different chromosomes in the same sample are uniformly organized according to preset rules to form sample-level input objects; The sample-level input object is segmented to generate local fragments; Each local segment is input into the first layer of the Transformer model, and the contextual relationships between different positions within the local segment are learned through the attention mechanism. The corresponding local segment representation is output. Multiple local segment representations are input into the second layer aggregation module to model the relationships between segments and across chromosomes, and generate a sample-level global representation. Input the sample-level global representation into the classification module, and output the tumor detection results of the tested sample.

[0006] Optionally, acquiring the tested sample data and extracting centromere region-related features includes: Let the raw observations at the t-th site on the c-th chromosome be: reference base type, sequencing depth, variant support count, soft truncation support count, and higher-order repetitive sequence region indicator. One-hot encoding of the reference base type yields a four-dimensional base encoding vector. For numerical features, logarithmic transformations are performed on sequencing depth, variant support count, and soft truncation support count to construct proportional features to characterize relative anomaly intensity. Finally, the feature vector of the t-th site on the c-th chromosome is obtained.

[0007] Optionally, this also includes: standardizing numerical features.

[0008] Optionally, the step of uniformly organizing centromere region-related features from different chromosomes in the same sample according to preset rules to form a sample-level input object includes: For the centromere-related regions on each chromosome, the first base site of the higher-order repetitive sequence region is used as the alignment origin; For a preset site on chromosome c, the chromosome is divided into bins according to a fixed bin length. The base-wise feature vectors within the same bin are aggregated to obtain the input unit sequence formed by chromosome c. The sequences of each chromosome input unit are spliced ​​together in chromosome number order, and separator vectors are inserted between adjacent chromosomes to form a sample-level input sequence.

[0009] Optionally, the step of segmenting the sample-level input object to generate local fragments includes: The sample-level input sequence is segmented using a fixed-length segmentation method; Calculate the proportion of high-order repeating sequence regions in each local segment, and divide the local segments into high-order repeating sequence enriched segments, boundary segments, and non-high-order repeating sequence segments based on the size of the region proportion.

[0010] Optionally, the step of inputting each local segment into the first layer of the Transformer model, learning the contextual relationships between different positions within the local segment through an attention mechanism, and outputting the corresponding local segment representation includes: A linear mapping is performed on each input unit vector in a local segment to obtain the hidden layer representation; The hidden layer representation is input into the local self-attention encoder to obtain the encoding result; Attention convergence is applied to the encoded results to obtain the representation vectors corresponding to the local segments.

[0011] Optionally, the step of inputting multiple local fragment representations into the second-layer aggregation module to model the relationships between fragments and across chromosomes, generating a sample-level global representation, includes: Arrange all the obtained local fragment representations in their original order to form a local fragment representation sequence; The local segment representation sequence is input into the second layer of the self-attention encoder to obtain the global encoding result; Attention convergence is performed on the global encoding results to calculate the sample-level global representation vector.

[0012] Optionally, the step of inputting the sample-level global representation into the classification module and outputting the tumor detection result of the tested sample includes: The obtained sample-level representation vector g is input into the classification layer to obtain the classification score of the sample: The classification score is input into the compression mapping function to obtain the predicted probability that the tested sample is tumor-positive.

[0013] Optionally, it also includes: using a binary cross-entropy loss function to optimize the model during the model training phase.

[0014] Optionally, a tumor detection system based on centromeres and hierarchical Transformers includes: an acquisition module for acquiring data of the tested sample and extracting centromere-related features; Preprocessing module: Used to organize the centromere region related features from different chromosomes in the same sample according to preset rules to form a sample-level input object; Segmentation module: Used to segment sample-level input objects to generate local segments; Hierarchical processing module: It is used to input each local segment into the first layer Transformer model, learn the contextual relationship between different positions within the local segment through the attention mechanism, and output the corresponding local segment representation. Multiple local segment representations are input into the second layer aggregation module to model the relationship between segments and across chromosomes, and generate a sample-level global representation. Output module: Used to input the sample-level global representation into the classification module and output the tumor detection results of the tested sample.

[0015] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a tumor detection method and system based on centromeres and layered Transformers, which has the following beneficial effects: This invention (1) improves the ability to characterize complex and weak signals in the centromere region. Given the high repetition of centromere region signals, wide cross-chromosomal distribution, and complex local patterns in cfDNA, sample-level inputs are constructed using centromere region sequence features and their derived features. This approach is no longer limited to shallow discrimination of a single region or a single short sequence, and is therefore more conducive to identifying tumor-related patterns distributed in multiple regions and fragments.

[0016] (2) Simultaneously consider local feature extraction and global relationship modeling

[0017] Employing a hierarchical Transformer structure, this invention first encodes local fragments and then aggregates multiple local fragments at the sample level. This allows for the simultaneous learning of intra-fragment dependencies as well as global relationships between fragments and across chromosomes. Compared to simple flattened input methods or single-layer shallow models, this invention is more suitable for processing biological signals with multiple regions and distinct hierarchical structures.

[0018] (3) It is beneficial to maintain structural information under limited input conditions.

[0019] By segmenting the sample-level input and preserving the order and boundary relationships of multiple chromosomes during the generation and selection of local segments, the loss of original structural information can be minimized while controlling computational complexity, making it more suitable for engineering applications in long input scenarios.

[0020] (4) It has good stability and scalability.

[0021] The method chain of this invention includes feature construction, structured input, local encoding, global aggregation and classification output, etc. The division of labor between each step is clear, which makes it easy to adjust the input features, training methods and output forms according to different detection scenarios. Therefore, it has good stability, scalability and engineering adaptability. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a tumor detection method based on centromeres and layered Transformers provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the multi-chromosome structured input provided by the present invention.

[0025] Figure 3 The hierarchical Transformer model structure diagram provided by this invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention discloses a tumor detection method based on centromeres and hierarchical Transformers, such as... Figure 1 As shown, it includes: Acquire data from the tested samples and extract relevant features of the centromere region; The centromere region features from different chromosomes in the same sample are uniformly organized according to preset rules to form sample-level input objects; The sample-level input object is segmented to generate local fragments; Each local segment is input into the first layer of the Transformer model, and the contextual relationships between different positions within the local segment are learned through the attention mechanism. The corresponding local segment representation is output. Multiple local segment representations are input into the second layer aggregation module to model the relationships between segments and across chromosomes, and generate a sample-level global representation. Input the sample-level global representation into the classification module, and output the tumor detection results of the tested sample.

[0028] In a specific embodiment, the present invention provides a tumor detection method based on centromeres and layered Transformers, comprising the following specific steps: Step 1: Acquire the data of the tested sample and extract relevant features of the centromere region. Obtain sample data from the subject of the test, preferably cfDNA data, and extract sequence information related to the centromere region and statistical features, structured features or combined features constructed based on the sequence information to form the initial input data.

[0029] The specific process involves extracting the base sites located in the centromere-related region and higher-order repetitive sequence region after alignment with the reference genome: Let the raw observations at the t-th locus on the c-th chromosome be: (refer to base type, sequencing depth, variant support count, soft truncation support count, and higher-order repetitive sequence region indicator). The reference base type represents the base class of the locus on the reference genome; sequencing depth represents the number of sequencing reads covering the locus; variant support count represents the number of reads supporting a variant at the locus; soft truncation support count represents the number of reads near the locus with soft truncation alignment markers; and higher-order repeat region indicator indicates whether the locus is located within a higher-order repeat region, with a value of one if it is located within the region and zero otherwise. One-hot encoding is performed on the reference base type to obtain a four-dimensional base encoding vector. For numerical features, logarithmic transformations are performed on sequencing depth, variant support count, and soft truncation support count, respectively. ; ; ; Furthermore, proportional features are constructed to characterize the relative anomaly intensity: ; ; Here, ε is a smoothing term used to avoid the denominator being zero.

[0030] Furthermore, the numerical features mentioned above are standardized. Let the mean of the k-th dimension numerical feature in the training samples be... The standard deviation is The standardization result is: ; in, To prevent extremely small positive numbers with a denominator of zero.

[0031] Therefore, the feature vector of the t-th locus on the c-th chromosome is constructed as follows: (One-hot encoding of bases, normalized depth features, normalized variation features, normalized soft truncation features, variation ratio features, soft truncation ratio features, high-order repeat sequence region indicator features) In this embodiment, each site forms a ten-dimensional feature vector, which is composed of four-dimensional base encoding, five-dimensional numerical features, and one-dimensional region indication features.

[0032] The purpose of this step is to convert the raw sequencing signal into site-level structured features that can be processed by the model, while completing the necessary feature processing and normalization.

[0033] The centromere region-related features can be either original sequence features or derived features obtained through further calculation, aggregation, or transformation based on the original sequence. This step aims to convert the raw cfDNA data into a model input basis capable of characterizing centromere region feature patterns, providing data support for subsequent structured organization and model building.

[0034] Step 2: Constructing a multi-chromosome structured input

[0035] like Figure 2 As shown, centromere region-related features from different chromosomes within the same sample are uniformly organized according to preset rules to form a sample-level input object. These preset rules can be manifested as organizing according to chromosome order, preserving boundary information between different chromosomes, recording the relative position information of different segments in the overall sample input, or distinguishing different chromosome origins through connection markers, boundary markers, or index information.

[0036] For the centromere-related region on each chromosome, the first base site of the higher-order repetitive sequence region is used as the alignment origin. For the position on chromosome c, ... The site is divided into bins according to a fixed bin length. In this embodiment, the bin length is 171 base pairs. The bin number to which this site belongs is defined as: ; Aggregate the base-by-base feature vectors within the same bin. Let the set of sites contained in the j-th bin of the c-th chromosome be denoted as . The input unit vector corresponding to this bin is defined as: ; in, This represents the input unit vector corresponding to the j-th bin of the c-th chromosome.

[0037] Thus, the c-th chromosome forms the input unit sequence: ; in, This represents the number of input units formed by the c-th chromosome within the centromere-related region.

[0038] The sequences of each chromosome input unit are concatenated in chromosome number order, and separator vectors are inserted between adjacent chromosomes to form a sample-level input sequence: ; Where C represents the number of chromosomes involved in the modeling.

[0039] By using the above method, the base-by-base site features are uniformly converted into sample-level sequence representations, which effectively reduces the model input length while preserving the chromosome sequence relationship and the spatial structure relationship of the centromere region.

[0040] The purpose of this step is to preserve the structural relationships from multiple chromosomes, so that subsequent models can not only process local regional features, but also utilize the tissue information at the overall sample level, thereby improving the ability to express complex biological signals.

[0041] Step 3: Segment the sample-level input to generate local fragments.

[0042] The sample-level input is divided into multiple local segments, which serve as the basic units for subsequent model processing. These local segments can be generated using fixed-length segmentation, sliding window segmentation, adaptive segmentation, or a combination thereof. When the model input budget is limited, local segments can be filtered according to preset selection rules to balance effective region coverage, preservation of the original structural order, and control of computational complexity. This step decomposes the long input into local segments suitable for Transformer processing, while providing a foundation for subsequent local modeling and global aggregation.

[0043] Let the sample-level input sequence obtained in step two be: ; Where L represents the total number of sample-level input units.

[0044] The sample-level input sequence is segmented using a fixed-length segmentation method. Let the length of each local segment be m, and the step size be s. Then, the i-th local segment is defined as: ; Where i = 1, 2, ..., N, the total number of local segments N satisfies: ; In this embodiment, if the end of the sample-level input sequence is less than a complete local segment, no further local segments will be constructed for the insufficient portion.

[0045] Furthermore, to control the model input budget and enhance training coverage, the proportion of high-order repetitive sequence regions in each local segment can be calculated. Let the region indication feature of the t-th input unit in the i-th local segment be... The region proportion of this local segment is defined as: ; Based on the size of the region, local segments can be divided into high-order repeat sequence enriched segments, boundary segments, and non-high-order repeat sequence segments for stratified sampling in subsequent training phases.

[0046] The purpose of this step is to divide the long sample-level input sequence into multiple local segments suitable for local modeling, thereby reducing computational complexity and preserving local contextual relationships.

[0047] Step 4: Encode the local segment using the first-layer Transformer.

[0048] Each local segment is input into the first layer of the Transformer model, and the attention mechanism is used to learn the contextual relationships between different positions within the local segment, and the corresponding local segment representation is output.

[0049] Before encoding local segments, linear mapping, positional encoding, normalization, masking, or combinations thereof can be performed on the input. After Transformer encoding, pooling, weighted summation, attention convergence, or combinations thereof can be used to generate a single local segment representation. The purpose of this step is to extract the sequence patterns, statistical patterns, and positional dependencies within the local segments, thereby forming a discriminative local representation.

[0050] Each local segment obtained in step three is input into the first layer of the self-attention encoder. First, a linear mapping is performed on each input unit vector in the local segment to obtain the hidden layer representation: ; in, This represents the input matrix corresponding to the i-th local segment. Represents the projection matrix. This represents the position encoding matrix.

[0051] Then, Inputting the local self-attention encoder yields the encoding result: ; in, This represents the hidden layer representation of the t-th input unit within the i-th local segment after the first layer of encoding.

[0052] To compress an entire local segment into a single segment representation, for Perform attention convergence. Let the attention weight at position t within the i-th local segment be: ; in, , and These are learnable parameters.

[0053] Then the representation vector corresponding to the i-th local segment is: ; In this way, the first-layer self-attention encoder completes the modeling of the contextual relationships between input units within a local segment and outputs a local segment-level representation vector.

[0054] Step 5: Generate sample-level representations using the second-layer aggregation module.

[0055] Multiple local fragment representations are input into the second-layer aggregation module to model the relationships between fragments and across chromosomes, generating a sample-level global representation. This second-layer aggregation module can employ a Transformer-based global relationship modeling approach, or it can utilize attention-based convergence, weighted pooling-based fusion, gating-based feature fusion, or a combination thereof. The purpose of this step is to elevate local fragment information to a sample-level global representation, enabling the model to integrate signals from multiple local regions and obtain an overall discriminative basis for tumor detection.

[0056] Arrange all the local fragment representations obtained in step four in their original order to form a local fragment representation sequence: ; Where N represents the number of local segments, This represents the representation vector of the i-th local segment.

[0057] The local segment representation sequence is input into the second-layer self-attention encoder to obtain the global encoding result: ; in, This represents the updated representation of the i-th local segment in the global context.

[0058] To obtain a sample-level global representation, attention convergence is performed on the global encoding results. Let the global attention weight of the i-th local segment be: ; in, , and These are learnable parameters.

[0059] The sample-level representation vector is then defined as: ; In this way, the second-layer self-attention encoder models the relationships between local segments and between segments across chromosomes, and elevates local information to a sample-level global representation.

[0060] Step 6: Output tumor detection results based on sample-level representation

[0061] like Figure 3As shown, the sample-level global representation is input into the classification module, which outputs the tumor detection result of the tested sample. The tumor detection result can be a classification result of tumor-related status or non-tumor status. The purpose of this step is to determine whether the tested sample has a tumor-related risk based on the sample-level representation learned by the hierarchical Transformer, thereby completing the tumor detection task.

[0062] Input the sample-level representation vector g obtained in step five into the classification layer to obtain the classification score of the sample: ; in, and These are the weight parameters and bias parameters of the classification layer, respectively.

[0063] The classification score s is input into the compression mapping function to obtain the predicted probability that the tested sample is tumor-positive: ; Set a judgment threshold τ, when When the value is ≥τ, the sample is considered to be tumor-positive; when... When the value is less than τ, the sample is considered negative.

[0064] During the model training phase, a binary classification cross-entropy loss function is used to optimize the model. For a sample with the true label y, the classification loss is defined as: ; For M samples in a batch, the overall classification loss is: ; The above methods enable the probability prediction and final classification of whether the tested sample has tumor-related risks.

[0065] In a specific embodiment, a biological sample, preferably a blood sample, is first collected from the subject, and cfDNA is extracted from it. Subsequently, sequence information related to the centromere region is obtained, or statistical and structured features are constructed based on this sequence information to form input data suitable for subsequent modeling. The features preferably include base composition-related features, coverage depth-related features, variation-related features, soft truncation-related features, and centromere-related region proportion features, etc.

[0066] Furthermore, centromere region-related features from multiple chromosomes within the same sample are uniformly integrated into a sample-level input object. To preserve the structural relationships of multi-chromosome samples, chromosome order information, boundary position information, or relative segment position information can be retained in the sample-level input object, and separator markers can be set as needed. In this way, relevant features from different chromosomes can be uniformly expressed in the same input object, providing a foundation for subsequent models to utilize local and overall structural information.

[0067] Based on this, the sample-level input is divided into multiple local segments, which serve as the basic units for model processing. These local segments can be generated using fixed-length segmentation, sliding window segmentation, adaptive segmentation, or a combination thereof. When the input length is long and the model budget is limited, the local segments can also be filtered according to preset rules to maintain the original structural order and boundary relationships as much as possible while controlling computational complexity.

[0068] In terms of model structure, this embodiment employs a hierarchical Transformer model. The first layer encodes the internal features of local segments and learns the contextual relationships between different locations within the local segment through an attention mechanism, resulting in a local segment representation. The second layer aggregates multiple local segment representations at the sample level to model the relationships between segments and across chromosomes, generating a global representation that characterizes the entire tested sample. Subsequently, the sample-level representation is input into the classification module, which outputs the tumor detection result. The output result can be a classification result for tumor-related or non-tumor states, or it can be a corresponding risk score or confidence information.

[0069] Furthermore, features can be normalized before input into the model, preferably sample-level normalization, to enhance the statistical comparability between different samples.

[0070] Furthermore, masking, suppression, replacement, or consistency control can be applied to some input features during the training or inference phase, or input segments can be optimized by combining filament-related region information, boundary information, or sequence information during the local segment selection phase, in order to improve the robustness and structure preservation ability of the model.

[0071] Furthermore, the hierarchical Transformer model can be obtained directly using supervised training, or it can be pre-trained and then supervised training using labeled samples. Through the above implementation methods, the present invention can construct sample-level inputs suitable for hierarchical Transformer modeling from the centromere region sequence features and their derived features in cfDNA, and achieve tumor detection through a method of local fragment encoding—sample-level aggregation—classification output.

[0072] In a specific implementation, a tumor detection method based on centromeres and hierarchical Transformers includes the following steps: First, obtain the sequencing data file corresponding to the tested object. The sequencing data file can be the raw data file formed after the sample is sequenced, the alignment result file, or a data file that can be directly used for site statistical analysis.

[0073] Based on the sequencing data file, site-by-site observation information was extracted from the centromere-related region and higher-order repetitive sequence regions. For the t-th base site on the c-th chromosome, the following raw features were extracted: reference base type. Sequencing depth Mutation support count Soft truncation support counting and area indication Wherein, when the site is located within a higher-order repeat sequence region, =1; otherwise, =0.

[0074] For reference base types One-hot encoding is performed to obtain a four-dimensional base encoding vector. For numerical features, logarithmic transformations were performed on sequencing depth, variant support count, and soft truncation support count, respectively, and defined as follows: ; ; ; Furthermore, to characterize the relative intensity of the abnormal signal at this site, variation ratio features and soft truncation ratio features are constructed: ; ; in, It is a smoothing constant greater than 0, used to avoid the denominator being 0.

[0075] To enhance the statistical comparability between different samples, the above numerical features are standardized. Let the mean and standard deviation of the k-th dimension numerical feature be respectively... and The standardization result is: ; in, To prevent extremely small positive numbers with a denominator of 0.

[0076] After the above processing, the feature vector of the t-th locus on the c-th chromosome is constructed as follows: ; in, It is a four-dimensional base encoding vector. , , For standardized numerical features, and These are the variation ratio feature and the soft truncation ratio feature, respectively. This serves as a regional indicator. Consequently, each base site forms a ten-dimensional feature vector.

[0077] In this embodiment, instead of directly inputting all site-specific features into the model, fixed-length binning and aggregation are further performed. Specifically, the starting position of the centromere higher-order repeat sequence region is used as the alignment origin. For the position on the c-th chromosome... The site is binned according to a fixed length of 171 base pairs. The bin number to which the site belongs is defined as follows: ; in, This indicates the starting position of the higher-order repeat sequence region of the centromere corresponding to chromosome c.

[0078] Let the set of loci contained in the j-th bin of the c-th chromosome be . The input unit vector corresponding to this bin is defined as: ; Thus, the c-th chromosome forms the input unit sequence: ; in, This represents the number of input units formed by the c-th chromosome within the centromere-related region.

[0079] Furthermore, the input unit sequences from multiple chromosomes are concatenated according to chromosome numbering order, and separator vectors are inserted between adjacent chromosomes to form sample-level input sequences: ; in, Let C represent the separator vector, and C represent the number of chromosomes involved in the modeling. This method allows for a unified representation of centromere-related features from different chromosomes within the same input object, while preserving chromosome order and boundary relationships.

[0080] Based on this, the sample-level input sequence is divided into multiple local segments, which serve as the basic units for model processing. Let the sample-level input sequence be: ; Where L represents the total number of sample-level input units. Local segments are generated using a fixed-length segmentation method. Let the length of each local segment be m and the step size be s, then the i-th local segment is defined as: ; Where i = 1, 2, ..., N, the total number of local segments N satisfies: ; In this embodiment, no new segments are constructed for portions that are less than a complete local segment at the end.

[0081] Furthermore, local segments can be hierarchically filtered based on the proportion of related input units in the high-order repeating sequence region within the local segment, so as to retain the core structural information and boundary information of the filament region while controlling the computational complexity.

[0082] In terms of model structure, this implementation adopts a hierarchical self-attention coding model. The first layer is used to encode the features within a local segment. Specifically, the input units in the local segment are first linearly mapped and then positional codes are superimposed to obtain the initial hidden layer representation: ; in, Represents the projection matrix. This represents the position encoding matrix. Subsequently, Input the local encoder to obtain the contextual representation of the local segment: ; in, This represents the hidden layer representation of the t-th position in the i-th local segment after encoding.

[0083] To obtain a local fragment-level representation, attention is converged on the representations of each position within the local fragment. Let the attention weight of the t-th position in the i-th local fragment be: ; Then the representation vector corresponding to the i-th local segment is: ; The second layer is used for sample-level aggregation of multiple local fragment representations. All local fragment representations are then arranged into a sequence in their original order. ; The local segment representation sequence is input into the global encoder to obtain the global context representation: ; in, This represents the updated representation of the i-th local segment in the global context.

[0084] Furthermore, attention is converged on the global representations of all local segments to obtain sample-level representation vectors: ; ; in, denoted by , where represents the weight of the i-th local segment in the sample-level aggregation, and g represents the global representation vector of the tested sample.

[0085] Subsequently, the sample-level representation vector is input into the classification layer, and the tumor detection result is output. The classification score is defined as: ; The classification score is input into a compression mapping function to obtain the predicted probability that the tested sample is tumor-positive: ; Set a judgment threshold τ, when When the value is ≥τ, the sample is considered to be tumor-positive; when... When the value is less than τ, the sample is considered negative. The output result is a classification result of tumor positive or negative, and the corresponding risk probability value can be further output.

[0086] During the model training phase, a binary classification cross-entropy loss function is used to optimize the model parameters. For a sample with the true label y, the classification loss is defined as: ; For M samples in a training batch, the overall loss is defined as: ; Furthermore, to avoid excessive concentration of sample-level attention weights on a few local segments, an entropy regularization term can be introduced into the weights of these local segments, defined as: ; The total loss function is then written as: ; Where λ is the non-negative regularization coefficient.

[0087] In a specific embodiment, sequencing data files corresponding to a tested object are obtained, and site-by-site statistical information located in centromere-related regions and higher-order repetitive sequence regions is extracted from them. At a certain site on chromosome 1, the sequencing depth is 20, the variant support count is 3, the soft truncation support count is 2, and this site is located within a higher-order repetitive sequence region. Therefore: d = log(1 + 20); v'=log(1+3); s' = log(1 + 2); If we take ε=10 -6 Then the mutation ratio and soft cutoff ratio are respectively: u=3 / (20+10 -6 )≈0.15; q=2 / (20+10 -6 )≈0.10; Suppose that after standardization, the three numerical transformation features corresponding to this site are 0.62, 0.48, and 0.31, respectively, and their variation ratio and soft truncation ratio are 0.15 and 0.10, respectively. The one-hot encoding of the base is [1, 0, 0, 0], and the region indicator is 1. Then, the feature vector of this site can be expressed as: x=[1, 0, 0, 0, 0.62, 0.48, 0.31, 0.15, 0.10, 1]; The average of the feature vectors from 171 consecutive base sites yields an input unit vector. All input units are concatenated in chromosomal order to form a sample-level input sequence, which is then segmented into multiple local fragments. After a first-layer local encoding and a second-layer global aggregation, the global representation vector of the sample is obtained. If the classification score output by the classification layer is 1.39, then the tumor-positive prediction probability of this sample is: ; When the threshold τ is set to 0.50, since 0.80 is greater than 0.50, the sample is determined to be a tumor-positive sample.

[0088] As can be seen from the above embodiments, the present invention can gradually convert the raw sequencing signals of centromere-related regions and higher-order repetitive sequence regions in sequencing data files into sample-level structured inputs suitable for processing by hierarchical self-attention coding models, and achieve tumor detection through a technical path of local fragment coding—sample-level aggregation—classification output. It has clear processing steps, clear mathematical expression, and good engineering feasibility.

[0089] In a specific implementation, a tumor detection system based on centromeres and hierarchical Transformers includes: an acquisition module for acquiring data of the tested sample and extracting centromere region-related features; Preprocessing module: Used to organize the centromere region related features from different chromosomes in the same sample according to preset rules to form a sample-level input object; Segmentation module: Used to segment sample-level input objects to generate local segments; Hierarchical processing module: It is used to input each local segment into the first layer Transformer model, learn the contextual relationship between different positions within the local segment through the attention mechanism, and output the corresponding local segment representation. Multiple local segment representations are input into the second layer aggregation module to model the relationship between segments and across chromosomes, and generate a sample-level global representation. Output module: Used to input the sample-level global representation into the classification module and output the tumor detection results of the tested sample.

[0090] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0091] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A tumor detection method based on centromeres and hierarchical Transformers, characterized in that, include: Acquire data from the tested samples and extract relevant features of the centromere region; The centromere region features from different chromosomes in the same sample are uniformly organized according to preset rules to form sample-level input objects; The sample-level input object is segmented to generate local fragments; Each local segment is input into the first layer of the Transformer model, and the contextual relationships between different positions within the local segment are learned through the attention mechanism. The corresponding local segment representation is output. Multiple local segment representations are input into the second layer aggregation module to model the relationships between segments and across chromosomes, and generate a sample-level global representation. Input the sample-level global representation into the classification module, and output the tumor detection results of the tested sample.

2. The tumor detection method based on centromeres and hierarchical Transformers according to claim 1, characterized in that, The process of acquiring the tested sample data and extracting centromere region-related features includes: Let the raw observations at the t-th site on the c-th chromosome be: reference base type, sequencing depth, variant support count, soft truncation support count, and higher-order repetitive sequence region indicator. One-hot encoding of the reference base type yields a four-dimensional base encoding vector. For numerical features, logarithmic transformations are performed on sequencing depth, variant support count, and soft truncation support count to construct proportional features to characterize relative anomaly intensity. Finally, the feature vector of the t-th site on the c-th chromosome is obtained.

3. The tumor detection method based on centromeres and hierarchical Transformers according to claim 2, characterized in that, Also includes: Standardize the numerical features.

4. The tumor detection method based on centromeres and hierarchical Transformer according to claim 1, characterized in that, The process of uniformly organizing centromere region-related features from different chromosomes within the same sample according to preset rules to form a sample-level input object includes: For the centromere-related regions on each chromosome, the first base site of the higher-order repetitive sequence region is used as the alignment origin; For a preset site on chromosome c, the chromosome is divided into bins according to a fixed bin length. The base-wise feature vectors within the same bin are aggregated to obtain the input unit sequence formed by chromosome c. The sequences of each chromosome input unit are spliced ​​together in chromosome number order, and separator vectors are inserted between adjacent chromosomes to form a sample-level input sequence.

5. The tumor detection method based on centromeres and hierarchical Transformers according to claim 1, characterized in that, The segmentation of the sample-level input object to generate local fragments includes: The sample-level input sequence is segmented using a fixed-length segmentation method; Calculate the proportion of high-order repeating sequence regions in each local segment, and divide the local segments into high-order repeating sequence enriched segments, boundary segments, and non-high-order repeating sequence segments based on the size of the region proportion.

6. The tumor detection method based on centromeres and hierarchical Transformer according to claim 1, characterized in that, The process involves inputting each local segment into the first layer of the Transformer model, learning the contextual relationships between different positions within the local segment through an attention mechanism, and outputting the corresponding local segment representation, including: A linear mapping is performed on each input unit vector in a local segment to obtain the hidden layer representation; The hidden layer representation is input into the local self-attention encoder to obtain the encoding result; Attention convergence is applied to the encoded results to obtain the representation vectors corresponding to the local segments.

7. The tumor detection method based on centromeres and hierarchical Transformer according to claim 1, characterized in that, The process of inputting multiple local fragment representations into the second-layer aggregation module to model the relationships between fragments and across chromosomes generates a sample-level global representation, including: Arrange all the obtained local fragment representations in their original order to form a local fragment representation sequence; The local segment representation sequence is input into the second layer of the self-attention encoder to obtain the global encoding result; Attention convergence is performed on the global encoding results to calculate the sample-level global representation vector.

8. The tumor detection method based on centromeres and hierarchical Transformer according to claim 1, characterized in that, The step of inputting the sample-level global representation into the classification module and outputting the tumor detection results of the tested sample includes: The obtained sample-level representation vector g is input into the classification layer to obtain the classification score of the sample: The classification score is input into the compression mapping function to obtain the predicted probability that the tested sample is tumor-positive.

9. A tumor detection method based on centromeres and hierarchical Transformers according to claim 8, characterized in that, Also includes: During the model training phase, the binary classification cross-entropy loss function is used to optimize the model.

10. A tumor detection system based on centromeres and hierarchical Transformers, characterized in that, Includes: Acquisition module: used to acquire data from the tested sample and extract relevant features from the centromere region; Preprocessing module: Used to organize the centromere region related features from different chromosomes in the same sample according to preset rules to form a sample-level input object; Segmentation module: Used to segment sample-level input objects to generate local segments; Hierarchical processing module: It is used to input each local segment into the first layer Transformer model, learn the contextual relationship between different positions within the local segment through the attention mechanism, and output the corresponding local segment representation. Multiple local segment representations are input into the second layer aggregation module to model the relationship between segments and across chromosomes, and generate a sample-level global representation. Output module: Used to input the sample-level global representation into the classification module and output the tumor detection results of the tested sample.

Images