Effective error correction method and system based on DNA data storage IDS channel multiple outputs

Abstract

The invention relates to the field of information storage, in particular to an effective error correction method and system based on DNA data storage IDS channel multiple outputs. The method comprises the following steps: carrying out internal coding on an input information bit sequence and then carrying out external coding to generate an external coding bit sequence; mapping to obtain a DNA base sequence, and segmenting the DNA base sequence into M sections of short sequences; m * N short output sequences are obtained through N parallel IDS channels; performing splicing through SPM algorithm processing to obtain a consensus sequence; decoding the consensus sequence through synchronous decoding, and then decoding through an ENMS algorithm; and iteration is carried out on the synchronous decoding and the ENMS algorithm through a joint iterative algorithm. According to the method, multiple outputs are obtained through the parallel IDS channels, the consensus sequence is obtained by combining the segmented progressive comparison algorithm with the homologous information of the multiple outputs, errors in the DNA synthesis and sequencing process can be effectively avoided, then decoding is performed by using the synchronous decoding and ENMS algorithm, joint iteration is performed, and the error correction performance is further improved.

Description

Technical Field

[0001] The present invention relates to the field of information storage, and more specifically, to an effective error correction method and system based on multiple outputs of a DNA data storage IDS channel. Background Art

[0002] With the rapid development of information technology, the amount of global data has exploded. The main storage media currently used include mechanical hard disks, solid-state drives, blue-ray memory and flash memory, but they all have problems such as limited storage density, short shelf life and environmental pollution. However, with the advancement of genome synthesis and assembly technology, research on DNA molecules as storage media has attracted increasing attention from the academic community. As a molecule that encodes biological information, DNA has the advantages of long storage cycle, high storage density and low energy consumption, and is considered to be a promising storage medium. DNA communication is different from binary communication. DNA data storage includes four major processes: encoding, synthesis, nanopore sequencing and decoding. Among them, errors in the form of insertion, deletion and substitution may be introduced during DNA synthesis and sequencing. This results in poor storage effect.

[0003] The prior art discloses a DNA storage error correction method and system based on a generative adversarial network, wherein the method includes: generating a DNA sequence data set based on a DNA template chain with uniform distribution of A, T, C, and G, and dividing it into a training set and a test set; constructing a generative adversarial network model GAN, training the generative adversarial network model GAN ​​based on the training set, and obtaining a trained GAN model after testing with a test set; sequencing the stored DNA molecular sequence, clustering and screening according to the sequencing results, and obtaining a DNA cluster; selecting a suitable DNA cluster according to a preset rule, performing image conversion on the sequence information obtained by sequencing according to the preset rule, and generating a corresponding sequence image; obtaining the corrected image information by passing the generated sequence image through the generator of the GAN model, and then restoring it to the coding sequence according to the rule to complete the error correction. However, this method still cannot solve the errors in the form of insertion, deletion, and substitution that may be introduced during DNA synthesis and sequencing. This results in poor storage effect. Summary of the invention

[0004] The purpose of the present invention is to disclose an effective error correction method and system based on multiple outputs of DNA data storage IDS channels with better storage effect.

[0005] In order to achieve the above object, the present invention provides an effective error correction method based on multiple outputs of a DNA data storage IDS channel, comprising:

[0006] S1: Perform internal encoding on the input information bit sequence to obtain an internal encoding bit sequence, and then perform external encoding on the internal encoding bit sequence to generate an external encoding bit sequence;

[0007] S2: Map the external coding bit sequence to obtain a DNA base sequence, and divide the DNA base sequence into M short sequences;

[0008] S3: Each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences;

[0009] S4: Process M*N short output sequences by SPM algorithm to obtain M short consensus sequences; splice the M short consensus sequences to obtain a consensus sequence;

[0010] S5: decoding the consensus sequence by synchronous decoding first, and then decoding it by ENMS algorithm to obtain information; decoding by synchronous decoding first, and then decoding by ENMS algorithm includes: iterating synchronous decoding and ENMS algorithm by joint iterative algorithm.

[0011] Further, in step S1, performing internal encoding on the input information bit sequence to obtain the internal encoding bit sequence includes: using (D E ,D S ) embedded marker code is used to internally encode the input information bit sequence to obtain the internal encoding bit sequence c′=(c1,c2,...,c k ), D E represents the interval between two marker bit sequences, D S Indicates the length of the marker code.

[0012] Further, in step S1, performing external encoding on the internal coding bit sequence to generate an external coding bit sequence includes: using a QCLDPC code with a code type of [n, k] as an external code to perform external encoding to generate an external coding bit sequence u = (u1, u2, ..., u n ), where (nk) is the length of the check bits.

[0013] Further, in step S2, mapping the external coding bit sequence to obtain a DNA base sequence includes: mapping the external coding bit sequence to a DNA base sequence according to a mapping scheme of 00-A, 01-C, 10-T, and 11-G, and the DNA base sequence includes four base symbols of ATCG.

[0014] Further, in step S3, one of the M short sequences is passed through N parallel IDS channels to obtain N short output sequences, including:

[0015] One of the short sequences passes through an IDS channel to obtain an output sequence as follows:

[0016] Let w=(w1,w2,...,w n) is a short sequence to be transmitted through the channel, which includes multiple base symbols w i ,y=(y1,y2,...,y n′ ) is the output sequence; base symbol w i When entering the channel, the bases of the received sequence may appear in four situations:

[0017] Base symbol w i With p i The probability of an insertion error occurs, where a uniformly random symbol a∈{A, T, C, G} is inserted into base y of the output sequence i′ On, the base symbol w i is held in the input channel and corresponds to the received symbol y i′+1 ;

[0018] Base symbol w i With p d The probability of sending is deleted, then no symbol is received, and then the base symbol w is sent i+1 ;

[0019] Base symbol w i The probability of transmission is p t =1-p i -p d However, there are also p s The probability of substitution error occurs, therefore, the probability that the input base symbol is successfully transmitted and correctly received is p t (1-p s ), and then receives the base symbol y i′ =w i ;

[0020] The base symbol is p t p s The probability of substitution error is , then a uniform random symbol a is received, where a≠w i ;

[0021] When the channel receives the last symbol w n When , the transmission of a short sequence is completed and an output sequence is obtained.

[0022] Furthermore, in step S4, it includes:

[0023] Quickly align any two sequences from multiple output sequences, and use the Levenshtein distance to calculate the scoring matrix, where matches, mismatches, and gaps are 1 point, -1 point, and 0 point, respectively; thereby obtaining the total distance score of the alignment of any two sequences to obtain a distance matrix of size M×(M-1) / 2, where M is the number of output sequences; establish a guidance tree and calculate the weight of each branch, where each node in the guidance tree corresponds to the order of comparison of multiple sequences, and finally compare them pairwise in the order of sequence alignment to obtain the base symbol weight of each sequence, and deduce the base symbol based on the base weight to obtain the consensus sequence.

[0024] Furthermore, in step S5, the synchronous decoding algorithm includes:

[0025] Introducing the offset variable {O k As the hidden state variable of the HMM model, the position of the k-th base symbol in the channel input sequence is shifted when it passes through the channel output, and the shift in the output sequence is to the position (k+d), the shift amount is d, recorded as O k = d, assuming the input sequence is w = (w1, w2, ..., w t ), the output sequence is y = (y1, y2, ..., y q ), t and q represent the length of the input sequence and the output sequence respectively. Only when the inserted base symbol matches the deleted base symbol, t and q are equal. Therefore, the first base symbol w1 and the last base symbol w t The offset is O1 = 0 and O t =qt, the output base symbol corresponding to the offset of the kth base symbol is k+O k +O k-1 ;

[0026] When entering the base symbol w i During the transmission process, the probabilities of the four states of insertion, deletion, replacement and transmission are represented by p i 、p d 、p s and p t Indicates that the maximum insertion length is I, which defines the offset limit between adjacent position steps. If position k+1(t k+1 ) is offset by d, and the position k(t k ) is offset by b, then the probability transfer function P between two adjacent hidden states is db =P(O k+1 =d|O k = b) is

[0027]

[0028] where λI =1 / 4 I (1-(p i ) I ), P db Represents two situations, when t i+1 When the base symbol is deleted, at t i and t i+1 Insert b-d+1 base symbols between, and t i+1 When the base symbol of is not deleted, at t i and t i+1 Insert bd base symbols between;

[0029] In the HMM model, the output sequence y = (y1, y2, ..., y q ) is the observed sequence in the HMM model. During the sequence transmission, the probability matrix F(w k ,y k+d ), probability matrix F(w k ,y k+d ) applies to base symbol w k-1 and w k The offset is d, that is, O k-1 =O k =d, indicating base symbol w k-1 and w k There is no offset between the base symbols from t k Send and k+d The probability function of receiving is defined as γ(y k+d ), expressed as:

[0030]

[0031] The non-labeled base p(w k ) is initialized to p(A)=1 / 4, p(T)=1 / 4, p(C)=1 / 4, p(G)=1 / 4. The labeled bases are obtained by mapping the labeled positions. If w k =A, p(A)=1, p(T)=0, p(C)=0, p(G)=0;

[0032] Based on the HMM model, the posterior probability of the observation sequence is derived by calculating the forward-backward algorithm: the forward function α k (d) indicates O k is the probability of receiving d and the first k-1+d base symbols, the initial condition of the forward recursion is α1(0)=1, and the coefficient recursive relationship of the forward algorithm is calculated as:

[0033]

[0034] Similarly, the initial condition for the backward recursion is β t (qt) = 1, the coefficient recursive relationship of the backward algorithm is calculated as:

[0035]

[0036] Assume that the input base symbol is at t k The maximum allowed offset is d max , so the offset range is d∈{-d max ,d max}, through the recursion of the forward-backward function coefficients, the posterior probability of each input base is calculated as:

[0037]

[0038] Convert the posterior probability of the base to LLR: The base sequence is converted into a binary sequence by inverse mapping, and each base symbol corresponds to two binary bits, the first bit is defined as the low bit and the second bit is defined as the high bit. The LLR of the marked bit is set to a value with a large absolute value. If the marked bit is 1, the corresponding LLR value is set to 100. If the marked bit is 0, the corresponding LLR value is set to -100. For unmarked bits, the LLR of the low bit and the high bit are calculated as follows:

[0039]

[0040] The calculated output LLR sequence is L = (L1, L2, ..., L n ).

[0041] Further, in step S5, the ENMS algorithm includes: setting the information bit length to k, the code length to n, and the correction factor to σ; iter max Denotes the maximum number of iterations of LDPC decoding, D m represents the set of marking bits, L j represents the initial LLR value of the jth bit, represents the total LLR, is the information transmitted from check node i to variable node j, Defined as the information passed from variable node j to the i-th check node, C(j) represents the set of all check nodes connected to the j-th bit, V(i) represents the set of all variable nodes connected to the i-th check node, C(j) / i represents C(j) excluding the i-th check node, and V(i) / j represents V(i) excluding the j-th variable node;

[0042] S5.1 Initialization: Initialize the variable nodes according to the input LLR sequence:

[0043]

[0044] S5.2 Check node update: The check node does not pass information to the variable node corresponding to the marker bit. If the jth bit belongs to the marker bit, then Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as:

[0045]

[0046] S5.3 Variable node update: The information transmitted to the check node by the variable node corresponding to the mark bit is not updated. If the jth bit belongs to the mark bit, the variable node is not updated. Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as:

[0047]

[0048] S5.4 Total LLR calculation and hard decision: Calculate the total LLR. If the jth bit is a marker bit, the total LLR is not updated. If the jth bit is not a marker bit, the total LLR is updated as:

[0049]

[0050] Make a hard decision on the total LLR to get the decoded codeword. but like but

[0051] S5.5 Judgment iteration: Obtaining estimated codeword sequence through hard decision like Or if the maximum number of iterations is reached, decoding is stopped; otherwise, the process goes to step S5.2 to continue decoding.

[0052] Further, in step S5, iterating the synchronous decoding and ENMS algorithm by a joint iterative algorithm includes:

[0053] When the ENMS algorithm fails to completely correct the error, the base symbols decoded by the ENMS algorithm will be fed back to update the LLR of the synchronous decoding algorithm. In the iterative decoding, the iterative decoding ends until the error is successfully corrected or the maximum number of iterations is reached: Let f represent the feedback LLR sequence, where f1 is the low-order LLR, f2 is the high-order LLR, and when the base symbol is unmarked, the prior probabilities P(A), P(C), P(T), and P(G) of the base symbol are updated as follows:

[0054]

[0055] Then return to execute the synchronous decoding algorithm until the number of iterations is reached.

[0056] In addition, the present invention provides an effective error correction system based on multiple outputs of DNA data storage IDS channels, including:

[0057] Coding module: performs internal coding on the input information bit sequence to obtain an internal coding bit sequence, and then performs external coding on the internal coding bit sequence to generate an external coding bit sequence;

[0058] Mapping module: maps the external coding bit sequence to obtain the DNA base sequence, and divides the DNA base sequence into M short sequences;

[0059] Channel module: each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences;

[0060] Consensus module: Process M*N short output sequences through the SPM algorithm to obtain M short consensus sequences; splice the M short consensus sequences to obtain a consensus sequence;

[0061] Decoding module: the consensus sequence is first decoded by synchronous decoding, and then decoded by the ENMS algorithm to obtain information; first decoding by synchronous decoding, and then decoding by the ENMS algorithm includes: iterating the synchronous decoding and the ENMS algorithm through a joint iterative algorithm.

[0062] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0063] The present invention obtains multiple outputs through parallel IDS channels, uses a segmented progressive alignment algorithm to combine the homologous information of multiple outputs to obtain a consensus sequence, which can effectively avoid errors in the DNA synthesis and sequencing process, and then uses synchronous decoding and ENMS algorithms for decoding and joint iteration, further improving the error correction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of the effective error correction method based on multiple outputs of the DNA data storage IDS channel described in Example 1;

[0065] Figure 2 This is a block diagram of an effective error correction system based on multiple outputs of a DNA data storage IDS channel as described in Example 3;

[0066] Figure 3 This is a simulation result diagram of the bit error rate performance and the substitution error probability coefficient β when the sequencing read sequence output M is 3, 5, 7, and 10 described in Example 4;

[0067] Figure 4 This is a comparison chart of the bit error rate performance of the traditional solution and the proposed solution under the channel multiple sequence output described in Example 4; DETAILED DESCRIPTION

[0068] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0069] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0070] Embodiment 1:

[0071] This embodiment provides Figure 1 The effective error correction method based on multiple outputs of DNA data storage IDS channel includes:

[0072] S1: Perform internal encoding on the input information bit sequence to obtain an internal encoding bit sequence, and then perform external encoding on the internal encoding bit sequence to generate an external encoding bit sequence;

[0073] S2: Map the external coding bit sequence to obtain a DNA base sequence, and divide the DNA base sequence into M short sequences;

[0074] S3: Each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences;

[0075] S4: Process M*N short output sequences by SPM algorithm to obtain M short consensus sequences; splice the M short consensus sequences to obtain a consensus sequence;

[0076] S5: decoding the consensus sequence by synchronous decoding first, and then decoding it by ENMS algorithm to obtain information; decoding by synchronous decoding first, and then decoding by ENMS algorithm includes: iterating synchronous decoding and ENMS algorithm by joint iterative algorithm.

[0077] This embodiment obtains multiple outputs through parallel IDS channels, and uses a segmented progressive alignment algorithm combined with homologous information of multiple outputs to obtain a consensus sequence, which can effectively avoid errors in the DNA synthesis and sequencing process, and then uses synchronous decoding and ENMS algorithms for decoding and joint iteration, which further improves the error correction performance.

[0078] Embodiment 2:

[0079] This embodiment further discloses the following on the basis of the first embodiment:

[0080] Further, in step S1, performing internal encoding on the input information bit sequence to obtain the internal encoding bit sequence includes: using (D E ,D S ) embedded marker code is used to internally encode the input information bit sequence to obtain the internal encoding bit sequence c′=(c1,c2,…,ck ), D E represents the interval between two marker bit sequences, D S Indicates the length of the marker code.

[0081] Further, in step S1, performing external encoding on the internal coding bit sequence to generate an external coding bit sequence includes: using a QCLDPC code with a code type of [n, k] as an external code to perform external encoding to generate an external coding bit sequence u = (u1, u2, ..., u n ), where (nk) is the length of the check bits.

[0082] Further, in step S2, mapping the external coding bit sequence to obtain a DNA base sequence includes: mapping the external coding bit sequence to a DNA base sequence according to a mapping scheme of 00-A, 01-C, 10-T, and 11-G, and the DNA base sequence includes four base symbols of ATCG.

[0083] Further, in step S3, one of the M short sequences is passed through N parallel IDS channels to obtain N short output sequences, including:

[0084] One of the short sequences passes through an IDS channel to obtain an output sequence as follows:

[0085] Let w=(w1,w2,…,w n ) is a short sequence to be transmitted through the channel, which includes multiple base symbols w i ,y=(y1,y2,...,y n′ ) is the output sequence; base symbol w i When entering the channel, the bases of the received sequence may appear in four situations:

[0086] Base symbol w i With p i An insertion error occurs with probability , where a uniformly random symbol a∈{A,T,C,G} is inserted into base y of the output sequence i′ On, the base symbol w i is held in the input channel and corresponds to the received symbol y i′+1 ;

[0087] Base symbol w i With p d The probability of sending is deleted, then no symbol is received, and then the base symbol w is sent i+1 ;

[0088] Base symbol w i The probability of transmission is p t =1-p i -pd However, there are also p s The probability of substitution error occurs, therefore, the probability that the input base symbol is successfully transmitted and correctly received is p t (1-p s ), and then receives the base symbol y i′ =w i ;

[0089] The base symbol is p t p s The probability of substitution error is , then a uniform random symbol a is received, where a≠w i ;

[0090] When the channel receives the last symbol w n When , the transmission of a short sequence is completed and an output sequence is obtained.

[0091] Furthermore, in step S4, it includes:

[0092] Quickly align any two sequences from multiple output sequences, and use the Levenshtein distance to calculate the scoring matrix, where matches, mismatches, and gaps are 1 point, -1 point, and 0 point, respectively; thereby obtaining the total distance score of the alignment of any two sequences to obtain a distance matrix of size M×(M-1) / 2, where M is the number of output sequences; establish a guidance tree and calculate the weight of each branch, where each node in the guidance tree corresponds to the order of comparison of multiple sequences, and finally compare them pairwise in the order of sequence alignment to obtain the base symbol weight of each sequence, and deduce the base symbol based on the base weight to obtain the consensus sequence.

[0093] Furthermore, in step S5, the synchronous decoding algorithm includes:

[0094] Introducing the offset variable {O k As the hidden state variable of the HMM model, the position of the k-th base symbol in the channel input sequence is shifted when it passes through the channel output, and the shift in the output sequence is to the position (k+d), the shift amount is d, recorded as O k = d, assuming the input sequence is w = (w1, w2, …, w t ), the output sequence is y=(y1,y2,…,y q ), t and q represent the length of the input sequence and the output sequence respectively. Only when the inserted base symbol matches the deleted base symbol, t and q are equal. Therefore, the first base symbol w1 and the last base symbol w t The offset is O1 = 0 and O t =qt, the output base symbol corresponding to the offset of the kth base symbol is k+O k +O k-1 ;

[0095] When entering the base symbol w i During the transmission process, the probabilities of the four states of insertion, deletion, replacement and transmission are represented by p i 、p d 、p s and p t Indicates that the maximum insertion length is I, which defines the offset limit between adjacent position steps. If position k+1(t k+1 ) is offset by d, and the position k(t k ) is offset by b, then the probability transfer function P between two adjacent hidden states is db =P(O k+1 =d|O k = b) is

[0096]

[0097] where λ I =14 I (1-(p i ) I ), P db Represents two situations, when t i+1 When the base symbol is deleted, at t i and t i+1 Insert b-d+1 base symbols between, and t i+1 When the base symbol of is not deleted, at t i and t i+1 Insert bd base symbols between;

[0098] In the HMM model, the output sequence y = (y1, y2, ..., y q ) is the observed sequence in the HMM model. During the sequence transmission, the probability matrix F(w k ,y k+d ), probability matrix F(w k ,y k+d ) applies to base symbol w k-1 and w k The offset is d, that is, O k-1 =O k =d, indicating base symbol w k-1 and w k There is no offset between the base symbols from t k Send and k+d The probability function of receiving is defined as γ(y k+d ), expressed as:

[0099]

[0100] The non-labeled base p(w k ) is initialized to p(A)=1 / 4, p(T)=1 / 4, p(C)=1 / 4, p(G)=1 / 4. The labeled bases are obtained by mapping the labeled positions. If w k =A, p(A)=1, p(T)=0, p(C)=0, p(G)=0;

[0101] Based on the HMM model, the posterior probability of the observation sequence is derived by calculating the forward-backward algorithm: the forward function α k (d) indicates O k is the probability of receiving d and the first k-1+d base symbols, the initial condition of the forward recursion is α1(0)=1, and the coefficient recursive relationship of the forward algorithm is calculated as:

[0102]

[0103] Similarly, the initial condition for the backward recursion is β t (qt) = 1, the coefficient recursive relationship of the backward algorithm is calculated as:

[0104]

[0105] Assume that the input base symbol is at t k The maximum allowed offset is d max , so the offset range is d∈{-d max ,d max}, through the recursion of the forward-backward function coefficients, the posterior probability of each input base is calculated as:

[0106]

[0107] Convert the posterior probability of the base to LLR: The base sequence is converted into a binary sequence by inverse mapping, and each base symbol corresponds to two binary bits, the first bit is defined as the low bit and the second bit is defined as the high bit. The LLR of the marked bit is set to a value with a large absolute value. If the marked bit is 1, the corresponding LLR value is set to 100. If the marked bit is 0, the corresponding LLR value is set to -100. For unmarked bits, the LLR of the low bit and the high bit are calculated as follows:

[0108]

[0109] The calculated output LLR sequence is L = (L1, L2, ..., L n ).

[0110] Further, in step S5, the ENMS algorithm includes: setting the information bit length to k, the code length to n, and the correction factor to σ; iter max Denotes the maximum number of iterations of LDPC decoding, D m represents the set of marking bits, L j represents the initial LLR value of the jth bit, represents the total LLR, is the information transmitted from check node i to variable node j, Defined as the information passed from variable node j to the i-th check node, C(j) represents the set of all check nodes connected to the j-th bit, V(i) represents the set of all variable nodes connected to the i-th check node, C(j) / i represents C(j) excluding the i-th check node, and V(i) / j represents V(i) excluding the j-th variable node;

[0111] S5.1 Initialization: Initialize the variable nodes according to the input LLR sequence:

[0112]

[0113] S5.2 Check node update: The check node does not pass information to the variable node corresponding to the marker bit. If the jth bit belongs to the marker bit, then Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as:

[0114]

[0115] S5.3 Variable node update: The information transmitted to the check node by the variable node corresponding to the mark bit is not updated. If the jth bit belongs to the mark bit, the variable node is not updated. Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as:

[0116]

[0117] S5.4 Total LLR calculation and hard decision: Calculate the total LLR. If the jth bit is a marker bit, the total LLR is not updated. If the jth bit is not a marker bit, the total LLR is updated as:

[0118]

[0119] Make a hard decision on the total LLR to get the decoded codeword. but like but

[0120] S5.5 Judgment iteration: Obtaining estimated codeword sequence through hard decision like Or if the maximum number of iterations is reached, decoding is stopped; otherwise, the process goes to step S5.2 to continue decoding.

[0121] Further, in step S5, iterating the synchronous decoding and ENMS algorithm by a joint iterative algorithm includes:

[0122] When the ENMS algorithm fails to completely correct the error, the base symbols decoded by the ENMS algorithm will be fed back to update the LLR of the synchronous decoding algorithm. In the iterative decoding, the iterative decoding ends until the error is successfully corrected or the maximum number of iterations is reached: Let f represent the feedback LLR sequence, where f1 is the low-order LLR, f2 is the high-order LLR, and when the base symbol is unmarked, the prior probabilities P(A), P(C), P(T), and P(G) of the base symbol are updated as follows:

[0123]

[0124] Then return to execute the synchronous decoding algorithm until the number of iterations is reached.

[0125] This embodiment obtains multiple outputs through parallel IDS channels, and uses a segmented progressive alignment algorithm combined with homologous information of multiple outputs to obtain a consensus sequence, which can effectively avoid errors in the DNA synthesis and sequencing process, and then uses synchronous decoding and ENMS algorithms for decoding and joint iteration, which further improves the error correction performance.

[0126] Embodiment three:

[0127] This embodiment provides Figure 2 The effective error correction system based on multiple outputs of DNA data storage IDS channels includes:

[0128] Coding module: performs internal coding on the input information bit sequence to obtain an internal coding bit sequence, and then performs external coding on the internal coding bit sequence to generate an external coding bit sequence;

[0129] Mapping module: maps the external coding bit sequence to obtain the DNA base sequence, and divides the DNA base sequence into M short sequences;

[0130] Channel module: each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences;

[0131] Consensus module: Process M*N short output sequences through the SPM algorithm to obtain M short consensus sequences; splice the M short consensus sequences to obtain a consensus sequence;

[0132] Decoding module: the consensus sequence is first decoded by synchronous decoding, and then decoded by the ENMS algorithm to obtain information; first decoding by synchronous decoding, and then decoding by the ENMS algorithm includes: iterating the synchronous decoding and the ENMS algorithm through a joint iterative algorithm.

[0133] This embodiment obtains multiple outputs through parallel IDS channels, and uses a segmented progressive alignment algorithm combined with homologous information of multiple outputs to obtain a consensus sequence, which can effectively avoid errors in the DNA synthesis and sequencing process, and then uses synchronous decoding and ENMS algorithms for decoding and joint iteration, which further improves the error correction performance.

[0134] Embodiment 4:

[0135] This embodiment uses Microsoft Visual Studio as a simulation platform and uses the bit error rate as a performance evaluation standard. First, the performance of the segmented progressive alignment algorithm proposed in this embodiment and the traditional sequence alignment algorithm are compared in the sequencing process on different sequencing output sequences M. Second, to verify the reliability of the decoding algorithm, this embodiment simulates the impact of the decoding effect of the decoding scheme proposed in this embodiment and the traditional scheme under the same conditions.

[0136] The IDS channel model selected for simulation is based on nanopore sequencing. The QC LDPC code with a code rate of 0.9 (4096, 4544) is used as the outer code in the cascade coding scheme with a storage efficiency R of 0.72. E =4,D S =18) is the internal code, and the marker symbol is set to AGTC...AGTC. Set the maximum number of iterations of ENMS decoding iter max The length of the segmented sequence is set to 20, and the length of the segmented sequence is set to N as 110. The probability of insertion and deletion errors in the synthesis process is 0.0005~0.005, so the maximum offset of the IDS channel is selected to Set the probability of insertion and deletion p i =0,0005 and p d =0,0065.

[0137] like Figure 2The figure shows the simulation results of the bit error rate performance and the substitution error probability coefficient β when the sequencing read sequence output M is 3, 5, 7, and 10. First, when the error transfer probability is β = 0.02, the substitution error probability of base AG is 0.04, and the substitution error probability of base CT is 0.12. It is calculated that the bit error rate of M = 10 is 64.2% lower than that of M = 7, and the error correction performance is better as the number of comparisons M increases. In addition, as the output sequence M of the comparison increases, the decoding complexity increases exponentially. Balancing complexity and decoding performance is crucial for practical applications.

[0138] Under the condition that the sequencing read sequence output M is 5 and 7 and the segmented progressive alignment algorithm is used, the encoding and decoding scheme proposed in this embodiment and the traditional scheme using the watermark code as the inner code use the bit error rate as the performance evaluation standard, such as Figure 3 As shown in the figure, the insertion and deletion error probabilities remain unchanged, and the substitution error coefficient β is simulated at 0.015 to 0.05. Compared with the traditional scheme, the joint iterative decoding method proposed in this embodiment reduces the bit error rate by 21.72% to 99.75%. When β = 0.04, the joint iterative decoding algorithm has a significant impact on the bit error rate. m =1 and J m =3, compared with the traditional scheme, the bit error rate is reduced by about 54% and 98% respectively. It can be seen that the decoding performance of this embodiment is very good for the IDS channel with a large error probability. Therefore, this embodiment combines the soft information of the embedded marker code with the joint iterative decoding scheme based on the embedded marker code as the inner code, and iteratively updates to reduce the bit error rate and improve the decoding performance.

[0139] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An effective error correction method based on multiple outputs of DNA data storage IDS channels, characterized in that: include: S1: Perform internal encoding on the input information bit sequence to obtain an internal encoding bit sequence, and then perform external encoding on the internal encoding bit sequence to generate an external encoding bit sequence; S2: Map the external coding bit sequence to obtain a DNA base sequence, and divide the DNA base sequence into M short sequences; S3: Each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences; S4: Process M*N short output sequences by SPM algorithm to obtain M short consensus sequences; splice the M short consensus sequences to obtain a consensus sequence; S5: The consensus sequence is first decoded by synchronous decoding, and then decoded by the ENMS algorithm to obtain information; Decoding by synchronous decoding first and then decoding by ENMS algorithm includes: iterating synchronous decoding and ENMS algorithm by joint iterative algorithm.

2. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S1, performing internal encoding on the input information bit sequence to obtain the internal encoding bit sequence includes: using (D E ,D S ) embedded marker code is used to internally encode the input information bit sequence to obtain the internal encoding bit sequence c′=(c1,c2,…,c k ), D E represents the interval between two marker bit sequences, D S Indicates the length of the marker code.

3. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S1, performing external encoding on the internal coding bit sequence to generate an external coding bit sequence includes: using a QCLDPC code with a code type of [n, k] as an external code to perform external encoding to generate an external coding bit sequence u = (u1, u2, ..., u n ), where (nk) is the length of the check bits.

4. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S2, mapping the external coding bit sequence to obtain a DNA base sequence includes: mapping the external coding bit sequence to a DNA base sequence according to a mapping scheme of 00-A, 01-C, 10-T, and 11-G, and the DNA base sequence includes four base symbols of ATCG.

5. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S3, one of the M short sequences is passed through N parallel IDS channels to obtain N short output sequences, including: One of the short sequences passes through an IDS channel to obtain an output sequence as follows: Let w=(w1,w2,…,w n ) is a short sequence to be transmitted through the channel, which includes multiple base symbols w i ,y=(y1,y2,...,y n′ ) is the output sequence; base symbol w i When entering the channel, the bases of the received sequence may appear in four situations: Base symbol w i With p i An insertion error occurs with probability , where a uniformly random symbol a∈{A,T,C,G} is inserted into base y of the output sequence i′ On, the base symbol w i is held in the input channel and corresponds to the received symbol y i′+1 ; Base symbol w i With p d The probability of sending is deleted, then no symbol is received, and then the base symbol w is sent i+1 ; Base symbol w i The probability of transmission is p t =1-p i -p d However, there are also p s The probability of substitution error occurs, therefore, the probability of successful transmission and correct reception of the input base symbol is p t (1-p s ), and then receives the base symbol y i′ =w i ; The base symbol is p t p s The probability of substitution error is , then a uniform random symbol a is received, where a≠w i ; When the channel receives the last symbol w n When , the transmission of a short sequence is completed and an output sequence is obtained.

6. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S4, it includes: Quickly align any two sequences from multiple output sequences, and use Levenshtein distance to calculate the scoring matrix, where matches, mismatches, and gaps are 1 point, -1 point, and 0 point, respectively; thereby obtaining the total distance score of the alignment of any two sequences to obtain a distance matrix of size M×(M-1) / 2, where M is the number of output sequences; establish a guidance tree and calculate the weight of each branch, where each node in the guidance tree corresponds to the order of comparison of multiple sequences, and finally compare them pairwise in the order of sequence alignment to obtain the base symbol weight of each sequence, and deduce the base symbol based on the base weight to obtain the consensus sequence.

7. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S5, the synchronous decoding algorithm includes: Introducing the offset variable {O k As the hidden state variable of the HMM model, the position of the k-th base symbol in the channel input sequence is shifted when it passes through the channel output, and the shift in the output sequence is to the position (k+d), the shift amount is d, recorded as O k = d, assuming the input sequence is w = (w1, w2, ..., w t ), the output sequence is y=(y1,y2,...,y q ), t and q represent the length of the input sequence and the output sequence respectively. Only when the inserted base symbol matches the deleted base symbol, t and q are equal. Therefore, the first base symbol w1 and the last base symbol w r The offset is O1 = 0 and O t =qt, the output base symbol corresponding to the offset of the kth base symbol is k+O k +O k-1 ; When entering the base symbol w i During the transmission process, the probabilities of the four states of insertion, deletion, replacement and transmission are represented by p i 、p d 、p s and p t Indicates that the maximum insertion length is I, which defines the offset limit between adjacent position steps. If position k+1(t k+1 ) is offset by d, and the position k(t k ) is offset by b, then the probability transfer function P between two adjacent hidden states is db =P(O k+1 =d|O k = b) is where λ I =1 / 4 I (1-(p i ) I ), P db Represents two situations, when t i+1 When the base symbol is deleted, at t i and t i+1 Insert b-d+1 base symbols between, and t i+1 When the base symbol of is not deleted, at t i and t i+1 Insert bd base symbols between; In the HMM model, the output sequence y=(y1,y2,...,y q ) is the observed sequence in the HMM model. During the sequence transmission, the probability matrix F(w k ,y k+d ), probability matrix F(w k ,y k+d ) applies to base symbol w k-1 and w k The offset is d, that is, O k-1 =O k =d, indicating base symbol w k-1 and w k There is no offset between the base symbols from t k Send and k+d The probability function of receiving is defined as γ(y k+d ), expressed as: The non-labeled base p(w k ) is initialized to p(A)=1 / 4, p(T)=1 / 4, p(C)=1 / 4, p(G)=1 / 4. The labeled bases are obtained by mapping the labeled positions. If w k =A, p(A)=1, p(T)=0, p(C)=0, p(G)=0; Based on the HMM model, the posterior probability of the observation sequence is derived by calculating the forward-backward algorithm: the forward function α k (d) indicates O k is the probability of receiving d and the first k-1+d base symbols, the initial condition of the forward recursion is α1(0)=1, and the coefficient recursive relationship of the forward algorithm is calculated as: Similarly, the initial condition for the backward recursion is β t (qt) = 1, the coefficient recursive relationship of the backward algorithm is calculated as: Assume that the input base symbol is at t k The maximum allowed offset is d max , so the offset range is d∈{-d max ,d max }, through the recursion of the forward-backward function coefficients, the posterior probability of each input base is calculated as: Convert the posterior probability of the base to LLR: The base sequence is converted into a binary sequence by inverse mapping, and each base symbol corresponds to two binary bits, the first bit is defined as the low bit and the second bit is defined as the high bit. The LLR of the marked bit is set to a value with a large absolute value. If the marked bit is 1, the corresponding LLR value is set to 100. If the marked bit is 0, the corresponding LLR value is set to -100. For unmarked bits, the LLR of the low bit and the high bit are calculated as follows: The calculated output LLR sequence is L = (L1, L2, ..., L n ).

8. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S5, the ENMS algorithm includes: setting the information bit length to k, the code length to n, and the correction factor to σ; iter max Denotes the maximum number of iterations of LDPC decoding, D m represents the set of marking bits, L j represents the initial LLR value of the jth bit, represents the total LLR, is the information transmitted from check node i to variable node j, Defined as the information passed from variable node j to the i-th check node, C(j) represents the set of all check nodes connected to the j-th bit, V(i) represents the set of all variable nodes connected to the i-th check node, C(j) / i represents C(j) excluding the i-th check node, and V(i) / j represents V(i) excluding the j-th variable node; S5.1 Initialization: Initialize the variable nodes according to the input LLR sequence: S5.2 Check node update: The check node does not pass information to the variable node corresponding to the marker bit. If the jth bit belongs to the marker bit, then Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as: S5.3 Variable node update: The information transmitted to the check node by the variable node corresponding to the mark bit is not updated. If the jth bit belongs to the mark bit, the variable node is not updated. Otherwise, the jth bit does not belong to the mark bit, and the check node is updated as: S5.4 Total LLR calculation and hard decision: Calculate the total LLR. If the jth bit is a marker bit, the total LLR is not updated. If the jth bit is not a marker bit, the total LLR is updated as: Make a hard decision on the total LLR to get the decoded codeword. but like but S5.5 Judgment iteration: Obtaining estimated codeword sequence through hard decision like Or if the maximum number of iterations is reached, decoding is stopped; otherwise, the process goes to step S5.2 to continue decoding.

9. The effective error correction method based on multiple outputs of DNA data storage IDS channels according to claim 1 is characterized in that: In step S5, iterating the synchronous decoding and ENMS algorithm by the joint iterative algorithm includes: When the ENMS algorithm fails to completely correct the error, the base symbols decoded by the ENMS algorithm will be fed back to update the LLR of the synchronous decoding algorithm. In the iterative decoding, the iterative decoding ends until the error is successfully corrected or the maximum number of iterations is reached: Let f represent the feedback LLR sequence, where f1 is the low-order LLR, f2 is the high-order LLR, and when the base symbol is unmarked, the prior probabilities P(A), P(C), P(T), and P(G) of the base symbol are updated as follows: Then return to execute the synchronous decoding algorithm until the number of iterations is reached.

10. An effective error correction system based on multiple outputs of DNA data storage IDS channels, characterized in that: include: Coding module: performs internal coding on the input information bit sequence to obtain an internal coding bit sequence, and then performs external coding on the internal coding bit sequence to generate an external coding bit sequence; Mapping module: maps the external coding bit sequence to obtain the DNA base sequence, and divides the DNA base sequence into M short sequences; Channel module: each of the M short sequences is passed through N parallel IDS channels to obtain M*N short output sequences; Consensus module: Process M*N short output sequences through the SPM algorithm to obtain M short consensus sequences; The M short consensus sequences are spliced ​​together to obtain a consensus sequence; Decoding module: The consensus sequence is first decoded by synchronous decoding, and then decoded by the ENMS algorithm to obtain information; Decoding is first performed by synchronous decoding and then by the ENMS algorithm, including: The synchronous decoding and ENMS algorithms are iterated through a joint iterative algorithm.

Images