CoProFibA, a lossless final compression algorithm, applied to DNA storage.

Abstract

Creation and method of Jérôme Chapoul's system, developed with personal funds, June 29, 2024

Description

Title of the invention: CoProFibA, a lossless final compression algorithm, applied to DNA storage. Context:

[0001] Disk storage is no longer viable in the long term. If we were to store all our current data on Blu-ray discs, we would have twenty-three stacks of discs stretching to the Moon. Furthermore, all hard drives would need to be replaced every 5 years to ensure data preservation. DNA storage allows us to store all the planet's data in a space roughly the size of a shoebox and is viable for over 10,000 years (preservation of mammoth DNA in ice).

[0002] However, storing this data is very expensive, as it must be sequenced for integration into capsules before cryopreservation. Currently, Biomemory charges €1,000 per kilobyte of data to be stored, which represents approximately €1 billion for a 1-gigabyte film...

[0003] To minimize these costs, different compression algorithms are used, Huffman, Lz78...

[0004] Nevertheless, the possibilities of compression remain limited, and the challenge of reducing storage costs rests essentially on the methods of compressing this data.

[0005] This data is stored in capsules, after having been sequenced into oligo chains of sizes defined in a text file using a program. My proposal:

[0006] The CoProFibA for Proportional Compression by Fibonacci DNA.

[0007] This algorithm, coded in Python for ease of use of floating-point numbers, can be used in various other languages ​​that support the creation of clean data, for example C++.

[0008] Let us take a DNA-ready sequencing file containing a number x of oligonucleotides of length 2500, where each iteration is a value of the Fibonacci sequence, i.e. 2500* 4= 10000 Fibonacci iterations in quaternary.

[0009] The maximum number in the sequence is therefore =

[0010] 33644764876431783266621612005107543310302148460680063906564769974680 08144216666236815559551363373402558206533268083615937373479048386526 82630408924630564318873545443695598274916066020998841839338646527313 00088830269235673613135117579297437854413752130520504347701602264758 31890652789085515436615958298727968298751063120057542878345321551510 38708182989697916131278562650331954871402142875326981879620469360978 79900350962302291026368131493195275630227837628441540360584402572114 334961180

[0011] suite :0230912082870460889239623288354615057765832712525460935911282 03925285393434620904245248929403901706233888991085841065183173360437 47073790855263176432573399371287193758774689747992630583706574283016 16374089691784263786242128352581128205163702980893320999057079200643 67426202389783111470054074998459250360633560933883831923386783056136 43535189213327973290813373264265263398976392272340788292817795358057 09936910491754708089318410561463223382174656373212482263830921032977 01648054726243842374862411453093812206564914032751086643394517512161 52654536133311131404243685480510676584349352383695965342807176877532 83482343455573667197313927462736291082106792807847180353291311767789 24659089938635459327894523777674406192240337638674004021330343297496 90202832814593341882681768389307200363479562311710310129195316979460 76327375892535307725523759437884345040677155557790564504430166401194 6258097221672975861502696844314695203461493229110597067624326851599283470989128470674086200858713501626031207190317208609408129832158107 72820763531866246112782455372085323653057759564300725177443150515396 00905168603220349163222640885248852433158051534849622434848299380905 07048348244932745373262456775587908918719080366205800959474315005240 25327097469953187707243768259074199396322659841474981936092852239450 39707165443156421328157688908058783183404917434556270520223564846495 19611246026831397097506938264870661326450766507461151267752274862159 86425307112984411826226610571635150692600298617049454250474913781151 54139941550671256271197133252763631939606902895650288268608362241082 050562430701794976171121233066073310059947366875

[0012] which is much too large!!!, as for pow(4,2500) in binary, the value causes an overflow in python.

[0013] The implementation of the Fibonacci sequence in our oligonucleotide as below is essential to obtain the expected result of the algorithm, in particular by its property which makes each linear sum in the oligonucleotide unique and cannot be found in another sum.

[0014] T=5 T=34 T = 233

[0015] G=3 G=21 G=144

[0016] C=2 C=13 C=89

[0017] A=1 A=8 A=55

[0018] The algorithm is simple, we just need to determine the maximum Fibonacci value of the last nucleotide of the oligonucleotide.

[0019] For example, if I have an oligonucleotide of 300 nucleotides, then I would find the value T of the last nucleotide by doing T = pow(4,(300*4)), remember that we are in quaternary code (A,C,G,T possible for the same nucleotide), hence the multiplication of the length of the oligonucleotide * 4.

[0020] Once this value is found, we take the real value of the oligonucleotide of 300 nucleotide, and add up each Fibonacci number for each nucleotide, i.e.: for a set of length n, containing different digits of a Fibonacci sequence of length d, sum all the elements of the set.

[0021] SE=Eni=l.di.

[0022] Then we only have to apply the proportional ratio (a+b) / a = x, where a = (maximum value of the last nucleotide, b=actual value of the oligonucleotide and x = a proportional coefficient.

[0023] We then obtain a decimal number that we can store in a standard data type of 7 to 18 decimal places to fit in a float, a long or a double as in c++ i.e. 4, 8 or 10 bytes

[0024] Thus, with this method, we can easily fit an oligonucleotide of 2500 nucleotides onto 40 nucleotides, i.e. a compression of 100 - ((40 / 2500) * 100 = 98.4%, i.e. a maximum compression factor in the best cases of 98.4%.

[0025] This algorithm makes it possible to compress a number tending towards infinity of nucleotides (oligonucleotide), by making it fit on a simple decimal number.

[0026] If necessary, and if the floating-point numbers are too large for the usual standard data, for example in c++ long double = 10 bytes, there are libraries that allow the creation of larger floating-point numbers, for example the boost::multiprecision library.

[0027] Quaternary decimal transposition

[0028] Recall that to go from quaternary to binary, it is sufficient to do as currently practiced. Example with 2 nucleotides

[0029] aa=1, ac=2, ag=3, at=4, ca=5, cc=6 cg=7... etcetera, etcetera.

[0030] Let us take an oligonucleotide of 2500 nucleotides, i.e. a max nucleotide value of pow(4,(2500*4)) = a.

[0031] Let us say that the actual oligonucleotide will have a totally random value of (pow(4,9981) + 6876833132386846849874531) = b. (value chosen randomly).

[0032] The sum of (a+b) / a = 1.0001069633103603 - Python code 3.11 below.

[0033] here we see that it has 16 decimal places and therefore fits on 40 bytes in long double type, i.e. 10 bytes in binary and 40 nucleotides in quaternary.

[0034] All that remains for us is to reconstruct the long double data type as it is implemented in C++ (sign + mantissa + exponent), i.e.:

[0035] 1 bit for the + or - sign

[0036] 64 bits for the mantissa, i.e. l8#446#744#073#709#551#616 max, i.e 10001069633103603

[0037] 15 bits for the exponent, i.e. 32768, therefore 101 * mantissa = 1.0001069633103603

[0038] therefore 80 bits, or 10 bytes. Our number therefore fits in a long double.

[0039] In quaternary nucleotide.

[0040] 1 nucleotide for the sign, C = +, T = -

[0041] 32 nucleotides for the mantissa: 8#446#744#073#709#551#616 max, 10001069633103603

[0042] 8 nucleotides for the exponent, i.e. 65#536 101 * mantissa = 1.0001069633103603

[0043] However, to compress further, we will change the exponent from 8 nucleotides to 3, or the quaternary integer value will represent the decimal point shift + 1 with respect to the starting integer of the mantissa.

[0044] Example for a mantissa value of 33333... if I have AAG = 3, the shift of the decimal digit will be 33.333... i.e. the shift of the decimal point of 2 + 1.

[0045]

[0046] Since 1+32+3 = 36 nucleotides are needed to represent a number, therefore 100 - ((36 / 2500) * 100 = 98.56% optimum compression for 18 decimal places

[0047] This compression value can indeed vary, as I said above, if there are more than 18 decimal places, the value of the mantissa must be increased to cover all the whole values ​​of the decimal places. Scientific equation, Python algorithm#:

[0048] def fibonacci(n):

[0049] ifn<=0:

[0050] return 0

[0051] elifn==l:

[0052] return 1

[0053] else:

[0054] a, b = 0, 1

[0055] for _ in range(2, n + 1):

[0056] a, b = b, a + b

[0057] return b

[0058] fib_2500 = (fibonacci(10000) + (fibonacci(9981) + 687683313238684684 9874531)) / fibonacci( 10000)

[0059] print(f"The proportionality coefficient compression is: {fib_2500}")

[0060] 1.0001069633103603 Scientific equation:

[0061] Since x is a base-4 representation with a sign, a mantissa and an exponent, we can reformulate the equation a+b=x as follows:

[0062] (a+b) / a=sm-4e

[0063] s is the sign of x, represented by a quaternary digit (G or T).

[0064] m is the base 4 mantissa, composed of nx quaternary digits.

[0065] e is the base 4 exponent, composed of 3 quaternary digits.

[0066] To express x as a quaternary number with a sign, a mantissa and an exponent:

[0067] x=(-l)s-Ei=lnxmi-4-i-4Ej=13ej-4-j

[0068] where mi are the quaternary digits of the mantissa and ej are the quaternary digits of the exponent.

[0069] Using the Fibonacci numbers a and b, you can solve for x in terms of these values ​​and the structure defined for x.

[0070] For example, if a and b are calculated as Fibonacci numbers, then x would be represented according to the specified quaternary structure. Therefore: (a+b) / a = x=(-l)s£i=lnxmi-4-i-4£j=13ej-4-j

[0071] (-4)* , m , 4 -v . & VJ « Find the oligonucleotide:

[0072] to find the real oligonucleotide b, it suffices to reverse the equation (a+b) / a =x, i.e. b=a(xl).

[0073] So in reality, in order to be able to decompress our decimal value, we will have to store in addition to our proportional coefficient, the number of nucleotides contained in our oligonucleotide.

[0074] That is, for 2500, 46 = 4096, an oligonucleotide of 6 additional nucleotides, the implementation of which is based on the quaternary / binary conversion mentioned above.

[0075] So a final compression of, 1+32+3 +6 = 42 nucleotides, therefore 100 -((42 / 2500) * 100 = 98.32% maximum compression.

[0076] That's it, everything has been said.

[0077] Conclusion:

[0078] The use of this algorithm has the possibility of reducing the size of certain lines by 98.32%, and of reducing costs from 1000 to 16.8 euros per MB of data...

[0079] A future foundation for an essential DNA storage solution, the ultimate compression! Can be used after standard compression algorithms, Huffman, LZ7, LZ8... Or on its own.

[0080] A lengthy process, but compression reducing the price of the nucleotide from 1000 euros per MB to 16.8 euros.

Claims

[Claim 1] Demands I claim to be the sole creator of the system, and to have carried out this research outside of a professional framework.