A large model-based text compression encryption method and system

Abstract

The application relates to a large model-based text compression encryption method and system. The method realizes safe and efficient dynamic encryption through a hybrid key architecture, including: generating an elliptic curve asymmetric key pair and encrypting a dynamically derived symmetric session key; processing the original text through the context input to the large model to generate a compressed text; mapping the compressed text to a binary bit stream through an arithmetic coding-based lossless compression model; generating a session key bit stream through a symmetric encryption (AES-GCM) algorithm with authentication and encryption functions to encrypt the session key bit stream to generate ciphertext; and splicing the encryption key and the ciphertext through zero bit padding encoding to form a structured bit stream. The scheme uses the context semantic modeling capability of the large language model to maintain the grammar rules, character types and semantic expressions of the decompressed text while ensuring the compression capability, and improves the encryption strength, and can be widely applied to high-speed communication (such as 6G) and other scenarios that need to consider security and high compression rate.

Description

Technical Field

[0001] This invention relates to the fields of data security and cryptography, specifically to a text compression and encryption system based on a large model. Background Technology

[0002] With the acceleration of digital transformation, data encryption technology faces the dual challenges of key management complexity and business practicality. Traditional solutions, while maintaining data format through lossless text compression algorithms, neglect the protection of the compressed sequence. Furthermore, mainstream international text compression technologies are primarily designed for encoding algorithms, exhibiting significant shortcomings when handling large-scale text requiring high security: First, encoding-based text compression struggles to further improve compression rates for large-scale text; second, traditional algorithms cannot actively switch compression modes to adapt to dynamic compression needs involving lossy and lossless compression; and third, because simultaneous compression and encryption are impossible, additional encryption is required to meet the privacy and security requirements of special texts such as legal documents and medical services, leading to low efficiency and unsuitability for real-world business scenarios. To achieve simultaneous compression and encryption, we propose a large-model-based compression and encryption technology that introduces key encapsulation using both symmetric and asymmetric encryption, combining the advantages of both technologies to improve data security while ensuring efficient compression.

[0003] While existing text compression methods based on traditional encoding can maintain data usability while compressing text, they require customized development for each business scenario, resulting in high costs for cross-domain adaptation. In recent years, Large Language Models (LLM) have made breakthroughs in text generation, but a technological gap remains when combining them with text compression: on the one hand, traditional text compression algorithms generate low compression ratios and do not consider leveraging the semantic understanding capabilities of LLM to improve compression ratios; on the other hand, sequences generated by directly encoding and compressing text lack cryptographic security guarantees, requiring additional encryption for sensitive data applications, thus reducing efficiency. This technological disconnect makes it difficult for text compression technology to achieve a balance between security and business efficiency, hindering its large-scale application in high-value scenarios such as high-speed secure communication. Summary of the Invention

[0004] To address the problems of existing technologies, this invention provides a text compression and encryption method and system based on a large model.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Firstly, a text compression and encryption method based on a large model, comprising the following steps: S1: Preprocess the text based on context requirements to generate the text to be compressed; S2: Map the text to a binary bitstream by compressing a large language model; S3: Encrypt the bitstream using a dynamically generated session key to generate ciphertext; S4: Generate an asymmetric encryption public-private key pair, and use the public key to encrypt the dynamically generated session key to obtain the encrypted session key; S5: Concatenate the ciphertext with the encrypted session key in a structured manner to form a ciphertext bit stream to be transmitted; S6: Based on the context semantic generation capability and encryption reversibility of the large language model, the compressed encrypted bit stream, after decompression, is consistent with the compression requirements of the large model context representation in S1.

[0006] This application achieves a breakthrough improvement in text compression technology by integrating a hybrid key architecture with the dynamic compression pattern generation capability of a large language model. Its core function is to effectively solve the problems of limited compression patterns and low encryption efficiency of compressed text caused by algorithmic constraints in traditional compression processes, specifically manifested in the following technical effects: Synergistic optimization of security and efficiency: Asymmetric encryption is used to protect dynamically generated symmetric keys, which significantly reduces the overall computational complexity of the system while ensuring the security of key transmission, and achieves better resource utilization compared to traditional single encryption methods. Dynamic adjustment of text compression mode: Using token probability distribution to map the bit stream obtained by arithmetic coding to the token sequence and the contextual semantic control of the large language model, the lossless compressed text maintains a high degree of consistency with the original plaintext in terms of character types and arrangement rules such as Chinese characters, numbers, and punctuation marks, avoiding business logic interruptions caused by format deviations; lossy compressed text efficiently preserves semantics while significantly improving compression capabilities, making it suitable for application scenarios that emphasize semantics and have limited hardware resources. Multi-scenario adaptive compatibility: Based on pseudo-random delimiter and dynamic adjustment technology of compression mode, it can flexibly adapt to the compression and encryption requirements of text integrity and non-integrity data in scenarios such as financial transaction messages, medical records, and high-speed communication, and shows unique advantages, especially in multi-mode compressed text processing. Dynamic encryption enhancement: Leveraging the probability distribution learning characteristics of large language models, this approach overcomes the dependence of traditional encoded text compression on algorithm performance. It generates optimal text bitstreams through real-time semantic modeling, improving randomness while ensuring data service availability. Simultaneously, a dynamic session key generator based on symmetric encryption ensures that each session is protected by a new key, guaranteeing the security of information transmission.

[0007] This solution, through the collaborative innovation of key management mechanism and large model generation component, provides a technical implementation path that balances encryption strength and compression efficiency for application scenarios such as high-speed communication and data sharing that require both high security and diverse data formats.

[0008] In one specific embodiment of the first aspect, in step S1: Based on contextual requirements, the large model can identify the user's need for lossless / lossy compression and process the original text accordingly. For lossless compression, the large model directly restates the original text to generate the text to be compressed; for lossy compression, the large model extracts semantics from the original text, preserving its semantic logic, and generates new text to be compressed.

[0009] In one specific embodiment of the first aspect, in step S4: The asymmetric encryption uses an elliptic curve cryptography system, and the symmetric encryption uses a symmetric encryption algorithm with authentication and encryption functions. The dynamic symmetric key is generated by negotiating the timestamp and the text to be compressed using a key derivation function.

[0010] In one specific embodiment of the first aspect, in step S5: Zero-bit stuffing encoding is used to perform format compatibility processing on the concatenated ciphertext, and pseudo-random delimiters are inserted to ensure transmission integrity.

[0011] In one specific embodiment of the first aspect, in steps S2 and S6: During compression, the target of lossless / lossless compression of the text can be determined by inputting the context content into the large language model according to the actual situation. The text to be compressed is mapped to a probability interval and compressed into a bit stream through arithmetic coding. By adjusting the input context to the large language model to be the same as that used during compression, it is ensured that the decompressed text is completely consistent with the original text in terms of Chinese characters, numbers, or mixed formats, just like the compression target during compression.

[0012] Secondly, a text compression and encryption system based on a large model includes: The large model processing module processes the original text into text to be compressed based on the context. What makes lossy compression special is that it uses the large model's understanding of semantics to preserve the semantic logic of the original text and generate a shorter text to be compressed. The key management module is used to generate asymmetric key pairs and dynamic symmetric keys, and to complete key encapsulation. Large Language Model Compression Processing Module: Uses a lossless compression large language model based on arithmetic coding to map the text to be compressed into a binary sequence through the token probability distribution inside the large model; The encryption module is used to perform bitstream encryption and ciphertext concatenation; The decryption module is used to parse the encrypted symmetric key using the private key and restore the original compressed bit sequence. The large language model decompression module is used to generate target text (consistent with the text to be compressed) from the binary bit sequence through the probability interval generated by the large model.

[0013] In one specific implementation of the second aspect, the large model processing module further includes: The context includes the compression mode. When selecting a lossy compression mode, constraints can be imposed on the content lost in the text. By leveraging the understanding capabilities of large models, text can be processed into text that meets the requirements for compression according to the context. In one specific implementation of the second aspect, the key management module further includes: Elliptic curve-based asymmetric encryption algorithm key generation unit is used to generate public and private key pairs that conform to the NISTSP800-186 standard. The key derivation unit generates a dynamic session key with authentication encryption (AES-GCM) based on the timestamp and the text to be compressed.

[0014] In one specific implementation of the second aspect, the large language model compression module achieves text compression in the following manner: Extract the token probability distribution output by the large language model; The encoded bitstream that uniquely matches the token sequence of the text to be compressed is selected based on the probability distribution of the tokens in the arithmetic encoding. In one specific implementation of the second aspect, the system supports encryption processing of binary bit sequences, specifically including: Dynamically generate the keys required for symmetric encryption algorithms; The bit stream obtained through compression is encrypted using a symmetric encryption algorithm; The key required for symmetric encryption algorithms (session key) is encrypted using the public key for asymmetric encryption; By concatenating ciphertext, the key required for the encrypted symmetric encryption algorithm (session key) is concatenated with the encrypted compressed bit stream to form a whole that is not format-compatible.

[0015] The beneficial effects of this invention are as follows: 1. By innovatively integrating a hybrid key architecture with large-scale text compression technology, a breakthrough in security and practicality is achieved across multiple dimensions. On one hand, a collaborative mechanism of asymmetric and symmetric encryption is employed to significantly improve key management efficiency while ensuring security, effectively addressing bottlenecks such as excessive key storage volume and lagging dynamic updates in real-world scenarios. On the other hand, a text compression algorithm designed for large-scale dynamic text compression modes in existing scenarios, through a synchronous encryption system and a context-aware dynamic compression mode, achieves for the first time triple guarantees in compression speed, semantic logic, and data security for lossless / lossless compressed text, breaking through the requirements of existing technologies for compression efficiency and data security, and overcoming the limitations of a single data compression mode. 2. A cross-domain adaptable dynamic encryption system has been further constructed. Its hierarchical key architecture and dynamic policy adjustment mechanism can flexibly meet the special compression and encryption requirements of data integrity and security levels in scenarios such as finance, healthcare, and government. Through deep coupling of large language models and cryptography, it autonomously maintains text style and business rules during lossy semantic compression, significantly reducing the cost of manual verification and providing a unified solution for the full lifecycle protection of lossy and lossless compressed data. This technical system effectively balances the contradiction between high-strength encryption and business availability, providing an innovative technical path for privacy protection needs in digital transformation. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the key functions of the present invention.

[0017] Figure 2 This is a schematic diagram of the basic framework of the present invention.

[0018] Figure 3 This is a schematic diagram of the key encapsulation of the present invention.

[0019] Figure 4 This is a schematic diagram of natural language text compression according to the present invention.

[0020] Figure 5 This is a schematic diagram of the compression mode selection process of the present invention. Figure 6 This is a schematic diagram of the lossless text compression process based on arithmetic coding and a large model according to the present invention.

[0021] Figure 7 This is a schematic diagram of the text probability sampling technique based on a large model according to the present invention.

[0022] Figure 8 This is a schematic diagram illustrating the operation of the text encoding technology based on arithmetic encoding of the present invention.

[0023] Figure 9 This is a schematic diagram of the ciphertext integration technology based on special sequences of the present invention.

[0024] Figure 10 This is a schematic diagram illustrating the operation of the key-encapsulated compressed bit sequence encryption technology of this invention.

[0025] Figure 11 This is a schematic diagram of the technical route of the present invention. Detailed Implementation

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figures 1 to 11 This paper presents a text compression and encryption method and system based on a large model.

[0028] like Figure 1 As shown, in the current field of traditional text compression, improving compression efficiency, constructing flexible text compression mechanisms, and achieving rapid compression results are key research directions. This invention aims to construct a text compression and encryption method and system based on a large language model. This method features an efficient key management mechanism to optimize speed and security during the encryption process. Simultaneously, a ciphertext integration mechanism is applied to facilitate the transmission and decryption of ciphertext. Furthermore, this method constructs a novel text compression mechanism based on existing large language models. Finally, addressing the limitation of traditional text compression algorithms in changing compression modes, a multi-mode compression mechanism is constructed based on the semantic understanding of the large model to meet the text compression needs of different scenarios.

[0029] like Figure 2 As shown, the large-scale model-based text compression and encryption system includes a flexible key management mechanism built on key encapsulation; a ciphertext integration mechanism based on zero-bit padding; a flexible text compression mechanism to address existing text compression problems; and a system method that combines compression and encryption to avoid the predicament of compressing before encrypting. Based on these innovations, the large-scale model-based text compression and encryption system achieves flexible and efficient data encryption while compressing.

[0030] 1. Build a convenient and efficient key management mechanism Traditional text compression and encryption methods employ compression followed by encryption, which is slow and complex. Furthermore, the use of symmetric encryption algorithms leads to complex key distribution, limiting the capacity of secure applications. Additionally, the characteristics of symmetric keys necessitate frequent key changes, requiring significant resources for secure key management.

[0031] To address the key management bottlenecks of traditional text compression and encryption, this study proposes a key encapsulation strategy that integrates asymmetric and symmetric encryption, building upon traditional symmetric-key encryption. Traditional methods typically use symmetric-key encryption for compressed text encryption. Building upon this, we utilize the public key of asymmetric encryption to perform secondary encryption on the symmetric encryption key. The encrypted compressed bit sequence and the encrypted session key are then concatenated into a single ciphertext for transmission. In this asymmetric-key system, the receiver of the compressed ciphertext can decrypt the key using their private key, and then use the decrypted symmetric key to decrypt the ciphertext, thus obtaining the actual bitstream required for encrypted transmission.

[0032] like Figure 3 As shown, this key encapsulation strategy retains the high efficiency of symmetric key encryption while combining the convenient and secure key management features of asymmetric encryption technology. This allows the entire encryption system to have higher carrying capacity and a more secure architecture. The solution adopts a hybrid design paradigm of "asymmetric encryption to manage key distribution and symmetric encryption to ensure business performance", which effectively solves the key explosion problem during large-scale deployment.

[0033] 2. Results of compression algorithms for natural text processing The effect of text compression is as follows Figure 4 As shown, through compression algorithms, natural language text will be encoded into bit sequences. The compression rate of traditional algorithms is limited by the information entropy theory proposed by Shannon. At the same time, the binary sequence can be restored to natural language text through decompression algorithms.

[0034] 3. Develop flexible text compression technology like Figure 5 As shown, users can flexibly choose the compression mode based on the context. In lossless compression, the text can bypass the semantic extraction of the large model. Lossy compression, however, requires us to integrate the compression mode and requirements into the context, leveraging the text understanding capabilities of the large model to compress the text into the shortest possible length that clearly expresses the semantics. The text generated in this process is the text to be compressed into the large model for lossless compression.

[0035] 4. Constructing text compression based on a large language model Traditional lossy text compression refers to a compression technique based on encoding algorithms, which cannot guarantee that the compressed text will be semantically and logically identical to the original text. However, traditional lossy text compression algorithms encounter inefficiencies when processing large-scale, high-security texts, particularly due to the sheer size of these texts, leading to slow encryption speeds. To address this issue, we propose a text compression method based on a large language model.

[0036] Regarding the choice of encoding methods, Huffman coding, due to its reliance on a discrete dictionary, has its encoding length limited by a pre-constructed tree structure, making it unable to fully utilize the continuous probability distribution of LLM. Block coding, by dividing the vocabulary into 2^k bins, with each bin storing only k bits of information, results in a low steganography embedding rate. In contrast, arithmetic coding can directly map messages to the [0,1] interval, avoiding the limitations of discrete encoding methods.

[0037] like Figure 6 As shown, this invention studies how to locate the token sequence of the text to be compressed to a high-precision interval within an initial large model using arithmetic encoding. Then, it iterates through the probability distribution of each token within the existing context of the large model to find the encoding interval of the token sequence corresponding to that interval. Essentially, this is a technique for compressing and encoding text using statistical learning of a large model. This technique relies on the superior learning ability of the large model and, through context constraints, ensures that the compression ratio of the generated compressed sequence exceeds the Shannon theory limit while maintaining lossless compression. This technique fully utilizes the large model's deep understanding of language structure and context, not only improving the compression ratio but also allowing the receiver to accurately extract the target text from the generated binary sequence during decoding, thereby restoring the original text information to be compressed. With the continuous advancement of large model technology, this large model-based text compression and encryption is expected to show greater potential in improving information compression efficiency and security.

[0038] 4.1 Process of sampling token probabilities in a large model The text to be compressed is mapped to a range using arithmetic encoding. real numbers within .like The binary representation of is Then it can be written as To generate the required lossless compressed bitstream `text_bit`, a suitable context is selected and provided to the LLM to ensure that the text represented by the generated sequence is semantically and logically consistent with the original message. Based on this, the LLM, according to the context... Calculate each generation The probability distribution of can be expressed as . in, Indicates LLM to The original rating, and It is in the vocabulary list The total number.

[0039] Based on generation probability and Given a defined probability range, the large-scale text compression model selects the appropriate token to compress and encode a unique bitstream.

[0040] like Figure 7 As shown, by initializing the context to determine the compression mode and special requirements, the probability distribution of tokens is traversed in the large model of the existing context to find the token sequence that can uniquely identify the original text content. In fact, this is a technique that maps text to a pseudo-random sequence generated by the large model. Based on the excellent generation capability of the large model, this technique can make the generated sequence represent text with the same semantics as the original text. This ensures that the technique is applicable to the compression of English and numbers, and also extends the scope of text compression and encryption technology to the compression of multiple languages. This provides applicable compression methods for key scenarios such as anonymizing financial transaction data in multilingual contexts and protecting the privacy of electronic medical records. With the continuous improvement of technology and standards, the current compression challenges can certainly be solved.

[0041] 4.2 Text Compression Based on Arithmetic Coding Traditional text data compression schemes such as Zip encoding and arithmetic encoding are out of step with modern AI technology and cannot adapt to LLM-driven intelligent business scenarios. At the same time, traditional text compression and encryption technologies are difficult to implement in commercial applications, and it is difficult to balance security and usability.

[0042] have Figure 1 Large Language Models (LLMs) are natural language processing systems trained on large-scale neural networks, possessing powerful text understanding, generation, and reasoning capabilities. Compared to traditional text compression methods, LLMs can generate compressed results with variable compression types while maintaining language fluency.

[0043] Text compression based on LLM ensures that lossy compressed text, after decompression, not only maintains semantic and logical consistency with the original text but also conforms to language conventions, avoiding disruption of the original data's business logic and improving usability. Lossless compressed text, after decompression, is completely identical to the original text. By introducing key-encapsulated encryption technology, text compression and encryption are not only suitable for structured data but also applicable to various natural language data such as chat logs, emails, logs, and database text fields. This ensures that information remains readable and usable while compressing it, while reducing the risk of information and privacy leaks.

[0044] By combining arithmetic coding with LLM (Limited Language Management), and comparing it with Huffman coding and Block coding, arithmetic coding can store bit information more efficiently, approaching the entropy limit of information theory and improving compression efficiency. At the same time, it ensures that the compressed text distribution is consistent with natural language, improving data usability and readability. Furthermore, by using different contextual inputs through LLM, the flexibility of encrypted messages is achieved, enabling dynamic compression.

[0045] like Figure 8 As shown, in text compression based on arithmetic coding, we combine the last layer of logits output by the large language model under specific contextual conditions to obtain the probability distribution of different tokens. Specifically, we map the probability of each token to the interval [0,1]. Through arithmetic coding, we can obtain a real number interval that corresponds one-to-one with the ciphertext. When we provide the large language model with the context that restricts the compression mode, the probability distribution of the model changes, thereby generating a compressed result consistent with the required mode.

[0046] In summary, we map the target text to be compressed onto probabilities in the [0,1] interval, and use arithmetic coding and LLM to output a special binary sequence of the target text. By controlling the context of the input to the large language model, we control the output compression mode type to be consistent with the compression requirements.

[0047] 5. Construct a ciphertext integration mechanism for processing ciphertext. The application of key encapsulation requires the sending end to provide the receiving end with two sets of ciphertext: the encrypted symmetric encryption key and the ciphertext encrypted with that key. Traditionally, the encrypted key is transmitted first to establish a secure communication environment through a session key. Under this environment, bidirectional communication can be completed by exchanging the encrypted ciphertext. However, text compression and encryption require a single encryption and decryption process, necessitating the search for a mechanism that integrates the ciphertext.

[0048] like Figure 9As shown, based on knowledge of computer networks, we introduce an encoding method called zero-bit stuffing to ensure the reliability and accuracy of data transmission. Specifically, bit stuffing is a technique that inserts extra bits into a data stream. It prevents the receiver from misinterpreting consecutive "1" bits as a boundary between two ciphertexts by inserting a "0" bit after every five consecutive "1" bits. The implementation of zero-bit stuffing is relatively simple. Before sending the ciphertext to the large model for encryption, the sender checks the bit sequence in the ciphertext. Whenever five consecutive "1" bits are detected, the sender inserts a "0" bit after these bits and embeds a special sequence like "01111110" between the two ciphertexts to form a pseudo-random bit sequence. After receiving the pseudo-random bit sequence, the receiver performs the opposite operation, deleting these inserted "0" bits and restoring the original data.

[0049] 6. Bitstream encryption mechanism based on key encapsulation like Figure 10 As shown, this key encapsulation strategy retains the high efficiency of symmetric key encryption while combining the convenient and secure key management features of asymmetric encryption technology. This allows the entire encryption system to have higher carrying capacity and a more secure architecture. The solution adopts a hybrid design paradigm of "asymmetric encryption to manage key distribution and symmetric encryption to ensure business performance", which effectively solves the key explosion problem during large-scale deployment.

[0050] 6.1 Core Architecture of Hybrid Key Encapsulation Mechanism The system enhances security through three key phases: 1. Key Generation Phase: Elliptic Curve Cryptography (ECC) is used to generate public and private key pairs. Among them, safety parameters satisfy 800-186 standard 2. Key encapsulation stage: using Generate temporary session key Plaintext is completed using a compression encryption algorithm (CAE). Text compression and encryption And use the public key to encrypt the session key. 3. Ciphertext synthesis stage: Zero-bit stuffing coding standard is used to synthesize the ciphertext. Structured splicing ensures format compatibility and transmission integrity. 6.2 Quantitative Improvement of Security and Performance Compared to traditional purely symmetric key systems, the hybrid architecture achieves breakthroughs in three dimensions: 1. Key management complexity: number of communication nodes Key storage capacity from Down to A system with tens of thousands of nodes can reduce key storage requirements by 99.9%. 2. Forward security assurance: through The key negotiation protocol enables dynamic updates of the key for each session, satisfying... 800-56C standard 6.3 Compatibility Innovations with Existing Standards The solution creatively integrates group encryption. Standards and Public key encryption specifications form a layered security protocol: Technical approach: like Figure 11 As shown, this project aims to achieve text compression and encryption using LLM and AC, while introducing a key encapsulation mechanism to improve encryption efficiency and security. First, the probability distribution of each token is output using the conditional probability distribution of LLM. Arithmetic coding is then used to map the compressed text to the [0,1] interval, outputting the corresponding binary compressed sequence. By adjusting the context of the large model input, we can obtain tokens of different compression types. By selecting a token identical to the compressed text sequence, we can output compressed text with the same semantics as the original text. Next, we use PKE to encrypt the symmetric key, which encrypts the aforementioned binary compressed sequence. Zero-bit padding is then introduced to connect the encrypted symmetric key to the compressed sequence, achieving text compression and encryption.

[0051] In summary, we map the target text to be compressed onto probabilities in the [0,1] interval, and use arithmetic coding and LLM to output a special binary sequence of the target text. By controlling the context input to the large language model, we ensure that the initial parameters of the large model used for decompression are consistent with those used for compression, and use the AES-GCM algorithm to encrypt the compressed text bit sequence under hierarchical key protection to obtain the encrypted ciphertext, thus achieving text compression and encryption.

[0052] Proposed problems to be solved: 1. Traditional data compression algorithms are gradually becoming unsuitable for current scenarios. Core Challenge: In the process of in-depth research and discussion of traditional compression algorithms, we often encounter a very prominent problem: these algorithms tend to have overly simplistic compression modes, lacking the necessary flexibility and diversity. This simplification significantly limits the performance and adaptability of the algorithms in various application scenarios, because in reality, different data types, different data structures, and different needs often require different compression strategies and methods. To better meet diverse data compression requirements, the design and implementation of compression algorithms must take this diversity into account, thereby providing more flexible and efficient solutions.

[0053] Technological breakthroughs: such as Figure 5 As shown, the compression mode and special requirements are determined by initializing the context, and the text to be compressed is output. Then, the compressed text is input into a large model with an existing context, and a token segmenter is used to divide the text into token sequences. Figure 7 The method shown obtains the probability distribution of tokens, traverses the probability distribution of tokens to find the token that uniquely matches the original text content, maps the interval to its distribution, and repeats this process multiple times until the token sequence is completely represented by a high-precision interval. The shortest binary decimal in the interval is taken as the compression result. In fact, text compression technology based on large models is a technique that maps text to a large model and uses arithmetic encoding to generate a binary sequence. This technique, based on the excellent generation capabilities of large models, can ensure that the generated sequence represents the exact same text as the original text. This ensures that the technology is applicable to the compression of English and numbers, and also extends the applicability of text compression and encryption technology to the compression of multiple languages. This provides applicable compression methods for key scenarios such as anonymizing financial transaction data in multilingual contexts, protecting the privacy of electronic medical records, and high-speed communication. With the continuous improvement of technology and standards, the current compression challenges can certainly be solved.

[0054] 2. The problem of elastic supply of key management Core challenges: Traditional text compression and encryption use symmetric key encryption methods. This encryption method is fast, but due to the characteristics of symmetric cryptography, the system faces two challenges: First, key distribution depends on secure channels, making it difficult to achieve reliable transmission in complex networks. Second, due to the characteristics of symmetric keys, encryption keys need to be changed frequently, which greatly increases the cost of key management.

[0055] In a distributed environment, existing symmetric key schemes require strict maintenance of pre-shared keys, leading to numerous difficulties in key distribution and management. First, for a communication network with n nodes, the number of keys required to achieve fully connected secure communication follows the combination formula C(n,2)=n(n-1) / 2. When the system scale reaches the thousand-node level (n=1000), the total number of keys surges to approximately 500,000 sets; if expanded to the tens of thousands of applications (n=10,000), the key management requirements will exceed 50 million sets. This leap in scale not only causes a geometric increase in key storage overhead but also causes key distribution latency to exceed the tolerance threshold of the business. Second, the dynamic key distribution requirements of modern distributed systems fundamentally conflict with the static key pre-setting mechanism of traditional symmetric encryption. Traditional key pre-allocation schemes based on predefined topology structures cannot adapt to dynamic changes in network topology, nor can they meet the immediate access needs of massive numbers of terminal devices. This structural contradiction between rigid architecture and flexible requirements has become a key obstacle restricting the penetration of compression encryption technology into emerging fields such as the Internet of Things and the Industrial Internet.

[0056] Technological breakthroughs: such as Figure 3 As shown, this scheme adopts a hybrid design paradigm of "asymmetric encryption for key distribution and symmetric encryption for business performance assurance," effectively mitigating the key explosion problem during large-scale deployments. Regarding key encapsulation, based on traditional symmetric key encryption, we propose a key encapsulation strategy that integrates asymmetric and symmetric encryption. Traditional methods typically use symmetric key encryption to compress and encrypt plaintext. Building upon this, we use the public key of asymmetric encryption technology to perform secondary encryption on the symmetric encryption key. We first encrypt the plaintext using the symmetric encryption key, and then, within the asymmetric key framework, concatenate the encrypted plaintext and the symmetric key into a single ciphertext for transmission. The ciphertext receiver can decrypt the ciphertext using their private key, and then use the decrypted symmetric key to decrypt the ciphertext, thus obtaining the plaintext that actually needs to be encrypted for transmission.

[0057] This key encapsulation strategy cleverly combines the high speed and efficiency of symmetric key encryption with the convenience and security of asymmetric encryption in key management, thereby giving the entire encryption system higher carrying capacity and a more robust security architecture.

[0058] 3. Text compression and encryption problems based on large models Core Challenge: Generative text compression and encryption is a text compression technique based on a large language model. It uses arithmetic coding to map text to the probability range generated by the large model. Unlike traditional text compression techniques, generative text compression and encryption not only focus on how to compress text more effectively within a carrier, but also ensure that the generated carrier does not expose the original information of the text in visual, auditory, or other perceptual dimensions. This process efficiently compresses information without theft.

[0059] Technological breakthroughs: such as Figure 7 As shown, we combine the last layer of logits output by the large language model in the context-free case to obtain the probability distribution of different tokens. Specifically, we map the probability of each token to the interval [0, 1]. Figure 8 Through arithmetic encoding, we can obtain a real number interval that uniquely corresponds to the text. When we provide the context of the desired compression type to a large language model, the model's probability distribution changes, thus generating a text compression result consistent with the requirements. For example... Figure 10 The compressed bitstream, protected by a hierarchical key, is then encrypted into encrypted bitstream ciphertext by a session key for data transmission.

[0060] Example: I. System Architecture and Encryption Process The system adopts a layered architecture, which includes a key management layer, a large model pattern processing layer, and a large model execution layer.

[0061] The encryption process consists of three stages: Key pre-distribution: A master key pair is generated through an asymmetric encryption system to establish a secure communication foundation; Dynamic encryption: Bitstream encryption is performed using a dynamic session key, and the bit sequence data obtained from plaintext compression is processed in real time; Ciphertext generation: Concatenate the encrypted compressed bitstream with the session key into a single unit for communication transmission; II. Adaptation and Implementation of Text Compression Step 1: Dynamic Mode Compression Use a large language model to perform semantic analysis on the context and divide the restricted units according to part of speech; Example: Context "lossy compression makes Chinese more official" → word segmentation into [compression mode][language type][other special requirements]; Text compression is performed independently on each unit without disrupting semantic logic and paragraph structure; Step 2: Original text compression processing The input original text is transformed into the text to be compressed according to the context constraints. Lossy semantic compression performs semantic filtering on the compressed text to obtain the best text to be compressed. The compressed text is then input into the lossless compression model. Lossless compression inputs the compressed text into the lossless compression model. Implement a large model compression module, which establishes a mapping between text and binary sequence through large model and arithmetic coding, ensuring that the compressed binary sequence represents a complete match between the text to be compressed; III. Hybrid Key Management Mechanism 1. Key generation and distribution The management center generates ECC asymmetric key pairs, and the public key is pre-installed on all terminal devices; Session key dynamic generation rules: By default, a key update is triggered every 50MB of data processed or after 1 hour of continuous use. Abnormal access triggers immediate key reset (more than 10 requests per second from a single IP). 2. Secure key storage A fragmented storage strategy is adopted to split the key into: Device feature fragments (stored in the terminal security zone); Dynamically validate shards (stored in the cloud HSM module); The two-factor activation mechanism requires simultaneous verification of the device fingerprint and dynamic password to synthesize a complete key; IV. Implementation of Large-Scale Model Integration 1. Model training optimization Construct a dedicated training corpus: The compressed text corpus accounts for ≥80%, covering 10 professional fields such as finance and healthcare, and nearly 30 commonly used language text types; Injecting 5% adversarial examples enhances the model's resistance to analysis. Implement quantitative compression: Construct memory variables to ensure lossless encryption; Maintain a lossy text compression accuracy of >99.9%; 2. Compressed Sequence Generation Control Context injection mechanism: Example: When text needs to be compressed, inject the context "[compression mode][language type][other special requirements]" into the model to obtain the text to be compressed through semantic filtering; Based on the probability interval mapping of the text to be compressed using a large model: The original text to be compressed is used to obtain a token sequence through a word segmenter. Select tokens sequentially from the token sequence and map them to the token probability distribution output by the large model using arithmetic encoding; The token sequence is compressed into a bit stream using arithmetic coding and probability distribution; V. Specific Examples Original text: "text": "Name: Decision (EU) 2019 / 276 of the European Parliament and of the Council of 12 December 2018 on the mobilization of the FlexibilityInstrument to reinforce key programs for the competitiveness of the EU and to finance immediate budgetary measures to address the ongoing challenges of migration, refugee inflows and security threats\n Type: Decision\n SubjectMatter: cooperation policy; budget; EU finance... Lossy semantic-preserving compression (optional): "text": "Decision (EU) 2019 / 276: Mobilisation of FlexibilityInstrument to enhance EU competitiveness programs and address migration,security challenges (2018)"Type: Decision / n Subject: EU budget, finance,cooperation policy... Compressed text (binary): 101100110001010101010000001001001110101010111000111111100101011000001101100111111100110101001110111101001010101001010001000001001001111101010001110000000101010011000000101010011000000100111011010000011100001100 011100111010011101110010000110011011111011111100100111000000100010110111111000111111011111000110010100010100101110100100011001001000001000110110000010111001000000001110100110110110111111010011101100100000110011010011... Compressed encrypted text: 80gYu843VHA8c3KRCePGETH / tdd9lFIUqcdjruWrPza+I62sRlT / XTdDBypWh1SBz6XKc472HVQR7qu6M15cA01afN / WPYITJXqFIXtmH+5 Xbe9Wrl1gZmk5jiFojWbQ1MUDK809rzOGJ9HbLP2QRngqp9yvOp7zeypVw2jHF6+ssCcbDX7Q6di2zYI5frlbjsAf922bf6JIbL8EvEdCLw nLUlsytCAV0HZlBP980WDMDA8RoFw80aP8+x+uFVgc9tt1I73iLzAXHFTzwh / K4iv9ircBAMnnPP9IletIlugwXlF+bitbPsvxQyHLQ2geS OPmSJkPGI9JRTaHuB3SXIMeks1qbeQmWAvkmgNR5E38ZzVzkbuoMzcTm3iH7kKKXu / C20EH36piKF3GqWf / uE+8lH877cVvUcJEeE6nL... VI. Performance Indicators Verified by the testing platform (configuration: 13th Gen Intel(R) Core(TM) i5-13500H 2.60 GHz, 16GB RAM): Text compression speed: 10.85 it / s (approximately 3 times faster than traditional compression algorithms); Text decompression speed: 5.75 it / s (approximately 3 times faster than traditional compression algorithms); Tens of thousands of key management: Key retrieval time <15ms (cluster deployment); Semantic logic accuracy: 99.9% (tested on a set of commonly used languages); Source file size: 10080 B (bytes) Compressed size: 115 bytes Size after compression and encryption: 160 bytes This solution achieves efficient text compression and encryption in a multilingual environment through technological innovation, and has significant application value in fields such as high-speed communication and data compression.

[0062] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A text compression and encryption method based on a large model, characterized in that, Includes the following steps: S1: Preprocess the text based on context requirements to generate the text to be compressed; S2: Map the text to a binary bitstream by compressing a large language model; S3: Encrypt the bit stream using a dynamic symmetric key to generate ciphertext; S4: Generate an asymmetric encryption public-private key pair, and use the public key to encrypt the dynamic symmetric key to obtain the encrypted session key; In step S4: The asymmetric encryption uses an elliptic curve cryptography system, and the symmetric encryption uses a symmetric encryption algorithm with authentication and encryption functions. The dynamic symmetric key is generated by negotiating the timestamp and the text to be compressed using a key derivation function. S5: Concatenate the ciphertext with the encrypted session key in a structured manner to form a ciphertext bit stream to be transmitted; In step S5, zero-bit stuffing encoding is used to perform format compatibility processing on the concatenated ciphertext. Specifically, during the concatenation process, when five consecutive "1" bits are detected, a "0" bit is inserted, and a pseudo-random delimiter is inserted in the middle of the ciphertext to ensure transmission integrity. The receiving end performs the reverse operation, deleting the inserted "0" bit to restore the original data. S6: Based on the context semantic generation capability and encryption reversibility of the large language model, the compressed encrypted bit stream, after decompression, is consistent with the compression requirements of the large model context representation in S1. In steps S2 and S6: During compression, the target of lossless / lossless compression of the text can be determined by inputting the context content into the large language model according to the actual situation. The text to be compressed is mapped to a probability interval and compressed into a bit stream through arithmetic coding. By adjusting the input context to the large language model to be the same as that used during compression, it is ensured that the decompressed text is completely consistent with the original text in terms of Chinese characters, numbers, or mixed formats, just like the compression target during compression.

2. The text compression and encryption method based on a large model according to claim 1, characterized in that: In step S1: Based on contextual requirements, the large model can identify the user's need for lossless / lossy compression and process the original text. In lossless compression, the large model directly restates the original text and generates the text to be compressed. In lossy compression, the large model extracts semantics from the original text, preserves the semantic logic of the original text, and generates new text to be compressed.

3. A text compression and encryption system based on a large model, characterized in that, include: The large model processing module processes the original text into text to be compressed based on the context. In lossy compression mode, it uses the large model's understanding of semantics to preserve the semantic logic of the original text and generate a shorter text to be compressed. The key management module is used to generate asymmetric key pairs and dynamic symmetric keys, and to complete key encapsulation. Large Language Model Compression Processing Module: Uses a lossless compression large language model based on arithmetic coding to map the text to be compressed into a binary sequence through the token probability distribution inside the large model; The encryption module is used to perform bitstream encryption and ciphertext concatenation; The decryption module is used to parse the encrypted symmetric key using the private key and restore the original compressed bit sequence. The large language model decompression module is used to generate target text from binary bit sequences through probability intervals generated by the large model; The system supports encryption processing of binary bit sequences, specifically including: Dynamically generate the keys required for symmetric encryption algorithms; The bit stream obtained through compression is encrypted using a symmetric encryption algorithm; The key required for symmetric encryption algorithms is encrypted using the public key of asymmetric encryption. By concatenating ciphertext, the key required for the encrypted symmetric encryption algorithm is combined with the compressed bit stream obtained after encryption to form a whole that is not compatible with the original format.

4. The text compression and encryption system based on a large model according to claim 3, characterized in that: The large model processing module further includes: The context includes the compression mode. When selecting a lossy compression mode, constraints can be imposed on the content loss of the text. By leveraging the understanding capabilities of the large model, the text can be processed into a text that meets the requirements of the context.

5. A text compression and encryption system based on a large model according to claim 3, characterized in that: The key management module further includes: Elliptic curve-based asymmetric encryption algorithm key generation unit is used to generate public and private key pairs that conform to the NISTSP800-186 standard. The key derivation unit generates a dynamic session key with authentication encryption (AES-GCM) based on the timestamp and the text to be compressed.

6. A text compression and encryption system based on a large model according to claim 3, characterized in that: The large language model compression module achieves text compression in the following ways: Extract the token probability distribution output by the large language model; The encoded bitstream that uniquely matches the token sequence of the text to be compressed is selected based on the probability distribution of the tokens in the arithmetic encoding.

Images