Semantic compression method based on generative large language model and huffman coding

Abstract

The application discloses a semantic compression method based on a generative large language model and Huffman coding, and comprises the following steps: S1, semantic extraction; S2, keyword coding based on Huffman coding; S3, keyword decoding based on Huffman coding; and S4, semantic recovery based on a generative language large model. The application takes keywords as carriers to extract the semantics of natural language, thereby optimizing unnecessary grammar and connectivity content without losing important semantic information, and reducing unnecessary bandwidth occupation.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and semantic communication technology, and more specifically, relates to a semantic compression method based on generative large language models and Huffman coding. Background Technology

[0002] Semantic communication refers to the refinement of information during communication by analyzing and understanding semantic information in language, in order to convey a more precise and efficient meaning. It utilizes natural language processing and semantic technologies to represent, transmit, and understand the meaning of information.

[0003] The key to semantic communication is using semantic understanding and analysis techniques to compress information, conveying richer meaning within limited bandwidth. This can significantly improve network communication efficiency and achieve a higher quality user experience.

[0004] The development of natural language processing technology has made it possible to extract semantic information, which has laid the foundation for the implementation of semantic compression and semantic communication technologies. Using semantic layers to compress information can filter out redundant and irrelevant semantics, transmitting only the key semantic information.

[0005] In the future, with the widespread adoption of 5G networks and the development of edge computing, semantic communication technology is expected to see widespread application. It can fully utilize the enhanced computing power and bandwidth of the network to achieve more refined semantic interaction and understanding. Users can then enjoy a more accurate and seamless experience.

[0006] Key research areas include semantic representation learning, semantic compression models, channel-based semantic compression methods, and real-time semantic interaction mechanisms. The development of these technologies will significantly improve the performance of semantic communication.

[0007] With advancements in artificial intelligence and natural language processing technologies, semantic communication technology is poised for rapid development, and its application scenarios will continue to expand. As 5G networks become more widespread, semantic communication technology will be extensively applied, enabling higher-quality user experiences. People's pursuit of a high-quality life will also drive semantic communication into more vertical industries, realizing intelligent manufacturing in sectors like industry and agriculture. The development of semantic communication standards and protocols will accelerate its commercialization, creating a broader market space.

[0008] Currently, research on semantic communication is still in its early stages. The main implementation methods for semantic communication technology are as follows:

[0009] Ontology-based semantic communication technology. An ontology, also known as a knowledge base, is a tool for formally representing general knowledge such as concepts, attributes, and relationships. It can be used to describe semantic information in a specific domain, thereby enabling semantic interaction within that domain.

[0010] Rule-based semantic communication technology. A rule is a tool for formally representing logical reasoning and inference processes. It can be used to describe semantic mappings between different languages ​​and cultures, thereby enabling cross-linguistic and cross-cultural semantic translation.

[0011] Semantic communication technology based on machine learning. Machine learning is a method that uses data and algorithms for knowledge acquisition and pattern recognition. It can be used to learn the semantic relationships between different languages, cultures, and domains, thereby achieving automated semantic matching and adaptation.

[0012] Furthermore, there is currently no complete semantic communication method specifically designed for semantic compression. Summary of the Invention

[0013] Based on the above situation, this invention provides a semantic compression method based on generative large language models and Huffman coding. It aims to reduce the cost of ontology construction and maintenance, cover semantic information across all domains and scenarios, reduce the difficulty of rule writing and updating, adapt to constantly changing and evolving language and cultural characteristics, and ensure the accuracy and efficiency of semantic communication.

[0014] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0015] According to one aspect of the present invention, a semantic compression method based on a generative large language model and Huffman coding is provided, comprising the following steps:

[0016] The method includes the following steps:

[0017] S1, semantic extraction;

[0018] S2. Keyword encoding based on Huffman coding;

[0019] S3. Keyword decoding based on Huffman coding;

[0020] S4. Semantic recovery based on generative language large models.

[0021] Furthermore, the specific steps of S1 are as follows:

[0022] S11. Collect domain-related corpora, use the corpora as input through prompt, and obtain the required semantic information through the API of the business big model;

[0023] S12. Filter the obtained semantic information, remove the suggestive text, and format the semantic information data;

[0024] S13. Combine the input corpus and output semantic information into a JSON file and use it as the training or fine-tuning dataset for the local large language model.

[0025] S14. Based on an open-source language model, train or fine-tune it using a JSON file to obtain a local model specifically for semantic extraction.

[0026] Furthermore, the specific steps of S2 are as follows:

[0027] S21. Perform word frequency statistics using the semantic information keywords in S11;

[0028] S22. Based on the word frequency, obtain a character frequency table; the character frequency table gives each character and its corresponding number of occurrences;

[0029] S23. Construct a Huffman tree based on the character frequency table. Merge two characters with high frequencies into one node, and the frequency of the node is the sum of the frequencies of the two characters. Repeat this step until a tree is constructed, where each leaf node represents a character.

[0030] S24. Assign a bit encoding to each character, starting from the root node, assign 0 bits to the left and 1 bit to the right; traverse to each leaf node to obtain the corresponding character encoding codebook;

[0031] S25. Perform Huffman coding on the words with corresponding codes that exist in the extracted keywords;

[0032] S26. Record the keywords in the original keyword string that correspond to the encoded words as 1, and the keywords that do not correspond to the encoded words as 0, and record the binary position information of whether the keywords are encoded in an orderly manner.

[0033] S27. Merge all encoded binary strings and the binary string recording position information, with the encoded sequence first and the position information second.

[0034] Furthermore, the specific steps of S3 are as follows:

[0035] S31. Start Huffman decoding from the beginning of the sequence;

[0036] S32. Count the total number of decoded keywords. Decoding stops when the length of the remaining binary string of position information is equal to the total number of decoded words.

[0037] S33. Based on the remaining position information binary string, sort the decoded keywords and the original keywords that have not been encoded to obtain the keyword string in the original order.

[0038] Furthermore, the specific steps of S4 are as follows:

[0039] S41. Swap the keywords and sentence names in the JSON file in S13 to get the result as the training or fine-tuning dataset for the local large language model.

[0040] S42. Based on an open-source language model, train or fine-tune it using a JSON file to obtain a local model specifically for semantic recovery.

[0041] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the semantic compression method based on a generative large language model and Huffman coding of the present invention.

[0042] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the semantic compression method based on a generative large language model and Huffman coding of the present invention.

[0043] Compared with existing technologies, the present invention has at least the following beneficial effects: The present invention uses keywords as a carrier to extract semantics from natural language, thereby optimizing unnecessary grammar and connectivity content without losing important semantic information, and reducing unnecessary bandwidth consumption.

[0044] This invention introduces a database based on Huffman coding during the transmission of keywords, further compressing high-frequency keywords and thus improving the transmission efficiency of semantic communication.

[0045] Given the relative complexity of existing semantic communication systems, this invention proposes a complete, feasible, and easy-to-implement semantic communication transceiver implementation scheme.

[0046] This invention further compresses high-frequency keywords while retaining low-frequency words, avoiding excessively long overall encoding code elements that could affect encoding efficiency. It also provides a sorting scheme to ensure that the order of keywords remains unchanged and that the correct word order of subsequent restored sentences is maintained. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.

[0048] Figure 1 This is a flowchart of the invention;

[0049] Figure 2 This is a flowchart of the keyword encoding and decoding operation of the present invention. Detailed Implementation

[0050] The following is in conjunction with the appendix Figure 1-2 The specific implementation details of the present invention are described in the examples.

[0051] Example 1:

[0052] like Figure 1 As shown, this invention provides a semantic compression method based on a generative large language model and Huffman coding, comprising the following steps:

[0053] Step A: Use web crawlers to obtain a large amount of publicly available corpus data on the communications industry from the communications section of news websites;

[0054] Step B: Format the corpus obtained in Step A, align and segment it into sentences, and output it as a CSV file;

[0055] Step C: Call the ChatGPT API to generate keywords for each sentence according to the following prompt: "Please provide the keywords for the following sentences, separated by spaces, and ensure that the keywords cover all the important information in the original sentence so that semantically identical sentences can be recovered using these keywords in the future."

[0056] Step D: Check the format of the output keywords and save the output results to the second column of the CSV table in Step B;

[0057] Step E: Format the contents of the CSV table into JSON format, use it as the training corpus for ChatGLM-6B, and obtain a local model for extracting keywords through fine-tuning;

[0058] Step F: Swap the first and second columns of the CSV table, and repeat step E to obtain another local model that recovers the statement based on keywords;

[0059] Step G: Perform word frequency statistics on the keywords collected in Step D, and construct a Huffman tree based on the character frequency table. Two characters with high frequencies are merged into one node, and the frequency of this node is the sum of the frequencies of the two characters. Repeat this step until a tree is constructed, where each leaf node represents a character.

[0060] Step H: Assign a bit encoding to each character. Starting from the root node, allocate 0 bits to the left and 1 bit to the right. By traversing to each leaf node, the corresponding character encoding codebook is obtained.

[0061] Step I: The sending end extracts keywords from the statement to be sent based on the model in step E.

[0062] Step J: Perform Huffman coding on the words with corresponding codes among the extracted keywords.

[0063] Step K: Record the keywords in the original keyword string that correspond to the encoded words as 1, and the keywords that do not correspond to the encoded words as 0, and record the binary position information of whether the keywords are encoded in an orderly manner.

[0064] Step L: Merge all encoded binary strings and the binary string recording position information, with the encoded sequence first and the position information second.

[0065] Step M: For unencoded keywords, retain the original format, that is, form a new string using spaces as separators.

[0066] Step N: Define a custom 8-bit binary sequence to character encoding mapping, such as 00000001 corresponding to α. Then, encode the binary string from Step L into a regular character string for easy visualization and information transmission, thus obtaining a new string.

[0067] Step O: Connect the new string obtained in step N with the unencoded keyword string in step M, and send them together to the receiving end through a traditional channel.

[0068] Step P: For the new string in step O, the receiving end first converts the characters into a binary string according to the custom encoding mapping, and then performs Huffman decoding on the binary string starting from the beginning.

[0069] Step Q: Count the total number of decoded keywords. Decoding stops when the length of the remaining binary string of position information equals the total number of decoded words.

[0070] Step R: Based on the remaining position information binary string, sort the decoded keywords and the original keywords that have not been encoded to obtain the keyword string in the original order.

[0071] Step S: Based on the other local model obtained in step F, input the keyword string obtained in step R, and obtain the recovered statement based on the keywords.

[0072] For those who want to compress further, the encoded and unencoded keywords obtained in step N can be compressed using traditional algorithms, and then decompressed at the receiving end in the same traditional way.

[0073] During the decoding operation, because the character encoding in step N may introduce extra zeros during byte alignment, it is necessary to check whether the remaining position information binary string is long enough after each Huffman decoding operation. If the length is insufficient, one decoding step needs to be reversed, and the zeros introduced by byte alignment are removed based on the already decoded portion.

[0074] Theoretically, when the Huffman code is short and the remaining position information binary string ends with a large number of zeros, there is a very small chance of incorrect decoding due to the zeros introduced during byte alignment. However, as long as the shortest Huffman code length is greater than 6, decoding errors are impossible. Specifically, when the total number of extracted keywords plus the shortest code length is greater than 7, no decoding errors will occur after comparing the length of the remaining position information binary string.

[0075] Current semantic communication systems typically use LSTM neural networks to encode words based on contextual relevance and part-of-speech tags, and merge words with high semantic similarity to generate an optimal binary tree. This approach is complex to implement and struggles to cover all possible words.

[0076] Compared to such methods, the keyword-based semantic extraction, semantic encoding / decoding, and semantic recovery method provided in this invention can ignore words of lower importance such as conjunctions, modifiers, quantifiers, qualifiers, and prepositional modifiers in a sentence, focusing only on keywords related to specific content. Furthermore, by combining an industry-specific keyword encoding library to convert keywords into codes or word vectors, a higher degree of semantic compression can be achieved. Due to the theoretically optimal properties of Huffman coding, the more closely the sentence relates to the industry, the higher the compression rate can be achieved.

[0077] Example 2:

[0078] The computer-readable storage medium of this embodiment stores a computer program that, when executed by a processor, implements the steps in the semantic compression method based on a generative large language model and Huffman coding in Embodiment 1.

[0079] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.

[0080] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0081] Example 3:

[0082] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the semantic compression method based on a generative large language model and Huffman coding in Embodiment 1.

[0083] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0084] Those skilled in the art will understand that the content disclosed in the embodiments can be provided as a method, system, or computer program product. Therefore, this solution can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.

[0085] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0089] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.

Claims

1. A semantic compression method based on a generative large language model and Huffman coding, characterized in that, The method includes the following steps: S1, semantic extraction; S2. Keyword encoding based on Huffman coding; S3. Keyword decoding based on Huffman coding; S4. Semantic recovery based on a large generative language model; The specific steps of S1 are as follows: S11. Collect domain-related corpus, use prompt to take the corpus as input, and obtain the required semantic information through the API of a qualified commercial large model. S12. Filter the obtained semantic information, remove the suggestive text, and format the semantic information data; S13. Combine the input corpus and output semantic information into a JSON file and use it as the training or fine-tuning dataset for the local large language model. S14. Based on an open-source language model, train or fine-tune it using a JSON file to obtain a local model specifically for semantic extraction. The specific steps of S2 are as follows: S21. Perform word frequency statistics using the semantic information keywords in S11; S22. Based on the word frequency, obtain a character frequency table; the character frequency table gives each character and its corresponding number of occurrences; S23. Construct a Huffman tree based on the character frequency table. Merge two characters with high frequencies into one node, and the frequency of the node is the sum of the frequencies of the two characters. Repeat this step until a tree is constructed, where each leaf node represents a character. S24. Assign a bit encoding to each character, starting from the root node, assign 0 bits to the left and 1 bit to the right; traverse to each leaf node to obtain the corresponding character encoding codebook; S25. Perform Huffman coding on the words with corresponding codes that exist in the extracted keywords; S26. Record the keywords in the original keyword string that correspond to the encoded words as 1, and the keywords that do not correspond to the encoded words as 0, and record the binary position information of whether the keywords are encoded in an orderly manner. S27. Merge all encoded binary strings and the binary string recording position information, with the encoded sequence first and the position information second.

2. The method according to claim 1, characterized in that, The specific steps of S3 are as follows: S31. Start Huffman decoding from the beginning of the sequence; S32. Count the total number of decoded keywords. Decoding stops when the length of the remaining binary string of position information is equal to the total number of decoded words. S33. Based on the remaining position information binary string, sort the decoded keywords and the original keywords that have not been encoded to obtain the keyword string in the original order.

3. The method according to claim 1, characterized in that, The specific steps of S4 are as follows: S41. Swap the keywords and sentence names in the JSON file in S13 to use as the training or fine-tuning dataset for the local large language model. S42. Based on an open-source language model, train or fine-tune it using a JSON file to obtain a local model specifically for semantic recovery.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps in the semantic compression method based on generative large language models and Huffman coding as described in any one of claims 1 to 3.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the semantic compression method based on generative large language models and Huffman coding as described in any one of claims 1 to 3.

Images