Information processing method based on large language model, receiving method and related devices thereof

Abstract

The application relates to an information processing method based on a large language model, a receiving method and related devices thereof, which comprises the following steps: obtaining an initial message to be processed, a target key and a current context; inputting the initial message to be processed into a target large language model, and obtaining a hash value corresponding to a current token in the initial message in a process of generating a token of a target text, and obtaining at least two candidate token sets according to the hash value; determining a target token set from the candidate token set according to the current token, and performing sampling control on candidate tokens in the target token set based on an original probability distribution of the current token to obtain a target token corresponding to a current generation position of the target text of the current token, and obtaining the target text. The scheme provided by the application can embed information by using a hash algorithm and a sampling control mechanism in the process of generating the text, and can improve the naturalness of the output text and the unbiasedness of the statistical distribution.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an information processing method, receiving method and related apparatus based on a large language model. Background Technology

[0002] Steganography is a technique for hiding secret information in seemingly harmless media. Traditional text steganography methods include synonym substitution, formatting manipulation, and zero-width character insertion. These methods typically have low information embedding capacity, encoding only a small number of bits per sentence, and require a large, manually designed thesaurus, resulting in inefficient information embedding and easy detection.

[0003] In related technologies, watermarking technology using large language models can generate specific identifiers in text for the purpose of detecting generated content. However, when the architecture is one-way embedding, it only considers embedding fixed identifiers in the output text and is not designed as a bidirectional, variable content information transmission protocol. Furthermore, it cannot ensure the naturalness of the language and the unbiasedness of the statistical distribution of the output text, which reduces the security of steganography in the output text and makes it difficult for the recipient to obtain the content steganized in the output text. Summary of the Invention

[0004] To address or partially address the problems existing in related technologies, this application provides an information processing method, receiving method, and related apparatus based on a large language model. By embedding information during the text generation process using hash algorithms and sampling control mechanisms, it can achieve high-capacity and reliable information embedding in the generated text, while improving the naturalness of the output text and the unbiasedness of its statistical distribution.

[0005] The first aspect of this application provides an information processing method based on a large language model, comprising: acquiring an initial message to be processed, a target key, and a current context; inputting the initial message to be processed into a target large language model, and during the generation of lexical units of the target text, obtaining a hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key, and dividing candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein, the candidate lexical units in each candidate lexical unit set are different; determining a target lexical unit set from the candidate lexical unit sets according to the current lexical unit, and sampling and controlling the candidate lexical units in the target lexical unit set based on the original probability distribution of the current lexical unit to obtain the target lexical unit at the current generation position of the target text corresponding to the current lexical unit; and obtaining the target text according to a plurality of the target lexical units.

[0006] In conjunction with the first aspect, in one possible implementation of the first aspect, before inputting the initial message to be processed into the target large language model, the method further includes: performing message preprocessing on the initial message to obtain the encoded sequence to be embedded; wherein, the message preprocessing includes at least one of forward error correction coding, synchronization sequence insertion, and verification information appending; the step of determining the target lexical set from the candidate lexical set based on the current lexical includes: determining the target lexical set corresponding to the target encoded content based on the target encoded content corresponding to the current generation position in the encoded sequence.

[0007] In conjunction with the first aspect, in one possible implementation of the first aspect, the step of sampling and controlling candidate words in the target word set based on the original probability distribution of the current word to obtain the target word at the current generation position of the target text corresponding to the current word includes: obtaining the KL divergence between the context probability distribution corresponding to the current context and the original output probability distribution of the large language model at the current generation position; if the KL divergence is not less than a preset threshold, sampling is performed based on the original output probability distribution to generate the target word at the current generation position; if the KL divergence is less than the preset threshold, a logarithmic bias is applied to the original output probabilities of the candidate words in the target word set, and the probability distribution after applying the bias is normalized to obtain a sampled probability distribution, so as to generate the target word at the current generation position based on the sampled probability distribution.

[0008] In conjunction with the first aspect, in one possible implementation of the first aspect, applying a logarithmic bias value to the original output probabilities of candidate lexical units within the target lexical set includes: obtaining the information entropy of the original output probability distribution; dynamically adjusting the logarithmic bias value based on the information entropy; wherein, the higher the information entropy, the larger the value of the logarithmic bias value.

[0009] In conjunction with the first aspect, in one possible implementation of the first aspect, the step of obtaining the hash value corresponding to the current word in the initial message according to a preset hash algorithm, the current context, and the target key, and dividing the candidate words in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate word sets, includes: obtaining at least a portion of the word sequences in the word sequence before the current generation position, and concatenating the at least a portion of the word sequences with the target key to obtain hash input data; obtaining the hash value corresponding to the current generation position based on the preset hash algorithm and the hash input data; determining the candidate word set identifier corresponding to each candidate word according to the mapping relationship between the hash value and each candidate word in the vocabulary; and dividing the candidate words in the vocabulary according to the candidate word set identifier to obtain at least two candidate word sets.

[0010] In conjunction with the first aspect, one possible implementation of the first aspect further includes: the number of candidate lexical units is 2. n Where n is an integer greater than or equal to 1.

[0011] A second aspect of this application provides a method for receiving information in a large language model, comprising: receiving target text; inputting the target text into a target large language model and performing word segmentation on the target text through the target large language model to obtain a target word sequence; generating a hash value corresponding to the current word sequence based on a target key, a preset hash algorithm, and the current word sequence corresponding to the current word in the target word sequence; dividing candidate words in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate word sets; comparing the current word with the candidate word sets to determine the target candidate word set corresponding to the current word; obtaining a plurality of target words according to the target candidate word set, and obtaining an initial message according to the target words.

[0012] A third aspect of this application provides an information processing apparatus based on a large language model, comprising: an acquisition module for acquiring an initial message to be processed, a target key, and a current context; a processing module for inputting the initial message to be processed into a target large language model, and during the generation of lexical units of the target text, obtaining a hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key, and dividing candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein the candidate lexical units in each candidate lexical unit set are different; a determination module for determining a target lexical unit set from the candidate lexical unit set according to the current lexical unit, and sampling and controlling the candidate lexical units in the target lexical unit set based on the original probability distribution of the current lexical unit to obtain the target lexical unit at the current generation position of the target text corresponding to the current lexical unit; and a generation module for obtaining the target text according to a plurality of the target lexical units.

[0013] A fourth aspect of this application provides an electronic device, comprising: Processor; and A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0014] A fifth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0015] The technical solution provided in this application may include the following beneficial effects: This application discloses an information processing method, receiving method, and related apparatus based on a large language model. The method includes: acquiring an initial message to be processed, a target key, and a current context; inputting the initial message to be processed into a target large language model, and during the generation of target text lexical units, obtaining the hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key; dividing candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein the candidate lexical units in each candidate lexical unit set are different; determining the target lexical unit set from the candidate lexical unit sets based on the current lexical unit, and sampling control of the candidate lexical units in the target lexical unit set based on the original probability distribution of the current lexical unit to obtain the target lexical unit at the current generation position of the target text corresponding to the current lexical unit; and obtaining the target text based on several target lexical units. In this method, by embedding information using a hash algorithm and sampling control mechanism during the text generation process, high-capacity and reliable information embedding can be achieved in the generated text, while improving the naturalness of the output text and the unbiasedness of the statistical distribution.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0018] Figure 1 This is a flowchart illustrating an information processing method based on a large language model, as shown in an embodiment of this application. Figure 2 This is a flowchart illustrating an information receiving method based on a large language model, as shown in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an information processing device based on a large language model, as shown in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0019] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0020] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] Steganography is a technique for hiding secret information in seemingly harmless media. Traditional text steganography methods include synonym substitution, formatting manipulation, and zero-width character insertion. These methods typically have low information embedding capacity, encoding only a small number of bits per sentence, and require a large, manually designed thesaurus, resulting in inefficient information embedding and easy detection.

[0023] In related technologies, watermarking technology using large language models can generate specific identifiers in text for the purpose of detecting generated content. However, when the architecture is one-way embedding, it only considers embedding fixed identifiers in the output text and is not designed as a bidirectional, variable content information transmission protocol. Furthermore, it cannot ensure the naturalness of the language and the unbiasedness of the statistical distribution of the output text, which reduces the security of steganography in the output text and makes it difficult for the recipient to obtain the content steganized in the output text.

[0024] To address the aforementioned issues, embodiments of this application provide an information processing method, receiving method, and related apparatus based on a large language model. By embedding information during the text generation process using hash algorithms and sampling control mechanisms, high-capacity and reliable information embedding can be achieved in the generated text, while simultaneously improving the naturalness of the output text and the unbiasedness of its statistical distribution.

[0025] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart illustrating an information processing method based on a large language model, as shown in an embodiment of this application.

[0027] See Figure 1 An information processing method based on a large language model includes: S110: Obtain the initial message to be processed, the target key, and the current context.

[0028] Specifically, the initial message to be processed is the raw information or data that needs to be transmitted covertly. This information can be any form of digital data, such as text, binary messages, etc. It can be generated by the user manually inputting a piece of text. The user can convert the message they want to send into the initial message to be processed, which can contain a large amount of information in bits. The target key is a secret key shared between the sender and the receiver, which can be used for encryption at the sender and decryption at the receiver. For example, it can be a shared string. The current context is the generated sequence of lexical units. When the large language model generates the first target lexical unit, the current context can be a preset initial prompt word. The large language model can use this context to predict the probability distribution of the next lexical unit, thereby improving the naturalness and coherence of the generated text. For example, if a user wants to send a secret message, he / she can manually input "This is a secret message" and specify a key "secretkey123", while providing an initial context "Please generate a description about natural scenery:".

[0029] S120: Input the initial message to be processed into the target large language model, and in the process of generating the target text lexical, obtain the hash value corresponding to the current lexical in the initial message according to the preset hash algorithm, the current context and the target key, and divide the candidate lexical in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical sets; wherein, the candidate lexical in each candidate lexical set is different.

[0030] Specifically, when generating each word in the target text, the large language model predicts the probability distribution of the next word based on the current context. The preset hash algorithm can be MD5 or SHA-1. The large language model can combine the current context and the target key to calculate a hash value. This hash value can be used to divide the entire vocabulary of the large language model into multiple non-overlapping sets of candidate words. For example, the vocabulary can be divided into two sets: one set contains words whose hash value and word index result in an even number, and the other set contains words whose result is odd, ensuring that each word in the vocabulary belongs to only one set.

[0031] S130: Based on the current word element, determine the target word element set from the candidate word element set, and based on the original probability distribution of the current word element, sample and control the candidate word elements in the target word element set to obtain the target word element at the current generation position of the target text corresponding to the current word element.

[0032] Specifically, after the vocabulary is divided into multiple candidate word sets, the first candidate word set is selected based on the information to be embedded. For example, if the bit corresponding to the information is 0, the second candidate word set is selected; if it is 1, the first candidate word set is selected. Then, based on the original probability distribution output by the large language model at the current generation position, which is the inherent probability distribution of the next word set output by the large language model across the entire vocabulary under the current context and without any other processing, sampling control is applied to the candidate words in the target word set. Sampling control refers to the process of selecting words from the vocabulary based on the modified probability distribution. Finally, the target word set at the current generation position of the target text corresponding to the current word set is obtained.

[0033] S140: Obtain the target text based on several target words.

[0034] Specifically, by repeating the above-mentioned word generation and sampling control process, target words are generated one by one until the information content corresponding to all initial messages is embedded or reaches the preset text length. These sequentially generated words are then concatenated to obtain the target text. Through this embedding method, the traditional method of "number of bits per sentence" can be improved to "each word can be encoded into bits", which improves the efficiency of information transmission. Moreover, it can systematize the sampling bias mechanism of large language models into a complete bidirectional communication protocol, enabling the receiver to accurately extract the message sent by the sender from the received text through reverse processing.

[0035] For example, user A needs to send a secret message "An emergency meeting is scheduled for 10:00 AM tomorrow" to user B. Before sending, user A and user B share a target key K, such as "SecureKey2025". User A first converts the secret message into a binary bit stream, for example, "An emergency meeting is scheduled for 10:00 AM tomorrow" is encoded as "01101001..." and used as the initial message. Then, this initial message is input into the target large language model, and an initial prompt word is set, such as "Please write a report summary on recent market trends:". Then, the target large language model begins to generate the tokens of the target text. When the large language model needs to generate the first token, it obtains the current context and the target key, and according to the preset hash algorithm, ... The current context and target key are concatenated to calculate a hash value H1. The vocabulary of the large language model is divided into at least two candidate word sets based on the hash value H1. For example, the vocabulary can be divided into set C0 and set C1 based on a specific bit (such as the least significant bit) of the hash value H1. Set C0 contains words whose hash value and word index result in 0, and set C1 contains words whose result is 1. If the first bit of the initial message to be embedded is 0, then set C0 is determined as the target word set from the candidate word sets. Then, the large language model performs sampling control on the candidate words in set C0 to obtain a new sampling probability distribution. Based on this new sampling probability distribution, the first target word, such as "given that", is sampled from set C0. The large language model then generates the second lexical unit. The current context is updated to "Please write a report summary on recent market trends: Given...". The large language model calculates a new hash value H2 based on the updated context and target key, and re-divides the vocabulary, obtaining new candidate lexical sets C'0 and C'1. If the second bit of the initial information to be embedded is 1, set C'1 is selected as the target lexical set. Then, based on the original probability distribution, sampling control is applied to the candidate lexical units in set C'1 to obtain a new sampling probability distribution. Based on this new sampling probability distribution, the second target lexical unit, such as "recent," is sampled from set C'1. The new current context is then updated to "Please write a report summary on recent market trends: Given...". By repeating the above process, the bits corresponding to the initial message are embedded one by one, and the corresponding target lexical units are generated. User A obtains the target text, which can be a report summary, such as: "Given the complexity of recent market trends, we have observed volatility in several key indicators, requiring us to closely monitor and adjust our strategies accordingly." This target text can then be sent to user B.

[0036] In one possible implementation, before inputting the initial message to be processed into the target large language model, the method further includes: preprocessing the initial message to obtain the encoded sequence to be embedded; wherein the message preprocessing includes at least one of forward error correction coding, synchronization sequence insertion, and verification information appending; determining the target word set from the candidate word set based on the current word, including: determining the target word set corresponding to the target encoded content based on the target encoded content corresponding to the current generation position in the encoded sequence.

[0037] Specifically, forward error correction coding adds redundant information to the initial message to detect and correct transmission errors. For example, cyclic redundancy check (CRC) codes can be used to embed error correction codes in each data block of the initial message, allowing the receiver to recover the original information even if some data is corrupted. Synchronization sequence insertion refers to periodically inserting specific bit sequences or lexical sequences into the data stream; for example, a predefined synchronization codeword can be inserted at a fixed position in the message. Checksum appending involves adding extra checksum data to the message to detect whether errors or tampering have occurred during transmission or processing. By preprocessing the initial message, its integrity and robustness can be enhanced. Then, the target lexical set is determined from the candidate lexical set based on the encoded sequence, enabling the subsequent large language model to more accurately reflect the intent of the original message when generating target text, reducing the risk of deviations in generated text due to errors or tampering in the initial message.

[0038] In one possible implementation, based on the original probability distribution of the current word element, sampling control is performed on candidate words in the target word element set to obtain the target word element at the current generation position of the target text corresponding to the current word element. This includes: obtaining the KL divergence between the context probability distribution corresponding to the current context and the original output probability distribution of the large language model at the current generation position; if the KL divergence is not less than a preset threshold, sampling is performed based on the original output probability distribution to generate the target word element at the current generation position; if the KL divergence is less than the preset threshold, a logarithmic bias value is applied to the original output probability of the candidate words in the target word element set, and the probability distribution after applying the bias is normalized to obtain a sampled probability distribution, so as to generate the target word element at the current generation position based on the sampled probability distribution.

[0039] Specifically, by obtaining the KL divergence value, we can determine the degree of difference between the word probability distribution implied in the current context and the original word probability distribution predicted by the large language model at the current generation position. If the KL divergence value is larger, it indicates a greater difference between the two distributions, and a greater difference between the original prediction of the large language model and the prediction of the current context; if the KL divergence value is smaller, it indicates a closer similarity between the two distributions, and a smaller difference between the original prediction of the large language model and the prediction of the current context. The preset threshold can be set in advance. When the KL divergence is not less than the preset threshold, it indicates that there is a difference between the current context and the original output of the large language model. At this time, sampling can be performed directly through the probability distribution of the original output to generate the target word at the current generation position. When the KL divergence is less than the preset threshold, it indicates that there is little difference between the current context and the original output of the large language model. At this time, a logarithmic bias value can be applied to the candidate words in the target word set to adjust the relative probability of these words, so that the sampling is more inclined to words that conform to the context, thereby improving the quality of the generated text. The logarithmic bias value can be set in advance, for example, it can be obtained according to the matching degree between the word and the context. In this way, the sampling strategy is adaptively adjusted through KL divergence, so that the generated words can better improve the semantic coherence of the context and the generation quality.

[0040] For example, when generating text, the current context is "Please send encrypted", and the target word set is {"file", "data", "information", "report"}. To generate the next word, first calculate the context probability distribution corresponding to the current context "Please send encrypted" and the original output probability distribution predicted by the large language model at the current position. If the calculated KL divergence is 0.08 and the preset threshold is 0.1, it indicates that the difference between the model's original output and the context is not significant. In this case, a logarithmic bias can be applied to the original output probabilities within the target word set {"file", "data", "information", "report"}. For example, if the original probability distribution is P("file")=0.3, P("data")=0.5, P("information")=0.1, P("report")=0.1, a positive logarithmic bias is applied to "file" and "data", and a negative logarithmic bias is applied to the other words. After biasing and normalization, the new sampling probability distribution can become P'("file")=0.4. P'("data")=0.55, P'("information")=0.025, P'("report")=0.025, and then sampling is performed based on the new sampling probability distribution, so that "data" or "file" can be used as the target word at the current generation position, which can improve the contextual coherence and relevance of the generated text.

[0041] In one possible implementation, applying a logarithmic bias value to the original output probabilities of candidate lexical units within the target lexical unit set includes: obtaining the information entropy of the original output probability distribution; and dynamically adjusting the logarithmic bias value based on the information entropy; wherein, the higher the information entropy, the larger the value of the logarithmic bias value.

[0042] Specifically, information entropy represents the randomness of the probability distribution of word elements output by a large language model at the current generation position. Higher information entropy indicates a more dispersed prediction of the next word element. Information entropy is calculated by taking the negative logarithm of the probability of each word element in the original output probability distribution and then weighting these values ​​across all word elements. After obtaining the information entropy, the logarithmic bias value can be dynamically adjusted using a linear relationship. When the information entropy is high, a larger logarithmic bias value is applied, thus increasing the relative probability of words in the target word element set when sampling and controlling candidate words within the target word element set. When the information entropy is low, a smaller logarithmic bias value is applied, avoiding excessive intervention in the model's original predictions and resulting in more precise sampling and control.

[0043] In one possible implementation, based on a preset hash algorithm, the current context, and the target key, the hash value corresponding to the current word in the initial message is obtained. Then, based on the hash value, candidate words in the vocabulary corresponding to the target large language model are divided to obtain at least two candidate word sets. This includes: obtaining at least a portion of the word sequences before the current generation position and concatenating these at least a portion of the word sequences with the target key to obtain hash input data; obtaining the hash value corresponding to the current generation position based on the preset hash algorithm and the hash input data; determining the candidate word set identifier corresponding to each candidate word based on the mapping relationship between the hash value and each candidate word in the vocabulary; and dividing the candidate words in the vocabulary based on the candidate word set identifier to obtain at least two candidate word sets.

[0044] Specifically, before generating the current word element, at least some words in the current word element sequence can be obtained, and these words can be concatenated with the target key in sequence to form a longer data string, resulting in hash input data. Then, according to a preset hash algorithm, such as SHA-256, the hash input data is processed to obtain the corresponding hash value, which is a unique identifier of the current generation position. Then, according to a preset mapping relationship, which can be a preset word element lookup table, the association between the hash value and each candidate word element in the vocabulary is determined, thereby determining which candidate word element set the current word element belongs to. Then, all candidate word elements in the entire vocabulary are grouped, and word elements with the same set identifier are assigned to the same candidate word element set. For example, if the set identifiers are 0 and 1, then all word elements with an identifier of 0 form the first set, and all word elements with an identifier of 1 form the second set.

[0045] In one possible implementation, the number of candidate lexical units is 2. n Where n is an integer greater than or equal to 1.

[0046] This application discloses an information processing method based on a large language model, comprising: acquiring an initial message to be processed, a target key, and a current context; inputting the initial message to be processed into a target large language model, and during the generation of target text lexical units, obtaining the hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key; dividing candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein the candidate lexical units in each candidate lexical unit set are different; determining the target lexical unit set from the candidate lexical unit set according to the current lexical unit, and sampling control of the candidate lexical units in the target lexical unit set based on the original probability distribution of the current lexical unit to obtain the target lexical unit at the current generation position of the target text corresponding to the current lexical unit; and obtaining the target text based on several target lexical units. In this method, by embedding information using a hash algorithm and sampling control mechanism during the text generation process, high-capacity and reliable information embedding can be achieved in the generated text, while improving the naturalness of the output text and the unbiasedness of the statistical distribution.

[0047] Figure 2 This is a flowchart illustrating an information receiving method based on a large language model, as shown in an embodiment of this application.

[0048] See Figure 2 A method for receiving information based on a large language model, comprising: S210: Receive target text.

[0049] S220: Input the target text into the target large language model, and use the target large language model to perform word segmentation on the target text to obtain the target word sequence.

[0050] S230: Based on the target key, the preset hash algorithm, and the current word sequence corresponding to the current word in the target word sequence, generate the hash value corresponding to the current word sequence.

[0051] S240: Based on the hash value, divide the candidate lexical units in the vocabulary corresponding to the target large language model to obtain at least two candidate lexical unit sets.

[0052] S250: Compare the current word element with the set of candidate word elements to determine the target set of candidate word elements corresponding to the current word element.

[0053] S260: Based on the target candidate word set, obtain several target words, and based on the target words, obtain the initial message.

[0054] Specifically, after receiving the target text "Given the complexity of recent market trends, we have observed volatility in several key indicators, requiring us to closely monitor and adjust our strategies accordingly," User B first inputs the target text into a target large language model. This model then segments the text into a sequence of target words, such as ["given", "recently", "market", ...]. For the first word "given" in the sequence, the large language model generates a hash value corresponding to the current word sequence based on a preset hash algorithm, the target key, and the prefix of that word. Based on this hash value, the model then divides the candidate words in the vocabulary corresponding to the target large language model, obtaining at least two sets of candidate words. For example, if the candidate word set is C0 and C1, the word "given" is compared with these two candidate word sets to determine which set it belongs to. For example, if "given" belongs to C0, it is decoded as "0"; if "given" belongs to C1, it is decoded as "1". The above process is repeated for each word. Finally, based on all the decoded target words, the binary message corresponding to the initial message is obtained and converted into the original information "The emergency meeting is scheduled for 10:00 AM tomorrow". In this way, the receiver only needs to compare the hash partition and set with the sender to decode the message sent by the sender. The decoding accuracy is high, and end-to-end covert information transmission is realized. This effectively solves the key problem of the lack of a complete communication protocol stack in the existing technology.

[0055] Corresponding to the aforementioned application function implementation method embodiments, this application also provides an information processing device, electronic device, and corresponding embodiments based on a large language model.

[0056] Figure 3 This is a schematic diagram of the structure of an information processing device based on a large language model, as shown in an embodiment of this application.

[0057] See Figure 3 An information processing device 300 based on a large language model includes: The acquisition module 310 is used to acquire the initial message to be processed, the target key, and the current context; The processing module 320 is used to input the initial message to be processed into the target large language model, and in the process of generating the target text's lexical units, obtain the hash value corresponding to the current lexical unit in the initial message according to the preset hash algorithm, the current context and the target key, and divide the candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein, the candidate lexical units in each candidate lexical unit set are different; The determination module 330 is used to determine the target word set from the candidate word set based on the current word, and to perform sampling control on the candidate word in the target word set based on the original probability distribution of the current word to obtain the target word at the current generation position of the target text corresponding to the current word; The generation module 340 is used to obtain the target text based on several target words.

[0058] In one possible implementation, the processing module 320 is further configured to perform message preprocessing on the initial message to obtain the encoded sequence to be embedded; wherein, the message preprocessing includes at least one of forward error correction coding, synchronization sequence insertion, and verification information appending; and to determine the target word set corresponding to the target encoded content based on the target encoded content corresponding to the current generation position in the encoded sequence.

[0059] In one possible implementation, the determining module 330 is further configured to obtain the KL divergence between the context probability distribution corresponding to the current context and the original output probability distribution of the large language model at the current generation position; if the KL divergence is not less than a preset threshold, sampling is performed based on the original output probability distribution to generate the target word at the current generation position; if the KL divergence is less than the preset threshold, a logarithmic bias is applied to the original output probability of the candidate word in the target word set, and the probability distribution after applying the bias is normalized to obtain the sampled probability distribution, so as to generate the target word at the current generation position based on the sampled probability distribution.

[0060] In one possible implementation, the determining module 330 is further configured to obtain the information entropy of the original output probability distribution; and dynamically adjust the logarithmic bias value according to the information entropy; wherein, the higher the information entropy, the larger the value of the logarithmic bias value.

[0061] In one possible implementation, the processing module 320 is further configured to obtain at least a portion of the lexical sequences preceding the current generation position, and concatenate the at least a portion of the lexical sequences with the target key to obtain hash input data; based on a preset hash algorithm and the hash input data, obtain the hash value corresponding to the current generation position; determine the candidate lexical set identifier corresponding to each candidate lexical according to the mapping relationship between the hash value and each candidate lexical in the vocabulary; and divide the candidate lexical in the vocabulary based on each candidate lexical set identifier to obtain at least two candidate lexical sets.

[0062] In one possible implementation, the processing module 320 is further configured to have a candidate lexical set of 2. n Where n is an integer greater than or equal to 1.

[0063] This application discloses an information processing device based on a large language model, comprising: an acquisition module for acquiring an initial message to be processed, a target key, and a current context; a processing module for inputting the initial message to be processed into a target large language model, and during the generation of target text lexical units, obtaining the hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key, and dividing the candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein the candidate lexical units in each candidate lexical unit set are different; a determination module for determining the target lexical unit set from the candidate lexical unit set according to the current lexical unit, and sampling control of the candidate lexical units in the target lexical unit set based on the original probability distribution of the current lexical unit to obtain the target lexical unit at the current generation position of the target text corresponding to the current lexical unit; and a generation module for obtaining the target text according to several target lexical units. In this method, by embedding information using a hash algorithm and sampling control mechanism during the text generation process, high-capacity and reliable information embedding can be achieved in the generated text, while improving the naturalness of the output text and the unbiasedness of the statistical distribution.

[0064] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0065] This application also provides an electronic device. Figure 4 This is a schematic diagram of the hardware structure of an embodiment of the electronic device of this application. The electronic device includes a memory 420 and at least one processor 410. The memory 420 is electrically connected to the at least one processor 410. The memory 420 stores instructions. The at least one processor 410 calls the instructions in the memory 420, causing the electronic device to execute the information processing method based on a large language model and the information receiving method based on a large language model according to any of the foregoing embodiments of this application.

[0066] Specifically, the processor 410 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0067] Memory 420 may include a large-capacity memory 420 for data or instructions. For example, and not limitingly, memory 420 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 420 may include removable or non-removable (or fixed) media. Where appropriate, memory 420 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 420 is non-volatile solid-state memory. In a particular embodiment, memory 420 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0068] In one example, the control device may also include a communication interface 430 and a bus 440. The processor 410, memory 420, and communication interface 430 are connected via the bus 440 and communicate with each other.

[0069] The communication interface 430 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0070] Bus 440 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 440 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0071] Furthermore, in conjunction with the information processing method and information receiving method based on a large language model in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores executable code, which, when executed by a processor, implements any one of the information processing method and information receiving method based on a large language model in the above embodiments.

[0072] This application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0073] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0074] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0075] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An information processing method based on a large language model, characterized in that, include: Retrieve the initial message to be processed, the target key, and the current context; The initial message to be processed is input into the target large language model. During the generation of the target text's lexical units, the hash value corresponding to the current lexical unit in the initial message is obtained according to the preset hash algorithm, the current context, and the target key. Based on the hash value, the candidate lexical units in the vocabulary corresponding to the target large language model are divided to obtain at least two candidate lexical unit sets. The candidate lexical units in each candidate lexical unit set are different. Based on the current word element, a target word element set is determined from the candidate word element set, and based on the original probability distribution of the current word element, the candidate word elements in the target word element set are sampled and controlled to obtain the target word element at the current generation position of the target text corresponding to the current word element; The target text is obtained based on several target lexical units.

2. The method according to claim 1, characterized in that, Before inputting the initial message to be processed into the target large language model, the process also includes: The initial message is preprocessed to obtain the encoded sequence to be embedded; wherein the message preprocessing includes at least one of forward error correction coding, synchronization sequence insertion, and check information appending. The step of determining the target word set from the candidate word set based on the current word element includes: Based on the target encoded content corresponding to the current generation position in the encoded sequence, determine the target word set corresponding to the target encoded content.

3. The method according to claim 1, characterized in that, The step of sampling and controlling candidate words in the target word set based on the original probability distribution of the current word element to obtain the target word element at the current generation position of the target text corresponding to the current word element includes: Obtain the KL divergence between the context probability distribution corresponding to the current context and the original output probability distribution of the large language model at the current generation position; If the KL divergence is not less than a preset threshold, then sampling is performed based on the original output probability distribution to generate the target word at the current generation position; If the KL divergence is less than the preset threshold, a logarithmic bias is applied to the original output probabilities of candidate words in the target word set, and the probability distribution after applying the bias is normalized to obtain a sampled probability distribution, so as to generate the target word at the current generation position based on the sampled probability distribution.

4. The method according to claim 1, characterized in that, Applying a logarithmic bias to the original output probabilities of candidate lexical units within the target lexical set includes: Obtain the information entropy of the original output probability distribution; The logarithmic bias value is dynamically adjusted based on the information entropy; wherein, the higher the information entropy, the larger the value of the logarithmic bias value.

5. The method according to claim 1, characterized in that, The method involves obtaining the hash value corresponding to the current word in the initial message based on a preset hash algorithm, the current context, and the target key, and then dividing the candidate words in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate word sets, including: Obtain at least a portion of the lexical sequence preceding the current generation position, and concatenate the at least a portion of the lexical sequence with the target key to obtain hash input data; Based on the preset hash algorithm and the hash input data, the hash value corresponding to the current generation position is obtained; Based on the mapping relationship between the hash value and each candidate lexical in the vocabulary, the candidate lexical set identifier corresponding to each candidate lexical is determined; Based on the identifiers of each candidate lexical set, the candidate lexical units in the vocabulary are divided to obtain at least two candidate lexical sets.

6. The method according to claim 5, characterized in that, Also includes: The number of candidate lexical units is 2. n Where n is an integer greater than or equal to 1.

7. An information receiving method based on a large language model, characterized in that, include: Receive the target text; The target text is input into the target large language model, and the target text is segmented by the target large language model to obtain the target word sequence; Based on the target key, the preset hash algorithm, and the current word sequence corresponding to the current word in the target word sequence, a hash value corresponding to the current word sequence is generated; Based on the hash value, the candidate lexical units in the vocabulary corresponding to the target large language model are divided to obtain at least two candidate lexical unit sets; The current word element is compared with the set of candidate word elements to determine the target set of candidate word elements corresponding to the current word element; Based on the target candidate word set, several target words are obtained, and based on the target words, an initial message is obtained.

8. An information processing device based on a large language model, characterized in that, include: The acquisition module is used to acquire the initial message to be processed, the target key, and the current context; The processing module is used to input the initial message to be processed into the target large language model, and during the process of generating the target text's lexical units, obtain the hash value corresponding to the current lexical unit in the initial message according to a preset hash algorithm, the current context, and the target key, and divide the candidate lexical units in the vocabulary corresponding to the target large language model according to the hash value to obtain at least two candidate lexical unit sets; wherein, the candidate lexical units in each candidate lexical unit set are different; The determination module is used to determine the target word set from the candidate word set based on the current word element, and to perform sampling control on the candidate word elements in the target word set based on the original probability distribution of the current word element, so as to obtain the target word element at the current generation position of the target text corresponding to the current word element; The generation module is used to obtain the target text based on several target lexical units.

9. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-7.

Images