Text generation method and device, equipment, storage medium and program product

Abstract

The invention discloses a text generation method and device, equipment, a storage medium and a program product, and belongs to the technical field of artificial intelligence. The method comprises the steps of dividing elements in a target word list into a plurality of element sets based on model identity information of a large language model; based on the user identity coding information, aggregating and rearranging the plurality of element sets to obtain at least one element group; based on the output probability of each element in the target word list, determining a grouping information entropy of at least one element group; and determining output elements in the target word list based on the grouping information entropy of the at least one element group and the user identity coding information, and generating a target text according to the output elements. In this way, the source or ownership of the generated text content can be identified, and the convenience of text content detection is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a text generation method, device, equipment, storage medium and program product. Background Art

[0002] With the rise of AI-generated content (AIGC) tools such as ChatGPT, the threshold for digital content creation has been significantly lowered, and a large amount of digital content can be generated in a short period of time. However, the generated content is less controllable and may contain sensitive, false or discriminatory content. Therefore, AIGC has gradually become the focus of Internet content blocking and review, and it has become an industry consensus to identify AIGC through technologies such as digital watermarking.

[0003] In the related text generation methods, the probability distribution of the next token is predicted based on the prompt word and the generated text; the random number seed is set by calculating the hash value of the last token; the vocabulary elements are randomly divided into green vocabulary and red vocabulary according to the random number seed; when generating text, all tokens are selected from the green vocabulary, and the vocabulary elements are randomly generated each time. Although this method can determine whether the generated text is generated by artificial intelligence, it lacks ownership identification information, and watermark detection mostly relies on large language model providers, and the convenience of text content detection is insufficient. Summary of the invention

[0004] The embodiments of the present application provide a text generation method, apparatus, device, storage medium and program product to at least solve the problem that existing text content generation methods lack ownership identification information and are not convenient for text content detection.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a text generation method, comprising: based on model identity information of a large language model, dividing each element in a target vocabulary into multiple element sets; the target vocabulary is a vocabulary used by the large language model when processing a text generation task; based on user identity coding information, aggregating and rearranging the multiple element sets to obtain at least one element group; based on the output probability of each element in the target vocabulary, determining the grouping information entropy of the at least one element group; the output probability is used to indicate the possibility of the large language model selecting each element at the next output; based on the grouping information entropy of the at least one element group and the user identity coding information, determining the output element in the target vocabulary, and generating a target text according to the output element.

[0006] In a second aspect, an embodiment of the present application provides a text generation device, comprising: an element division module, for dividing each element in a target vocabulary into multiple element sets based on model identity information of a large language model; the target vocabulary is a vocabulary used by the large language model when processing text generation tasks; a group acquisition module, for aggregating and rearranging the multiple element sets based on user identity coding information to obtain at least one element group; an information entropy determination module, for determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the output probability is used to indicate the possibility of the large language model selecting each element at the next output; a text generation module, for determining the output elements in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity coding information, and generating a target text according to the output elements.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect above are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the method described in the first aspect above.

[0010] In the embodiment of the present application, based on the model identity information of the large language model, each element in the target vocabulary is divided into multiple element sets; based on the user identity coding information, multiple element sets are aggregated and rearranged to obtain at least one element group; based on the output probability of each element in the target vocabulary, the grouping information entropy of at least one element group is determined; based on the grouping information entropy of at least one element group and the user identity coding information, the output element in the target vocabulary is determined, and the target text is generated according to the output element. In this way, the group selection of each element in the vocabulary is controlled by the user identity coding information, and then the user digital identity information is embedded to facilitate the identification of the source or ownership of the text content, thereby improving the convenience of text content detection.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0013] Figure 1 One of the example diagrams of the digital watermarking method based on vocabulary token element grouping is shown; Figure 2 The second example diagram shows a digital watermarking method based on word list token element grouping; Figure 3 A schematic diagram of a flow chart of a text generation method provided in an embodiment of the present application is shown; Figure 4 Another schematic diagram of a text generation method provided in an embodiment of the present application is shown; Figure 5 A schematic diagram of the structure of a text generation device provided in an embodiment of the present application is shown; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0015] Digital watermarking is a technology that hides information in a carrier. It can not only reduce the risk of "deep fakes" and facilitate subsequent traceability, but also protect the ownership of high-quality digital content and promote the creation and dissemination of digital cultural industries. Digital watermarking is a technology that embeds invisible information into digital content such as text, audio, images or videos. By embedding digital watermarks in AIGC content, it is possible to determine whether the content has been tampered with or AI-generated. When the watermark is associated with a specific identifier, it can also be used for traceability and authenticity verification.

[0016] Digital watermarking technology is widely used in image and video scenarios, including DwtDctSvd, RivaGAN, StegaStamp and self-supervised learning algorithms, but there is little research on text content. Therefore, in LLM (Large Language Model), such as ChatGPT, LLaMA, etc., how to design a high-efficiency, low-complexity digital watermarking algorithm has become a problem that needs to be solved urgently.

[0017] The related scheme proposes a digital watermark method based on the grouping of vocabulary token elements (Token Grouping for Hidden Robustness and Longevity, hereinafter referred to as TGHRL method). Among them, token represents the smallest unit of vocabulary segmentation, such as a single word. When the method outputs the tth token, it sets the random number seed according to the hash value of the previous token element, and divides all vocabulary elements into two groups, red and green. Finally, sampling is performed in the green group to ensure that all elements come from the green group, that is, they have implicit digital watermarks. The specific steps include the following: Initialization input: prompt word, length N , denoted as s -N , s -N+1 , ... , s -1 ,make t =0; Step 1: Based on the prompt word and the generated text content s -N , s -N+1 , ... , s t-1 , output the probability p0, p1, ..., p of each element in the vocabulary corresponding to the next token V-1 ,in, V is the number of elements in the LLM model vocabulary; Step 2: According to t -1 token to calculate the hash value, i.e. hash( s t-1 ), to set the random number seed; Step 3: According to the random number seed, all vocabulary elements are randomly divided into two groups: green G and red R; Step 4: From the green G group, sample and output according to the preset rules (such as maximum probability) s t ; Step 5: Loop iteration, let t = t +1, return to Step 1, until t ≥T, where T is the length of the generated text.

[0018] like Figure 1As shown in the figure, when LLM generates the following content "I saw a cat on a" based on the prompt word, and then outputs the next token, it will calculate the hash value based on "a" and use it to generate a random number seed, and then divide all the elements in the vocabulary into two groups, red and green. Finally, only the element with the highest probability "carpet" is sampled from the green group.

[0019] In the above TGHRL method, when generating text content, token elements are selected from the green group, and all vocabulary elements are randomly generated each time, which may result in the inability to select the correct element, especially when it comes to factual statements such as fixed entities or time, thus affecting the content quality. The following example is used to illustrate. Figure 2 As shown, assuming that the LLM has generated content "Today is September 21, Monday", according to the TGHRL algorithm rules, a random number seed is generated according to the "period" and the tokens are grouped. In this case, the correct value "four" may be placed in the red group, so that only the "one" with the highest probability can be selected in the green group, that is, the factual error of "Today is September 21, Monday" occurs, resulting in a decrease in the quality of AIGC generation.

[0020] Although the quality of AIGC generation can be improved to a certain extent through soft segmentation, group segmentation is required for each token generation, and the ratio is fixed, that is, the number of elements in the green group and the red group remains unchanged, and token sampling is always performed in the green group. Therefore, although this method can determine whether the generated text is generated by artificial intelligence, it lacks ownership identification information, and watermark detection mostly relies on large language model providers, and the convenience of text content detection is insufficient.

[0021] In response to the above-mentioned problems of the large language model in the text generation process, an embodiment of the present application provides a text generation method, which controls the group selection of each element in the vocabulary through user identity coding information, and then embeds the user's digital identity information to facilitate identification of the source or ownership of the text content, thereby improving the convenience of text content detection.

[0022] Figure 3A flow chart of a text generation method provided in an embodiment of the present application is shown. The execution subject of the method may be a terminal device or a server, wherein the terminal device may be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device may be a terminal device used by a user. The server may be an independent server, or a server cluster composed of multiple servers, and the server may be a background server of a certain business, or a background server of a certain platform or application (for example, a digital copyright management platform, a text generation system, a content recommendation system, etc.). In the embodiment of the present application, the execution subject is taken as an example to illustrate. For the case of a terminal device, it can be processed according to the following relevant content, which will not be repeated here. As shown in the figure, the text generation method 300 may include the following steps: Step 301: Based on the model identity information of the large language model, each element in the target vocabulary is divided into multiple element sets.

[0023] The target vocabulary is the vocabulary used by the large language model when processing text generation tasks, and the model identity information includes version configuration information, organization code information, etc.

[0024] In a specific implementation, the target vocabulary used by the large language model when processing text generation tasks is obtained, and the elements in the target vocabulary are divided into multiple element sets according to the model identity information such as the large language model version configuration and the organization code. For example, they can be classified according to semantic categories, word frequencies, etc.; or the model identity information of the large language model can be hashed to obtain a model identity hash value, and the model identity hash value is used as a random number seed to classify the elements in the target vocabulary according to the random number seed. In this way, the model identity information can be presented in a hidden manner to achieve the embedding of the model identity information.

[0025] Step 302: Based on the user identity coding information, the multiple element sets are aggregated and rearranged to obtain at least one element group.

[0026] The user identity coding information is obtained by encoding the user digital identity information.

[0027] In a specific implementation, the user's digital identity information such as the user ID number and mobile phone number can be obtained, and the user's digital identity information is encoded to obtain the user's identity encoding information, and then the multiple element sets obtained in the above step 101 are aggregated and rearranged according to the user's identity encoding information to obtain at least one element group. In this way, the user's digital identity information can be presented in a hidden manner to achieve the embedding of the user's digital identity information. Step 303: Determine the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the output probability is used to indicate the possibility of the large language model selecting each element in the next output.

[0028] In a specific implementation, the probability of the large language model selecting each element in the target vocabulary at the next output, that is, the output probability of each element, can be obtained in advance, and then the grouping information entropy of at least one element group can be determined according to the output probability of each element. Optionally, for each element group, the group probability corresponding to the element group can be determined according to the output probability of the elements in the element group, and the grouping information entropy of the element group can be determined according to the group probability. In this way, the diversity of the text content in each element group can be measured by the grouping information entropy of the element group.

[0029] Step 304: Based on the grouping information entropy of the at least one element group and the user identity encoding information, determine the output element in the target vocabulary, and generate the target text according to the output element.

[0030] In a specific implementation, a target element group can be selected from at least one element group based on the grouping information entropy of at least one element group and the user identity coding information, and then an output element can be determined from the target element group, and a target text can be generated based on the output element. In this way, the generated target text not only meets user needs, but also has a certain information diversity. At the same time, it can effectively identify the source and ownership of text generation, thereby improving the convenience of text content detection.

[0031] In a possible implementation, in the above step 301, based on the model identity information of the large language model, each element in the target vocabulary is divided into an element set, including: The model identity information of the large language model is hashed to obtain a model identity hash value; the model identity hash value is determined as a random number seed, and each element in the target vocabulary is divided into element sets according to the random number seed.

[0032] In an exemplary embodiment, the model identity hash value is calculated based on the version configuration information of the large language model and the model identity information such as the organization code information. h ,Right now ; in, version Configuration information for the version. org_code Institution code information, f 1(·) is the joint calculation function, hash (·) is the hash calculation function.

[0033] Based on the model identity hash h, set the random number seed, and separate the elements in the target vocabulary of the large language model according to the random number seed to obtain M sets of non-overlapping elements, expressed as v m ( ).

[0034] In a possible implementation, in the above step 302, the plurality of element sets are aggregated and rearranged based on the user identity code information to obtain at least one element group, including: The user digital identity information is encoded to obtain a binary bit sequence; the binary bit sequence is interleaved according to a preset interleaving rule to obtain user identity coding information; and the multiple element sets are aggregated and rearranged according to the user identity coding information, the target number of element groups and a preset rearrangement function to obtain at least one element group.

[0035] In an exemplary embodiment, in order to clarify the ownership of the creator of the generated digital content, user-level digital watermark embedding is adopted to extract the user's corresponding user digital identity information DID; the user digital identity information is encoded to obtain a binary bit sequence, and the binary bit sequence is interleaved according to a preset interleaving rule to improve the robustness of digital watermark detection, that is, ; ; in, code_config To encode configuration information, Encode (·) is the encoding function, b enc is the binary bit sequence obtained after encoding, Interleave(·) is the interleaving function, b The user identity coding information is the user identity coding information after the user digital identity information is encoded and interleaved.

[0036] Taking the (6, 3) linear block code as an example, its corresponding generator matrix is G for: ; Assuming that the binary sequence corresponding to the user's digital identity is [1,1,0], the encoded binary bit sequence is: ; It should be noted that if the bit length corresponding to the user digital identity exceeds 3 or is not a multiple of 3, it can be segmented or padded with zeros, then encoded and concatenated in sequence. Finally, the binary bit sequence is interleaved, that is, the bit order is shuffled according to the preset rules.

[0037] Encode information based on the encoded user identityb Aggregate and rearrange the multiple element sets obtained by the initial segmentation to obtain L element groups, i.e. ,L); in, f 2(·) is a rearrangement function that transforms the element set v m Belong to the element group l . At the same time, the number of elements in each group is as equal as possible, and satisfies ; in, k is a positive integer, k The larger the value, the more groups are divided, and the more information the digital watermark carries, but the quality of the generated content text will also decrease. The value is usually not greater than 3. At the same time, ensure that the length of the user identity encoding information 𝒃 is an integer multiple of 𝑘, otherwise the tail is padded with zeros.

[0038] Assume that the initial segmentation is M = a set of 12 elements, k =2, it needs to be re-divided into L = 4 element groups, then 3 element sets can be grouped together to obtain at least one element group. For example, the first 3 element sets v 1, v 2, v 3 is the element group l =1, followed by a set of 3 elements v 4. v 5. v 6 is the element group l =2, the last 3 elements are element groups l =4.

[0039] In a possible implementation, in the above step 303, determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary includes: According to the output probability of each element in the target vocabulary, the group probability of the at least one element group is determined; according to the group probability of the at least one element group, the grouping information entropy of the at least one element group is determined.

[0040] In the embodiment of the present application, the output probability of each element in the target vocabulary can be , determine the group probability of at least one element group, for example, sum the output probabilities of each element in the element group to obtain the group probability of the element group P l : ; in, J l is the set of group element serial numbers, , L is the number of element groups.

[0041] Then, according to the group probability of at least one element group, the group information entropy of at least one element group is determined. Specifically, the group information entropy of the element group can be determined according to the following formula: H : ; In a possible implementation, in the above step 304, determining the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity encoding information includes: A target element group is selected from the at least one element group according to the parameter information of the element group and the user identity code information; and an output element is selected from the target element group according to the grouping information entropy of the target element group.

[0042] In an embodiment of the present application, firstly, a target element group is selected from L element groups according to parameter information of the element group, such as the number of element groups and user identity coding information, and then an output element is selected from the target element group according to the grouping information entropy of the target element group.

[0043] The step of selecting a target element group from the at least one element group according to the parameter information of the element group and the user identity code information includes: Determine the interception position of the target sub-bit string according to the parameter information of the element group; extract the target sub-bit string in the user identity coding information according to the interception position of the target sub-bit string; perform numerical conversion on the target sub-bit string to obtain a group number; and select a target element group from the at least one element group according to the group number.

[0044] In an exemplary embodiment, the group number of the current output element is calculated ,Right now: ; ; Among them, mod(·) is the remainder function, bin2dec ( b , i , k ) means to intercept the binary bit sequence b Middle To sub-bit strings and convert them to decimal integers. For example, the binary bit sequence b = [0,0,1,1,0,1,1,0], i , k are 1 and 2 respectively, then P =8, intercept the 3rd to 4th sub-bit string [1, 1], and further convert it into a decimal integer 3. At this time, .

[0045] In a possible implementation, the selecting the output element from the target element group according to the grouping information entropy of the target element group includes: A target element set is selected from the target element group according to the grouping information entropy of the target element group; and elements in the target element set whose output probability meets a preset probability threshold are determined as output elements.

[0046] In a specific implementation, the grouping information entropy of the target element group can be compared with a preset threshold, and a target element set can be selected from the target element group based on the comparison result, and the elements in the target element set whose output probability meets the preset probability threshold can be determined as output elements. For example, the element with the largest output probability in the target element set can be determined as the output element. The step of selecting a target element set from the target element group according to the grouping information entropy of the target element group includes: Acquire a first threshold value and a second threshold value, wherein the first threshold value is less than the second threshold value; When the group information entropy of the target element group is less than or equal to the first threshold, selecting a set of elements whose set probability meets a first preset condition from the target element group as the target element set; When the group information entropy of the target element group is greater than the first threshold value and less than or equal to the second threshold value, elements whose distances are less than a preset distance threshold and whose set probability satisfies the second preset condition element set are selected from the target element group and combined as the target element set.

[0047] In an exemplary embodiment, if the group information entropy of the target element group is less than or equal to the first threshold ,Right now , then select the element set whose set probability meets the first preset condition from the target element group as the target element set, for example, select the target element set group with the maximum probability: ; If the group information entropy of the target element group is between the first threshold The second threshold ,Right now , then select the element with the maximum probability from the target element set with relatively small Hamming distance, and its group is: ; Among them, dist(·) represents the Hamming distance metric function, which converts the variable into a binary bit sequence and then calculates the Hamming distance between the two. Indicates the preset fault tolerance threshold.

[0048] Furthermore, from In the element groups, the corresponding vocabulary element is selected as the sample output according to the principle of maximizing the output probability. s t , and its corresponding serial number is: ; This determines the output elements.

[0049] In a possible implementation, in the above step 303, before determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary, the step further includes: The output probability of each element in the target vocabulary is determined according to the preset prompt words and the historical text generated by the large language model.

[0050] In the embodiment of the present application, the historical text can be generated according to the prompt word input by the user and the large language model, that is, s -N , s -N+1 , ... , s t-1 , output the output probability of each element of the vocabulary corresponding to the next token p v (1≤ v ≤ V ).

[0051] In an exemplary embodiment, if Figure 4 As shown, the above text generation method includes the following steps: Step 401: Parameter initialization: Get the prompt word input by the user of the large language model, the length is N, recorded as s -N , s -N+1 , ... , s -1 , user identity encoding information b , the bit length is recorded as P, the number of elements in the full vocabulary of the large language model V , element group parametersL and k , the element number set of each group is J l (1≤l≤L), the double threshold information entropy threshold is and ( > ), let t=0; Step 402: Element probability calculation: According to the user input prompt word and the historical text generated by the large language model, that is s -N , s -N+1 , ... , s t-1 , output the output probability of each element of the vocabulary corresponding to the next token p v (1≤ v ≤ V ); Step 403: Grouping information entropy: According to the element probability and element grouping results, calculate the grouping information entropy of each element group; Step 404: Element group selection: Determine the interception position of the target sub-bit string according to the parameter information of the element group; extract the target sub-bit string in the user identity code information according to the interception position of the target sub-bit string; perform numerical conversion on the target sub-bit string to obtain a group sequence number; select a target element group from at least one element group according to the group sequence number; Step 405: Element sampling output: The element with the highest output probability in the target element set is determined as the output element; Step 406: loop iteration to determine whether the text content has been output: Let t=t+1, and return to step 402 until all text contents are output.

[0052] In a possible implementation, in the above step 304, after generating the target text according to the target element, the step further includes: The text feature information of the target text is stored in the target blockchain, wherein the text feature information includes at least one of the following: model identity information, user digital identity information, text content information of the target text, and prompt words.

[0053] In the embodiment of the present application, blockchain evidence storage aims to realize on-chain evidence storage of key data of generated text digital content based on the technical advantages and characteristics of blockchain such as decentralization, immutability, and traceability, and provide on-chain verification capabilities to ensure the credibility of the digital watermark identification results of generated digital content. After generating the target text according to the target element, the text feature information of the target text is stored in the target blockchain. The text feature information includes at least one of the following: model identity information (such as large language model version information), user digital identity information (such as user digital identity), text content information of the target text (such as a hash summary of the generated text content), and prompt word Prompt. Exemplarily, the on-chain evidence data includes but is not limited to the following as shown in Table 1: Table 1. Data stored on the blockchain during the text generation process

[0054] The embodiment of the present application provides a text generation method, which divides each element in the target vocabulary into multiple element sets based on the model identity information of the large language model; aggregates and rearranges multiple element sets based on the user identity coding information to obtain at least one element group; determines the grouping information entropy of at least one element group based on the output probability of each element in the target vocabulary; determines the output element in the target vocabulary based on the grouping information entropy of at least one element group and the user identity coding information, and generates the target text according to the output element. In this way, the group selection of each element in the vocabulary is controlled by the user identity coding information, and then the user digital identity information is embedded to facilitate the identification of the source or ownership of the text content, thereby improving the convenience of text content detection.

[0055] In addition, by adopting the one-time segmentation and dynamic group selection method, users can flexibly modify the grouping parameters to control the number of vocabulary element groups. When sampling and outputting elements, the quality of generated text content and the robustness of digital watermarks are reasonably balanced based on information entropy and double threshold principles.

[0056] Figure 5 The text generation device provided in the embodiment of the present application is shown in the structural diagram. The text generation device can be implemented as follows: Figure 3 In the embodiment shown, all or part of the content, the text generation device 500 includes: An element division module 510 is used to divide each element in a target vocabulary into multiple element sets based on the model identity information of the large language model; the target vocabulary is a vocabulary used by the large language model when processing a text generation task; A group acquisition module 520, configured to aggregate and rearrange the plurality of element sets based on the user identity code information to obtain at least one element group; An information entropy determination module 530, configured to determine the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the output probability is used to indicate the possibility of the large language model selecting each element at the next output; The text generation module 540 is used to determine the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity encoding information, and generate the target text according to the output element.

[0057] In a possible implementation, the element segmentation module 510, when used to segment each element in the target vocabulary into element sets based on the model identity information of the large language model, is specifically used to: Perform hash calculation on the model identity information of the large language model to obtain the model identity hash value; The model identity hash value is determined as a random number seed, and each element in the target vocabulary is divided into element sets according to the random number seed.

[0058] In a possible implementation, the group acquisition module 520, when used to aggregate and rearrange the multiple element sets based on the user identity code information to obtain at least one element group, is specifically used to: Encode the user's digital identity information to obtain a binary bit sequence; According to a preset interleaving rule, the binary bit sequence is interleaved to obtain user identity coding information; According to the user identity code information, the target number of element groups and a preset rearrangement function, the multiple element sets are aggregated and rearranged to obtain at least one element group.

[0059] In a possible implementation, the information entropy determination module 530, when used to determine the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary, is specifically used to: Determining a group probability of the at least one element group according to the output probability of each element in the target vocabulary; The grouping information entropy of the at least one element group is determined according to the group probability of the at least one element group.

[0060] In a possible implementation, the text generation module 540, when used to determine the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity encoding information, is specifically used to: selecting a target element group from the at least one element group according to the parameter information of the element group and the user identity code information; An output element is selected from the target element group according to the grouping information entropy of the target element group.

[0061] The text generation module 540, when used to select a target element group from the at least one element group according to the parameter information of the element group and the user identity code information, is specifically used to: Determining a cut position of a target sub-bit string according to parameter information of the element group; extracting the target sub-bit string in the user identity encoding information according to the interception position of the target sub-bit string; Performing numerical conversion on the target sub-bit string to obtain a group sequence number; A target element group is selected from the at least one element group according to the group sequence number.

[0062] The text generation module 540, when used to select an output element from the target element group according to the grouping information entropy of the target element group, is specifically used to: selecting a target element set from the target element group according to the grouping information entropy of the target element group; The elements in the target element set whose output probability meets a preset probability threshold are determined as output elements.

[0063] The text generation module 540, when used to select a target element set from the target element group according to the grouping information entropy of the target element group, is specifically used to: Acquire a first threshold value and a second threshold value, wherein the first threshold value is less than the second threshold value; When the group information entropy of the target element group is less than or equal to the first threshold, selecting a set of elements whose set probability meets a first preset condition from the target element group as the target element set; When the group information entropy of the target element group is greater than the first threshold value and less than or equal to the second threshold value, elements whose distances are less than a preset distance threshold and whose set probability satisfies the second preset condition element set are selected from the target element group and combined as the target element set.

[0064] In a possible implementation, the information entropy determination module 530 is further configured to: The output probability of each element in the target vocabulary is determined according to the preset prompt words and the historical text generated by the large language model.

[0065] In a possible implementation, the text generation module 540 is further configured to: The text feature information of the target text is stored in the target blockchain, wherein the text feature information includes at least one of the following: model identity information, user digital identity information, text content information of the target text, and prompt words.

[0066] The embodiment of the present application provides a text generation device, including an element division module, a group acquisition module, an information entropy determination module and a text generation module; the element division module divides each element in the target vocabulary into multiple element sets based on the model identity information of the large language model; the group acquisition module aggregates and rearranges the multiple element sets based on the user identity coding information to obtain at least one element group; the information entropy determination module determines the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the text generation module determines the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity coding information, and generates the target text according to the output element. In this way, the group selection of each element in the vocabulary is controlled by the user identity coding information, and then the user digital identity information is embedded to facilitate the identification of the source or ownership of the text content, thereby improving the convenience of text content detection.

[0067] Figure 6 The hardware structure diagram of the electronic device provided by the embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device 600 includes a processor 610, and optionally, includes an internal bus 620, a network interface 630, and a memory 640. Among them, the memory 640 may include a memory 641, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory 642 (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device 600 may also include hardware required for other services.

[0068] The processor 610, the network interface 630 and the memory may be interconnected via an internal bus 620, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0069] The memory 640 stores programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions. The memory 640 may include a memory 641 and a non-volatile memory 642, and provide instructions and data to the processor 610.

[0070] The processor 610 reads the corresponding computer program from the non-volatile memory 642 into the memory and then runs it, forming a device for locating the target user at the logical level. The processor 610 executes the program stored in the memory and specifically performs: Figure 3 or Figure 4 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0071] The above application Figure 3 or Figure 4 The method disclosed in the illustrated embodiment can be applied to the processor 610 or implemented by the processor 610. The processor 610 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 610. The above processor 610 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0072] The computer device can also execute the methods described in the foregoing method embodiments, and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0073] Of course, in addition to software implementation, the electronic device 600 of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0074] The embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes Figure 3 or Figure 4 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0075] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0076] Furthermore, an embodiment of the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the following process is implemented: Figure 3 or Figure 4 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0077] The embodiments of the present application can be applied to various electronic device collaboration or interconnection scenarios, including: collaboration and interconnection between mobile phones and laptops / tablet computers; collaboration and interconnection between mobile terminals and smart TVs / displays; collaboration and interconnection between mobile phones or tablet computers and in-car entertainment systems; collaboration and interconnection between mobile terminals and smart conference systems, etc., so as to meet the needs of users in various scenarios such as smart homes, smart offices, and smart travel.

[0078] In short, the above is only a preferred embodiment of the present application and does not limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0079] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0080] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0082] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A text generation method, characterized in that: include: Based on the model identity information of the large language model, each element in the target vocabulary is divided into multiple element sets; The target vocabulary is a vocabulary used by the large language model when processing a text generation task; Based on the user identity code information, the plurality of element sets are aggregated and rearranged to obtain at least one element group; Determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the output probability is used to indicate the possibility of the large language model selecting each element at the next output; Based on the grouping information entropy of the at least one element group and the user identity encoding information, an output element in the target vocabulary is determined, and a target text is generated according to the output element.

2. The method according to claim 1, characterized in that: The model identity information based on the large language model divides each element in the target vocabulary into an element set, including: Perform hash calculation on the model identity information of the large language model to obtain the model identity hash value; The model identity hash value is determined as a random number seed, and each element in the target vocabulary is divided into element sets according to the random number seed.

3. The method according to claim 1, characterized in that The step of aggregating and rearranging the plurality of element sets based on the user identity coding information to obtain at least one element group includes: Encode the user's digital identity information to obtain a binary bit sequence; According to a preset interleaving rule, the binary bit sequence is interleaved to obtain user identity coding information; According to the user identity code information, the target number of element groups and a preset rearrangement function, the multiple element sets are aggregated and rearranged to obtain at least one element group.

4. The method according to claim 1, characterized in that: The determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary comprises: Determining a group probability of the at least one element group according to the output probability of each element in the target vocabulary; The grouping information entropy of the at least one element group is determined according to the group probability of the at least one element group.

5. The method according to claim 1, characterized in that The determining the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity encoding information comprises: selecting a target element group from the at least one element group according to the parameter information of the element group and the user identity code information; An output element is selected from the target element group according to the grouping information entropy of the target element group.

6. The method according to claim 5, characterized in that The step of selecting a target element group from the at least one element group according to the parameter information of the element group and the user identity code information includes: Determining a cut position of a target sub-bit string according to parameter information of the element group; extracting the target sub-bit string in the user identity encoding information according to the interception position of the target sub-bit string; Performing numerical conversion on the target sub-bit string to obtain a group sequence number; A target element group is selected from the at least one element group according to the group sequence number.

7. The method according to claim 5, characterized in that The step of selecting an output element from the target element group according to the grouping information entropy of the target element group comprises: selecting a target element set from the target element group according to the grouping information entropy of the target element group; The elements in the target element set whose output probability meets a preset probability threshold are determined as output elements.

8. The method according to claim 7, characterized in that The step of selecting a target element set from the target element group according to the grouping information entropy of the target element group includes: Acquire a first threshold value and a second threshold value, wherein the first threshold value is less than the second threshold value; When the group information entropy of the target element group is less than or equal to the first threshold, selecting a set of elements whose set probability meets a first preset condition from the target element group as the target element set; When the group information entropy of the target element group is greater than the first threshold value and less than or equal to the second threshold value, elements whose distances are less than a preset distance threshold and whose set probability satisfies the second preset condition element set are selected from the target element group and combined as the target element set.

9. The method according to any one of claims 1 to 8, characterized in that: Before determining the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary, the method further includes: The output probability of each element in the target vocabulary is determined according to the preset prompt words and the historical text generated by the large language model.

10. The method according to any one of claims 1 to 8, characterized in that: After generating the target text according to the target element, the method further includes: The text feature information of the target text is stored in the target blockchain, wherein the text feature information includes at least one of the following: model identity information, user digital identity information, text content information of the target text, and prompt words.

11. A text generation device, characterized in that: include: An element partitioning module is used to partition each element in the target vocabulary into multiple element sets based on the model identity information of the large language model; The target vocabulary is a vocabulary used by the large language model when processing a text generation task; A group acquisition module, configured to aggregate and rearrange the plurality of element sets based on the user identity code information to obtain at least one element group; An information entropy determination module, used to determine the grouping information entropy of the at least one element group based on the output probability of each element in the target vocabulary; the output probability is used to indicate the possibility of the large language model selecting each element at the next output; The text generation module is used to determine the output element in the target vocabulary based on the grouping information entropy of the at least one element group and the user identity encoding information, and generate the target text according to the output element.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to perform the steps of the method according to any one of claims 1 to 10.

Images