Text generation method and device, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a text generation method and device, electronic equipment and a readable storage medium. The method comprises the following steps: in response to a received text generation instruction, determining an initial reference text corresponding to the text generation instruction; generating a target statement according to the initial reference text and a first constraint condition; utilizing the target statement to construct prompt information, and determining at least one target reference text quoted by the target statement from the initial reference text according to the prompt information; and according to a second constraint condition, generating a reference mark number for the target reference text and the target statement, the reference mark number representing a reference relationship between the target reference text and the target statement, and taking the target statement with the reference mark number as a result text of the text generation instruction. The method can ensure that the generated reference mark number is more accurate. Meanwhile, it is guaranteed that only the reference mark numbers of the constraint range are generated, and the problem that in the prior art, the formats of the reference mark numbers are wrong and inconsistent is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a text generation method, apparatus, electronic device, and readable storage medium. Background Art

[0002] With the development of the field of large language models, more and more text tasks can be completed with the help of large language models to improve the processing efficiency of text tasks. For example, when a large language model generates text according to actual needs, the large language model can add citation labels after generating the text to clarify the citation source of the text, so that users can know the origin of the generated text, thereby improving the credibility of the generated text. However, in the prior art, when generating text with citation labels, there are problems of inconsistent citation label formats and incorrect citation relationships between the output labels and the cited documents. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a text generation method, apparatus, electronic device, and readable storage medium to solve the problems in the prior art of inconsistent citation label formats and incorrect citation relationships between the output labels and the cited documents.

[0004] In a first aspect of embodiments of the present application, a text generation method is provided, and the method includes:

[0005] In response to a received text generation instruction, determining an initial reference text corresponding to the text generation instruction; generating a target statement according to the initial reference text and a first constraint condition; constructing prompt information by using the target statement, and determining at least one target reference text cited by the target statement from the initial reference text according to the prompt information; generating citation labels for the target reference text and the target statement according to a second constraint condition, where the citation labels represent the citation relationship between the target reference text and the target statement, and taking the target statement with citation labels as the result text of the text generation instruction.

[0006] In a second aspect of embodiments of the present application, a text generation apparatus is provided, and the apparatus includes:

[0007] A first determination module, configured to determine an initial reference text corresponding to the text generation instruction in response to the received text generation instruction; a first generation module, configured to generate a target statement according to the initial reference text and a first constraint condition; a second determination module, configured to construct prompt information by using the target statement, and determine at least one target reference text cited by the target statement from the initial reference text according to the prompt information; a second generation module, configured to generate citation labels for the target reference text and the target statement according to a second constraint condition, where the citation labels represent the citation relationship between the target reference text and the target statement, and taking the target statement with citation labels as the result text of the text generation instruction.

[0008] In a third aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0009] In a fourth aspect of the embodiments of the present application, a readable storage medium is provided. The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0010] The beneficial effects of the embodiments of the present application compared with the prior art are as follows:

[0011] In the embodiments of the present application, in response to a received text generation instruction, an initial reference text corresponding to the text generation instruction is determined; a target statement is generated according to the initial reference text and a first constraint condition; a prompt message is constructed by using the target statement, and at least one target reference text cited by the target statement is determined from the initial reference text according to the prompt message; a reference label is generated for the target reference text and the target statement according to a second constraint condition, and the reference label represents the reference relationship between the target reference text and the target statement. The target statement with the reference label is used as the result text of the text generation instruction. It is ensured that every time the target statement appears, a reference label is generated for the target statement and the target reference text it cites, avoiding the influence of the context relationship on the generation of the reference label after all the text is generated, and ensuring that the generated reference label is more accurate. At the same time, it is ensured that the generated reference label conforms to the label generation range and label generation form constrained by the second constraint condition, avoiding the problem of unstable reference label format, easy occurrence of format errors or inconsistencies caused by the traditional method of letting the large language model generate the reference label of the text, and ensuring that only the reference labels within the constrained range are generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 is a schematic flowchart of a text generation method provided by an embodiment of the present application;

[0014] Figure 2 is a schematic structural diagram of a text generation device provided by an embodiment of the present application;

[0015] Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0016] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, techniques, etc. are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0017] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category and do not limit the number of objects. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0018] In addition, it should be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusively, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device including the elements.

[0019] In today's digital age, large language models have developed rapidly, and pre-trained models based on the Transformer architecture have occupied an important position in natural language generation tasks. They can quickly generate text and demonstrate powerful capabilities in many fields such as text summarization, machine translation, intelligent writing, etc., bringing many conveniences to people's work and life.

[0020] However, when these large language models are applied to practical scenarios such as knowledge Q&A and document generation, the credibility and verifiability of the generated content have become key issues that need to be urgently solved. After all, in practical applications, accurate and reliable information is crucial. To effectively solve this problem, adding citation marks to the generated content has become a common means. Through citation marks, the source or basis of the content can be clearly explained, thereby enhancing the credibility and verifiability of the content.

[0021] Currently, there are mainly two ways to implement citation marking: post - processing citation marking and Prompt - controlled citation generation. Post - processing citation marking analyzes the corresponding citation sources with the help of external tools after the content is generated, and then manually or automatically adds citation labels. However, this method has obvious drawbacks. The generation of citation labels is separated from the content generation, which easily leads to the inconsistency between the content and the labels. Moreover, the post - processing process requires additional time and computing resources, increasing the time cost and computational complexity. In some scenarios with high real - time requirements, it is difficult to meet the need for quick response.

[0022] Prompt - controlled citation generation, on the other hand, adds prompt words when inputting into the model to make the model directly output citation labels. However, this method also has defects. The format of the citation labels generated by the model is unstable, often showing formatting errors or inconsistencies. And the generation of citation labels highly depends on the Prompt prompt and is greatly affected by the context, making it difficult to ensure the accuracy of the labels, thus affecting the reliability and authority of the content. In view of this, the present application proposes a text generation method, device, electronic device, and readable storage medium to solve the problems existing in the prior art.

[0023] Next, a text generation method, device, electronic device, and readable storage medium according to an embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0024] Figure 1 is a schematic flowchart of a text generation method provided by an embodiment of the present application. As Figure 1 shown, the method includes:

[0025] S101, in response to a received text generation instruction, determine an initial reference text corresponding to the text generation instruction;

[0026] S102, generate a target statement according to the initial reference text and a first constraint condition;

[0027] S103, construct prompt information using the target statement, and determine at least one target reference text cited by the target statement from the initial reference text according to the prompt information;

[0028] S104, generate citation labels for the target reference text and the target statement according to a second constraint condition, where the citation labels represent the citation relationship between the target reference text and the target statement, and use the target statement with citation labels as the result text of the text generation instruction.

[0029] Specifically, in response to the received text generation instruction, the text generation instruction can be a text instruction, an image instruction, or a voice instruction. When the text generation instruction is received, the text included in the text generation instruction can be extracted. Based on this text, the initial reference text corresponding to the text can be determined, providing a basic data source for subsequent text generation and reference determination. Ensure that the subsequent target statement is generated based on the initial reference text to guarantee the accuracy and credibility of the target statement.

[0030] It should be noted that during the process of generating the target statement based on the initial reference text, the limitation of the first constraint condition needs to be satisfied. The first constraint condition represents the constraint condition that can determine whether the target statement has been generated, so as to determine whether the generated target statement meets the requirement for adding reference labels, and avoid the generated target statement affecting the determination of the subsequent target reference text due to excessive or insufficient content. In addition to using the first constraint condition to constrain the target statement, the generated statement can also be restricted and guided in terms of content, grammar, semantics, etc., to ensure that the generated target statement meets certain requirements, such as smooth sentences and reasonable logic.

[0031] Furthermore, use the generated target statement to construct a prompt message, and then screen out at least one target reference text cited by the target statement from the initial reference text according to the prompt message. For example, the target statement is "The new teaching method has significantly improved students' grades, which is consistent with Li Hua's research in 2022", and the initial reference text is multiple educational research reports and references. At this time, extract "Li Hua", "2022", "new teaching method", and "grade improvement" from the target statement as the prompt message. Search in the references. If there is "Li Hua, 2022. Research on the impact of the new teaching method on students' grades. Educational Exploration, 35(2), 25-32. It is found that this teaching method can effectively improve students' grades", then the target reference text is screened and determined. There can be multiple such target reference texts, and the specific quantity is not specifically limited in this embodiment. In this way, the knowledge source of the target statement is clarified, a clear connection is established between the generated target statement and the reference text, and it is ensured that the generated content is well-founded. At the same time, after the target statement is generated, a reference relationship can be established between the target statement and the target reference text, avoiding the problem of the separation of reference label generation and content generation in the prior art, reducing the computational complexity, and meeting the real-time requirement.

[0032] Further, reference labels are generated for the target reference text and the target statement according to the second constraint condition, where the second constraint condition represents the condition for restricting the output range and output form of the reference labels. Finally, the target statement with reference labels is output as the result text of the text generation instruction, completing the process of the entire text generation and adding reference marks, so that the output range and output form of the reference labels meet the set constraint conditions, avoiding problems such as unstable reference label formats, easy occurrence of format errors, and being greatly affected by the context.

[0033] It should be noted that the above example is described by generating a single target statement. If multiple target statements need to be generated according to the initial reference text, after adding reference marks to the previous target statement, continue to execute the generation steps of the next target statement. The specific implementation steps are not elaborated here.

[0034] In addition, it should also be noted that the form of the above reference labels is not limited to numerical labels, but can also be English labels, Roman numeral labels, etc. This embodiment does not specifically limit this. In addition, the position where the label is added to the target statement can be not only at the end, but also at the beginning, in the middle, or any position of the target statement. This embodiment also does not specifically limit this.

[0035] In some examples, if there are at least two target reference texts referred to by the target statement, after generating reference labels for the target reference text and the target statement according to the second constraint condition, it further includes: determining the correspondence between the target statement and the at least two target reference texts referred to, and allocating the same color for display to each of the target reference texts referred to and its corresponding part in the target statement according to the correspondence.

[0036] As an example, if the target statement simultaneously refers to multiple target reference texts, such as C, D, and E, then in the target statement, the part corresponding to C is marked in red, the part corresponding to D is marked in blue, and the part corresponding to E is marked in green; and for the target reference texts C, D, and E, they are also marked with the corresponding red, blue, and green colors respectively. Through this color identification method, it can present a distinct and intuitive visual display, enabling readers to quickly identify the reference relationships between the various parts of the target statement and the target reference texts.

[0037] In addition, the target text type of the target reference text can be determined, and then the target type identifier corresponding to the target text type can be determined from the preset text types and type identifiers, and the target type identifier can be added to the target statement and the target reference text cited by the target statement. For example, a book icon (target type identifier) is used to represent the target reference text from an academic paper (text type), and a web page icon (target type identifier) is used to represent the target reference text sourced from a web page (text type). In such a complex information environment, users can quickly distinguish the types of information sources through intuitive icons. At the same time, the credibility of the target statement can also be determined through the target type identifier. Since the authority of academic papers is generally higher than the text generation results obtained from web pages, users can also determine the credibility of the target statement through the target type identifier.

[0038] According to the technical solution provided by the embodiment of the present application, in response to the received text generation instruction, determine the initial reference text corresponding to the text generation instruction; generate a target statement according to the initial reference text and the first constraint condition; use the target statement to construct a prompt message, and determine at least one target reference text cited by the target statement from the initial reference text according to the prompt message; generate a reference label for the target reference text and the target statement according to the second constraint condition, where the reference label represents the citation relationship between the target reference text and the target statement, and use the target statement with the reference label as the result text of the text generation instruction. Ensure that every time a target statement appears, a reference label is generated for the target statement and the target reference text it cites, avoiding the influence on the generation of reference labels due to the context relationship after all the text is generated, and ensuring that the generated reference labels are more accurate. At the same time, ensure that the generated reference labels meet the label generation range and label generation form constrained by the second constraint condition, avoiding the problems of unstable reference label formats, easy occurrence of format errors or inconsistencies caused by the traditional method of letting the large language model generate reference labels for text, and ensuring that only reference labels within the constrained range are generated.

[0039] In some embodiments, generating a target statement according to the initial reference text and the first constraint condition includes: generating target content according to the initial reference text; using a target callback function to detect the target content. If the detection result indicates that the target content appears with a preset identifier, obtain the statement corresponding to the target content and use the statement as the target statement.

[0040] Specifically, generating target content according to the initial reference text can make the target content generated based on the initial reference text, ensuring the accuracy and credibility of the target content. The first constraint condition indicates that if it is detected that the generated content appears with a preset identifier, it can be considered that the target statement has been generated this time.

[0041] In some examples, during the process of generating the target content, a large language model can be used to generate it streamingly based on the initial reference text, and a target callback function can be utilized to detect the target content, ensuring that the large language model is aware of each generated character during the content generation process, and avoiding the situations of misidentifying the preset identifier and failing to identify the preset identifier. The specific implementation of the target callback function can be customized according to the type of the preset identifier and the detection requirements. For example, methods such as regular expressions and string matching can be used to implement it.

[0042] It can be understood that the preset identifier is pre-set and can determine the identifier of the target statement. For example, the pre-set identifiers are (full stop., question mark?, exclamation mark! etc.) or line breaks, etc. During the process of generating the target content, if the generated content is identified as a preset identifier such as (full stop., question mark?, exclamation mark! etc.) or a line break, etc., it is considered that the current target content has been generated. At this time, the statement corresponding to the preset identifier will be obtained and used as the target statement to improve the accuracy of statement recognition, ensuring that the finally obtained target statement not only contains the initial reference text but also meets the preset identifier, making the target statement more accurate and targeted.

[0043] According to the technical solution provided by the embodiment of the present application, the target content is generated according to the initial reference text; the target content is detected by using the target callback function. If the detection result indicates that the preset identifier appears in the target content, the statement corresponding to the target content is obtained and used as the target statement. It can improve the accuracy and credibility of the generated target statement, and at the same time improve the accuracy of statement recognition.

[0044] In some embodiments, generating reference labels for the target reference text and the target statement according to the second constraint condition includes: inputting the target reference text and the target statement into the large language model; obtaining the original logical values generated by the large language model, where the number of the original logical values is multiple, each original logical value corresponds to each label, and each original logical value represents the unnormalized score of each label; determining the target logical value according to the original logical values and the predefined element set; generating reference labels for the target reference text and the target statement according to the target logical value.

[0045] Specifically, in the traditional method of generating citation labels, generally after the text is generated, the large language model is used to generate citation labels for the text by constructing prompt information. The format of the citation labels generated in this way is unstable, and it is easy to have problems such as format errors or inconsistencies. At the same time, since the citation labels are generated after the text is generated, it is easily affected by the context of the generated text, and it is difficult to ensure the accuracy of the citation labels and the cited documents, resulting in the problem that the citation labels do not correspond to the cited documents. For example, according to the normal citation format, it should be marked as "[1]". By constructing prompt information, it may be wrongly marked as "(1)". And because the citation labels are generated after the text is generated, at this time the content and structure of the text have been determined, then when adding citation labels, it is very easy to be interfered by the context, resulting in the citation labels not matching the actually cited documents. For example, the place that should be normally marked as "[2]" is marked as "1.", in this way, the readability and traceability of the generated text are very poor.

[0046] It can be understood that in order to avoid the above problems, in this embodiment, after generating the target statement and the target reference text cited by the target statement, the target reference text and the target statement are input into the large language model, and the large language model is used to generate citation labels, so as to ensure that every time the target statement appears, citation labels are generated for the target statement and the target reference text it cites, avoiding the influence of the context on the generation of citation labels after all the text is generated.

[0047] In some examples, the target reference text and the target statement are input into the large language model to obtain the raw logits generated by the large language model. The number of raw logits is multiple, and each raw logit corresponds to each label, and each raw logit represents the unnormalized score of each label; for example, assuming that there are many different tokens in the vocabulary, each token corresponds to a label, then the model may generate the same number of raw logits as the number of tokens, and each raw logit corresponds to a token (label). These raw logits can be understood as the large language model's preliminary judgment on each token (label) as the next possible output, but at this time they have not been normalized and cannot be directly represented as probabilities, that is, it is uncertain which label the large model will output.

[0048] It can be understood that the pre-defined set of elements represents a pre-defined set of valid tokens (label set), where the pre-defined set of elements can pre-define the generation range and generation form of labels, etc. That is, the pre-defined set of elements can ensure the generation range and generation form of labels, ensuring that in the subsequent process of generating labels, only labels that conform to the range and form are generated. For example, it is defined to only generate reference labels of [1], [2], [3], and not generate reference labels of (1), (2), (3) and not generate labels outside the range of numbers 1, 2, 3; it should be noted that the number of elements in the element set can be determined according to the number of target reference texts or can be set by the user according to requirements, and this example does not specifically limit this. According to the original logical values and the pre-defined set of elements, the target logical values are determined; that is, according to the pre-set rules, the valid parts are selected from the numerous original logical values to obtain the target logical values. For example, the pre-defined set of elements may contain the token IDs corresponding to the labels "1", "2", "3". At this time, if the target logical values are to be determined, then all the original logical values are traversed, and the original logical values corresponding to the labels in the pre-defined set of elements are found and determined as the target logical values. Finally, according to the target logical values, reference labels are generated for at least one reference text and the target statement. This can ensure that the generated reference labels conform to the label generation range and label generation form defined in the pre-defined set of elements, avoiding the problems of unstable reference label formats, easy occurrence of format errors or inconsistencies caused by the traditional method of using construction prompt information to let the large language model generate text reference labels.

[0049] According to the technical solution provided by the embodiment of the present application, the target reference text and the target statement are input into the large language model; the original logical values generated by the large language model are obtained, where the number of original logical values is multiple, each original logical value corresponds to each label, and each original logical value represents the unnormalized score of each label; according to the original logical values and the pre-defined set of elements, the target logical values are determined; according to the target logical values, reference labels are generated for the target reference text and the target statement. This can ensure that the generated labels conform to the preset range and form, avoiding the problems of uncertainty in traditional label generation and inconsistent generation forms, and at the same time avoiding being affected by the context of the generated text during the process of generating labels.

[0050] In some embodiments, determining the target logical values according to the original logical values and the pre-defined set of elements includes: determining a first original logical value and a second original logical value according to the original logical values and the pre-defined set of elements; respectively masking the first original logical value and the second original logical value to obtain the target logical values.

[0051] Specifically, continuing with the above example, based on the original logical values and a predefined set of elements, determine the first original logical value and the second original logical value; if the predefined set of elements contains the token IDs corresponding to the labels "[1]", "[2]", and "[3]". At this time, the original logical values corresponding to the labels "[1]", "[2]", and "[3]" in the original logical values can be determined as the first original logical value, and the original logical values corresponding to the remaining labels other than "[1]", "[2]", and "[3]" are determined as the second original logical value. The obtained first original logical value and second original logical value are the generation probabilities before normalization for all labels.

[0052] It can be understood that after obtaining the first original logical value and the second original logical value, the large language model does not clearly distinguish between the first original logical value and the second original logical value during the processing, that is, it is unable to determine the special different meanings of the first original logical value and the second original logical value in the actual generation process, but only makes a formal division based on the predefined set of elements. Further, it is necessary to perform masking processing on the first original logical value and the second original logical value respectively. That is, by means of masking, it is determined whether to retain the generation probability of the first original logical value or the generation probability of the second original logical value.

[0053] In some examples, assume that the original logical values logits generated by the generation model at a certain step are L = [L1, L 2, ...L n , and the predefined set of elements is V = {1, 2, 3}, corresponding to the token IDs of the labels [1], [2], and [3] respectively. At this time, the first original logical value can be determined as L = [L1, L 2, L3], and the second original logical value L = [L4...L n . Mask the first original logical value and the second original logical value respectively, that is, use different masking methods to determine whether to retain the generation probability of the first original logical value or the generation probability of the second original logical value, and finally obtain the target logical value, and generate reference labels for the target reference text and the target statement according to the target logical value.

[0054] According to the technical solution provided by the embodiment of the present application, based on the original logical value and the predefined set of elements, determine the first original logical value and the second original logical value; mask the first original logical value and the second original logical value respectively to obtain the target logical value, which can ensure that only the labels within the predefined range are generated for the subsequent generation of reference labels, avoid the possibility of adding incorrect labels, and at the same time ensure that the form of the labels is not incorrect.

[0055] In some examples, masking the first original logical value and the second original logical value respectively to obtain a target logical value includes: constructing a first masking vector and a second masking vector; masking the first original logical value with the first masking vector to obtain a first masked logical value, and masking the second original logical value with the second masking vector to obtain a second masked logical value; constructing the target logical value according to the first masked logical value and the second masked logical value.

[0056] Specifically, to construct the first masking vector and the second masking vector, since different masks need to be applied to the first original logical value and the second original logical value, two completely different masking vectors, namely the first masking vector and the second masking vector, need to be constructed.

[0057] In some examples, continuing from the above example, the first masking vector is preferably zero. In this way, after the first original logical value is processed by the first masking vector, the generation probability of the label corresponding to the first original logical value remains unchanged, ensuring the normal generation of the label corresponding to the first original logical value; the second masking vector is preferably negative infinity. In this way, after the second original logical value is processed by the second masking vector, the generation probability of the label corresponding to the second original logical value becomes zero, ensuring that in the subsequent process of generating reference labels, no label corresponding to the second original logical value is generated. Finally, the target logical value is constructed according to the first masked logical value and the second masked logical value. Therefore, the reference label generated according to this target logical value can only be the label corresponding to the first original logical value.

[0058] In some examples, construct a first masking vector M1 and a second masking vector M2. The first masking vector M1 is set to zero, and the first masking vector M2 is set to negative infinity. At this time, add the vector L corresponding to the first original logical value to the first masking vector M1. Since M1 is zero, the addition result is L, so the first original logical value is retained. Add the vector L corresponding to the second original logical value to the first masking vector M1. Since M2 is negative infinity, the addition result is negative infinity, which makes the generation probability corresponding to the second original logical value approach 0 in the subsequent normalization process.

[0059] According to the technical solution provided by the embodiments of the present application, construct a first masking vector and a second masking vector; mask the first original logical value with the first masking vector to obtain a first masked logical value, and mask the second original logical value with the second masking vector to obtain a second masked logical value; construct the target logical value according to the first masked logical value and the second masked logical value. It can ensure the label corresponding to the first original logical value, suppress the label corresponding to the second original logical value, ensure that only the predefined range and form of labels are generated when generating reference labels, and improve the generation accuracy of reference labels.

[0060] In some embodiments, a target activation function is used to normalize a target logical value to obtain the generation probability of the label corresponding to the target logical value; according to the generation probability of the label, a label that can be used for reference is determined, and reference labels are generated for at least one reference text and a target statement according to the label that can be used for reference.

[0061] Specifically, in machine learning and deep learning, the activation function plays a role in non-linearly transforming the input signal. In this embodiment, the target activation function is used to perform a normalization operation on the target logical value. The target activation function preferably adopts the softmax function. Since the target logical value is an unnormalized value, which represents the original scores or tendencies of the large language model for different labels, but cannot be directly used as a probability. Therefore, it is necessary to use the target activation function to convert the target logical value into a probability distribution, so that the generation probability of the label corresponding to the target logical value can be obtained, and then the label with the appropriate probability (generally the highest generation probability) is selected as the reference label of the target statement.

[0062] Further, according to the generation probability of the label, a label that can be used for reference is determined, and the label with the appropriate probability (generally the highest generation probability) is selected as the reference label of the target statement. And reference labels are generated for at least one reference text and the target statement according to the label that can be used for reference.

[0063] In some embodiments, determining an initial reference text corresponding to a text generation instruction includes: determining a vector representation of the text corresponding to the text generation instruction; performing matching in a preset vector text library according to the vector representation to obtain a first reference text; extracting keywords included in the text generation instruction, and performing matching in a preset text library according to the keywords to obtain a second reference text; determining the initial reference text according to the first reference text and the second reference text.

[0064] Specifically, the text generation instruction may be a text instruction, an image instruction, or a voice instruction. When receiving the text generation instruction, the text included in the text generation instruction can be extracted and vectorized. The text vector representation refers to representing the semantics of the text with a numerical vector, and representing the text information in a numerical form that can be understood and processed by a machine for subsequent calculation and processing.

[0065] The preset vector text library contains vectorized representations of multiple reference texts. Specifically, the HuggingFace embedding model can be used to convert the text information in each reference text into a vector representation. Meanwhile, FAISS or Pinecone is utilized to construct a vectorized index, and multiple reference text vectors are stored in a structure similar to an index table through FAISS or Pinecone for subsequent quick lookup and retrieval. After receiving a text generation instruction, it is also converted into a text vector through the same HuggingFace embedding model to ensure that it is in the same vector space as the previous reference text vectors for constructing the vectorized document index. The similarity formula is used to calculate the similarity between the vectorized representation of the text corresponding to the text generation instruction and each of the reference text vectors stored in the FAISS or Pinecone index before. The query result vectors with similarity greater than the preset threshold are selected as at least two reference texts, and at the same time, the metadata information of these documents, such as index_id (the unique identifier of the document in the index) and source (the source of the document), is returned. The first reference text obtained in this way can provide the user with the document that best matches the text generation instruction corresponding to it and some basic information about these documents, so as to improve the accuracy of the target statement.

[0066] In some examples, the methods for vectorizing text include word vector models, sentence vector models, etc. Specifically, it can include methods such as the Bag of Words Model, Term Frequency-Inverse Document Frequency (TF-IDF), Word2vec, Doc2vec, etc. The method for vectorized representation is not limited here.

[0067] In addition, the keywords contained in the text generation instruction are extracted and matched in the preset text library to obtain the second reference text, and the initial reference text is determined based on the first reference text and the second reference text. In this way, it is further ensured that the initial reference text is highly relevant to the text generation instruction, improving the accuracy of the generated target statement.

[0068] According to the technical solution provided by the embodiment of the present application, the vectorized representation of the text corresponding to the text generation instruction is determined; matching is performed in the preset vector text library according to the vectorized representation to obtain the first reference text; the keywords contained in the text generation instruction are extracted and matched in the preset text library to obtain the second reference text; the initial reference text is determined based on the first reference text and the second reference text. Thus, it is ensured that there is a high similarity between the text generation instruction and the reference text, ensuring the efficiency and accuracy of the subsequent generated target statement.

[0069] It should be understood that the sequence numbers of the steps in the above embodiments do not indicate the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the process of the embodiments of the present application.

[0070] All of the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be elaborated here one by one.

[0071] The following is an embodiment of the apparatus of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the embodiment of the apparatus of the present application, please refer to the method embodiment of the present application.

[0072] Figure 2 It is a schematic structural diagram of a text generation apparatus provided by an embodiment of the present application. As Figure 2 shown, the apparatus includes:

[0073] A first determination module 201, configured to determine an initial reference text corresponding to the text generation instruction in response to the received text generation instruction;

[0074] A first generation module 202, configured to generate a target statement according to the initial reference text and a first constraint condition;

[0075] A second determination module 203, configured to construct a prompt message by using the target statement, and determine at least one target reference text cited by the target statement from the initial reference text according to the prompt message;

[0076] A second generation module 204 generates reference labels for the target reference text and the target statement according to a second constraint condition, where the reference labels represent the reference relationship between the target reference text and the target statement, and uses the target statement with reference labels as the result text of the text generation instruction.

[0077] In some embodiments, the first generation module 202 is further configured to generate target content according to the initial reference text; detect the target content by using a target callback function, and if the detection result indicates that a preset identifier appears in the target content, obtain the statement corresponding to the target content, and use the statement as the target statement.

[0078] In some embodiments, the second generation module 204 is further configured to input the target reference text and the target statement into a large language model; obtain the original logical values generated by the large language model, where the number of original logical values is multiple, each original logical value corresponds to each label, and each original logical value represents the unnormalized score of each label; determine the target logical value according to the original logical value and a predefined element set; generate reference labels for the target reference text and the target statement according to the target logical value.

[0079] In some embodiments, the second generation module 204 is further configured to determine a first original logical value and a second original logical value according to an original logical value and a predefined set of elements; respectively mask the first original logical value and the second original logical value to obtain target logical values.

[0080] In some embodiments, the second generation module 204 is further configured to construct a first mask vector and a second mask vector; mask the first original logical value with the first mask vector to obtain a first masked logical value, and mask the second original logical value with the second mask vector to obtain a second masked logical value; construct a target logical value according to the first masked logical value and the second masked logical value.

[0081] In some embodiments, the second generation module 204 is further configured to normalize the target logical value by using a target activation function to obtain a generation probability corresponding to the label of the target logical value; determine a label that can be used for reference according to the generation probability of the label, and generate a reference label for the target reference text and the target statement according to the label that can be used for reference.

[0082] In some embodiments, the first determination module 201 is further configured to determine a vectorized representation of the text corresponding to the text generation instruction; perform matching in a preset vector text library according to the vectorized representation to obtain a first reference text, where the preset vector text library includes vectorized representations of multiple reference texts; extract keywords included in the text corresponding to the text generation instruction, and perform matching in a preset text library according to the keywords to obtain a second reference text, where the preset text library includes multiple reference texts; determine an initial reference text according to the first reference text and the second reference text.

[0083] According to the apparatus provided in the embodiments of the present application, in response to a received text generation instruction, an initial reference text corresponding to the text generation instruction is determined; a target statement is generated according to the initial reference text and a first constraint condition; a prompt message is constructed by using the target statement, and at least one target reference text referred to by the target statement is determined from the initial reference text according to the prompt message; a reference label is generated for the target reference text and the target statement according to a second constraint condition, the reference label represents a reference relationship between the target reference text and the target statement, and the target statement with the reference label is used as the result text of the text generation instruction. Ensure that each time a target statement appears, a reference label is generated for the target statement and its referred target reference text, avoiding the influence of the context relationship on the generation of the reference label after all the text is generated, and ensuring that the generated reference label is more accurate. At the same time, ensure that the generated reference label conforms to the label generation range and label generation form constrained by the second constraint condition, avoiding the problem of unstable reference label format, easy occurrence of format errors or inconsistencies caused by the traditional method of letting the large language model generate reference labels for text, and ensuring that only reference labels within the constrained range are generated.

[0084] Figure 3 is a schematic diagram of the electronic device 3 provided by an embodiment of the present application. As Figure 3 shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 301 executes the computer program 303, the functions of the various modules / units in the above-mentioned device embodiments are implemented.

[0085] The electronic device 3 can be a desktop computer, a notebook, a palm computer, a cloud server, and other electronic devices. The electronic device 3 may include, but is not limited to, the processor 301 and the memory 302. Those skilled in the art can understand that Figure 3 merely examples of the electronic device 3, which do not constitute a limitation on the electronic device 3, and may include more or fewer components than shown in the figure, or different components.

[0086] The processor 301 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0087] The memory 302 may be an internal storage unit of the electronic device 3, for example, the hard disk or memory of the electronic device 3. The memory 302 may also be an external storage device of the electronic device 3, for example, a plug-in hard disk equipped on the electronic device 3, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 302 may also include both an internal storage unit and an external storage device of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0088] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0089] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, to implement all or part of the processes in the above embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in the readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. The computer program can include computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The readable storage medium can include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0090] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A text generation method, characterized in that, including: responding to the received text generation instruction, determining an initial reference text corresponding to the text generation instruction; generating a target statement according to the initial reference text and a first constraint condition; constructing a prompt message by using the target statement, and determining at least one target reference text cited by the target statement from the initial reference text according to the prompt message; generating reference labels for the target reference text and the target statement according to a second constraint condition, where the reference labels represent the citation relationship between the target reference text and the target statement, and taking the target statement with reference labels as the result text of the text generation instruction.

2. The method according to claim 1, wherein The generating a target statement according to the initial reference text and a first constraint condition includes: generating target content according to the initial reference text; detecting the target content by using a target callback function, if the detection result indicates that a preset identifier appears in the target content, obtaining the statement corresponding to the target content, and taking the statement as the target statement.

3. The method according to claim 1, wherein The generating reference labels for the target reference text and the target statement according to a second constraint condition includes: inputting the target reference text and the target statement into a large language model; obtaining the original logical values generated by the large language model, where the number of the original logical values is multiple, each of the original logical values corresponds to each label, and each of the original logical values represents the unnormalized score of each label; determining a target logical value according to the original logical values and a predefined element set; generating reference labels for the target reference text and the target statement according to the target logical value.

4. The method according to claim 3, characterized in that, The determining a target logical value according to the original logical values and a predefined element set includes: determining a first original logical value and a second original logical value according to the original logical values and a predefined element set; respectively masking the first original logical value and the second original logical value to obtain the target logical value.

5. The method according to claim 4, wherein The respectively masking the first original logical value and the second original logical value to obtain the target logical value includes: constructing a first mask vector and a second mask vector; masking the first original logical value by using the first mask vector to obtain a first masked logical value, and masking the second original logical value by using the second mask vector to obtain a second masked logical value; constructing the target logical value according to the first masked logical value and the second masked logical value.

6. The method according to claim 3, wherein The generating reference labels for the target reference text and the target statement according to the target logical value includes: normalizing the target logical value by using a target activation function to obtain the generation probability of the label corresponding to the target logical value; determining the labels that can be used for citation according to the generation probability of the label, and generating reference labels for the target reference text and the target statement according to the labels that can be used for citation.

7. The method according to claim 1, wherein Determining an initial reference text corresponding to the text generation instruction includes: determining the vectorized representation of the text corresponding to the text generation instruction; Match in a preset vector text library according to the vectorized representation to obtain a first reference text, where the preset vector text library contains vectorized representations of multiple reference texts; Extract keywords included in the text corresponding to the text generation instruction, and match according to the keywords in the preset text library to obtain a second reference text, where the preset text library contains multiple reference texts; Determine the initial reference text according to the first reference text and the second reference text.

8. A text generation method, characterized in that, Includes: A first determination module configured to determine an initial reference text corresponding to the text generation instruction in response to the received text generation instruction; A first generation module configured to generate a target statement according to the initial reference text and a first constraint condition; A second determination module configured to construct prompt information using the target statement, and determine at least one target reference text cited by the target statement from the initial reference text according to the prompt information; A second generation module generates reference labels for the target reference text and the target statement according to a second constraint condition, the reference labels represent the reference relationship between the target reference text and the target statement, and use the target statement with reference labels as the result text of the text generation instruction.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images