Text generation method, apparatus, device, medium, and program product

Abstract

Embodiments of the present application provide a text generation method, device, equipment, medium and program product, and belong to the technical field of natural language processing. The above method comprises: determining N nodes matched with a query vector in a node database based on a text generation task; screening the N nodes based on context information of a position where the text generation task is located to obtain M nodes; determining a prompt word sequence according to a target text template corresponding to the text generation task, the context information and the M nodes; inserting a target context into the prompt word sequence to obtain a model input sequence; and inputting the model input sequence and the target text template into a preset text generation model to generate a target text. In the embodiments of the present application, in the process of generating the target text, the target text is generated by node data stored in the node database, instead of being limited to node data of adjacent paragraphs, so that the calling of cross-paragraph information is realized, and the accuracy of the target text is improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a text generation method, apparatus, device, medium, and program product. Background Technology

[0002] With the development of natural language processing technology, more and more users are using auxiliary tools for writing. Official document writing usually requires the integration of various heterogeneous information and ensuring the logical closure of the text. It has the characteristics of content standardization, logical coherence, and text structure. However, manual writing of official documents has problems such as low efficiency and difficulty in information integration.

[0003] Currently, various document writing assistance solutions based on large-scale models have emerged. However, the contextual constraints in these solutions are limited to the connection between adjacent paragraphs. When generating extremely long documents with large amounts of data and multiple paragraphs, they cannot access information across paragraphs, resulting in logical breaks, missing information, and content deviations in the generated text. Therefore, existing technologies suffer from poor document text accuracy. Summary of the Invention

[0004] This application provides a text generation method, apparatus, device, medium, and program product that can improve the accuracy of the generated text.

[0005] In a first aspect, embodiments of this application provide a text generation method, the method comprising:

[0006] Based on the text generation task, N nodes matching the query vector are determined in the node database. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1.

[0007] Based on the context information of the location of the text generation task, the N nodes are filtered to obtain M nodes, where M is a positive integer greater than 1 and less than N;

[0008] Based on the target text template corresponding to the text generation task, the context information, and the M nodes, a prompt word sequence is determined, wherein the prompt word sequence includes multiple prompt word nodes;

[0009] The target context is inserted into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes.

[0010] The model input sequence and the target text template are input into a preset text generation model to generate the target text.

[0011] Optionally, the step of filtering the N nodes based on the context information of the location of the text generation task to obtain M nodes includes:

[0012] Calculate the semantic similarity between the context information and each of the N nodes;

[0013] Delete the nodes whose semantic similarity is less than the first preset threshold from the N nodes to obtain K nodes, where K is a positive integer greater than M and less than N;

[0014] Using the text generation model, M nodes out of the K nodes are identified that have a semantic relationship with the text generation task.

[0015] Optionally, determining the prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes includes:

[0016] For any prompt word, the semantic matching degree corresponding to the prompt word is determined based on the semantic type of the blank slot in the target text template and the node type of the node corresponding to the prompt word.

[0017] The semantic relationship matching degree of the prompt word is determined based on the semantic relationship between the context information and the node corresponding to the prompt word;

[0018] Based on the context information and the text data stored in the node corresponding to the prompt word, determine the context matching degree of the prompt word;

[0019] The semantic matching degree, semantic relationship matching degree and context matching degree corresponding to the prompt word are weighted and summed to obtain the weight value corresponding to each prompt word;

[0020] A sequence of prompt words is generated based on the order of their weight values ​​from highest to lowest.

[0021] Optionally, after inputting the model input sequence and the target text template into a preset text generation model to generate the target text, the method further includes:

[0022] Perform text validation on the target text to obtain the text validation value corresponding to the target text;

[0023] If the text verification value is less than the second preset threshold, the context information is retrieved again, and the process of determining N nodes that match the query vector in the node database based on the text generation task is returned.

[0024] If the text verification value is greater than or equal to the second preset threshold, the target text is input into the node database to update the node database.

[0025] Optionally, the step of performing text validation on the target text to obtain the text validation value corresponding to the target text includes:

[0026] Based on the numerical text included in the target text and the numerical text included in the node database, determine the text consistency value corresponding to the target text;

[0027] Based on the clause text included in the target text and the clause text included in the node database, determine the text accuracy value corresponding to the target text;

[0028] Based on the semantic relationships between the paragraphs included in the target text, determine the text integrity value corresponding to the target text;

[0029] The text consistency value, text accuracy value, and text integrity value corresponding to the target text are weighted and summed to obtain the text verification value corresponding to the target text.

[0030] Optionally, before the text generation task determines N nodes in the node database that match the query vector, the method further includes:

[0031] Obtain the reference text;

[0032] The reference text is preprocessed to obtain a text sequence vector, which is a vector representation of the reference text;

[0033] The text sequence vector is subjected to semantic segmentation and deduplication merging operations to obtain multiple semantic units;

[0034] Information extraction and relation extraction are performed on the multiple semantic units to obtain multiple nodes, each node storing node type, text data and node number;

[0035] The node database is constructed based on the semantic relationships between the multiple nodes.

[0036] Secondly, embodiments of this application provide a text generation apparatus, the apparatus comprising:

[0037] The first determining module is used to determine N nodes that match the query vector in the node database based on the text generation task. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1.

[0038] The filtering module is used to filter the N nodes based on the context information of the location of the text generation task to obtain M nodes, where M is a positive integer greater than 1 and less than N;

[0039] The second determining module is used to determine a prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes, wherein the prompt word sequence includes multiple prompt word nodes;

[0040] An insertion module is used to insert the target context into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes.

[0041] The generation module is used to input the model input sequence and the target text template into a preset text generation model to generate the target text.

[0042] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the text generation method as described in the first aspect.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the text generation method as described in the first aspect.

[0044] Fifthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the text generation method as described in the first aspect.

[0045] In this embodiment, a prompt word sequence is determined based on the target text template, context information, and multiple nodes stored in the node database. The context information and text data stored in the multiple nodes are concatenated to obtain the target context. The target context is then inserted into the prompt word sequence to obtain the model input sequence. Finally, the model input sequence and the target text template are input into a preset text generation model to generate the target text. Thus, in the process of generating the target text, the target text is generated using node data stored in the node database, rather than being limited to node data from adjacent paragraphs. This enables the retrieval of cross-paragraph information and improves the accuracy of the generated target text. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of a text generation method provided in an embodiment of this application;

[0048] Figure 2 This is a schematic diagram illustrating the model input sequence generation and application process provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of some nodes stored in the node database provided in an embodiment of this application;

[0050] Figure 4 This is an application flowchart of a text generation method provided in an embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the structure of a text generation device provided in an embodiment of this application;

[0052] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] See Figure 1 , Figure 1 This is a flowchart of a text generation method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0055] Step 101: Based on the text generation task, determine N nodes in the node database that match the query vector. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1.

[0056] In this step, the text generation task can be determined based on the current position of the generated text. For example, when the text is in the "Current Status of Digital Economy Development" chapter, the text generation task is determined to be "Generate the target text for the chapter on the current status of digital economy development".

[0057] The text generation task is vectorized to obtain the query vector. Optionally, the task description statement represented by the text generation task can be vectorized to obtain the query vector.

[0058] In a pre-defined node database, the query vector is matched to identify N nodes that match it. Optionally, this node database is also called a global memory, and the nodes are memory nodes. Each memory node stores a node type, text data, and a node number. The node type represents the type of text data stored in the memory node, such as policy, data, or task. The node number represents the location of the text data stored in the memory node within the reference text; for example, a node number of Doc001-Ch03-Pg15-Sen02 indicates that the text data stored in this memory node is the second sentence of the fifteenth paragraph of Chapter 3 in the first reference text.

[0059] Optionally, the query vector can be matched with the text data of all nodes in the node database using cosine similarity, and nodes with a similarity value higher than 0.7 can be identified as nodes that match the query vector.

[0060] Step 102: Based on the context information of the location of the text generation task, filter the N nodes to obtain M nodes, where M is a positive integer greater than 1 and less than N.

[0061] In this step, after determining N nodes, the N nodes are filtered based on context information to obtain M nodes. In other words, the M nodes are a subset of the N nodes. The context information refers to the text data at the location of the text generation task.

[0062] Step 103: Determine the prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes. The prompt word sequence includes multiple prompt word nodes.

[0063] It's important to understand that various text templates are pre-set. In application scenarios for generating official documents, templates covering 15 common document types, such as notices, reports, requests for instructions, approvals, minutes, and proposals, can be set. Each text template contains predefined blank slots with specific semantic types, and the semantic types corresponding to the blank slots have a one-to-one or one-to-many mapping relationship with the node types stored in the node database.

[0064] In this step, the target text template corresponding to the text generation task is determined. For example, if the text generation task is represented as "generating notification documents", then the notification type document text template can be determined as the target text template.

[0065] Furthermore, based on the target text template, context information, and M nodes, a sequence of prompt words is determined. For specific implementation details, please refer to subsequent embodiments.

[0066] Step 104: Insert the target context into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes.

[0067] In this step, the text data stored in M ​​nodes is concatenated with context information to generate the target context. This target context is then inserted into the prompt word sequence to obtain the model input sequence.

[0068] An alternative implementation involves setting the starting position of a sequence as the insertion position of the target context, that is, inserting the target context at the starting position of a sequence, and then filling the remaining positions of the sequence with prompt word sequences to obtain the model input sequence.

[0069] Another alternative implementation involves inserting the target context into a sequence at a position adjacent to the location of the text generation task, and then filling the remaining positions in the sequence with a sequence of prompt words to obtain the model input sequence.

[0070] Optionally, during the subsequent inference process of the text generation model, a higher attention weight can be assigned to the inserted target context to improve the accuracy of the model's inference results and enhance the accuracy and consistency of the generated target text.

[0071] Step 105: Input the model input sequence and the target text template into the preset text generation model to generate the target text.

[0072] In this step, the model input sequence and the target text template are input into the preset text generation model. The text generation model uses the prompt words in the model input sequence to call the text data stored in the nodes of the model input sequence to fill the blank slots in the target text template and generate the target text.

[0073] It should be understood that in the process of generating target text, the text generation model prioritizes generating the text content corresponding to the prompt words that appear first in the input sequence of the model. This ensures that the core meaning and overall logical thread of the target text are established first, thereby improving the logicality of the target text content.

[0074] Optionally, the above text generation models include large language models, deep learning models, etc.

[0075] In this embodiment, a prompt word sequence is determined based on the target text template, context information, and multiple nodes stored in the node database. The context information and text data stored in the multiple nodes are concatenated to obtain the target context. The target context is then inserted into the prompt word sequence to obtain the model input sequence. Finally, the model input sequence and the target text template are input into a preset text generation model to generate the target text. Thus, in the process of generating the target text, the target text is generated using node data stored in the node database, rather than being limited to node data from adjacent paragraphs. This enables the retrieval of cross-paragraph information and improves the accuracy of the generated target text.

[0076] Optionally, the step of filtering the N nodes based on the context information of the location of the text generation task to obtain M nodes includes:

[0077] Calculate the semantic similarity between the context information and each of the N nodes;

[0078] Delete the nodes whose semantic similarity is less than the first preset threshold from the N nodes to obtain K nodes, where K is a positive integer greater than M and less than N;

[0079] Using the text generation model, M nodes out of the K nodes are identified that have a semantic relationship with the text generation task.

[0080] In this embodiment, for each of the N nodes, the semantic similarity between that node and the context information is calculated to obtain the semantic similarity between the context information and each node. Then, from the N nodes, nodes with semantic similarity less than a first preset threshold are deleted, resulting in K nodes, where K is a positive integer greater than M and less than N; that is, the K nodes are a subset of the N nodes. Optionally, the first preset threshold is 0.8.

[0081] After obtaining K nodes, the logical relationship between the K nodes and the text generation task is analyzed. Optionally, the prompt words corresponding to the text generation task and the above K nodes are input into the text generation model, and the text generation model determines M nodes among the K nodes that have semantic association with the text generation task.

[0082] For example, if the text generation task represents "generating the target text for the chapter on the current state of digital economic development", then K nodes and the prompt word corresponding to the text generation task, "please identify the nodes that have a semantic relationship with the chapter on the current state of digital economic development", can be input into the text generation model. The text generation model is based on the semantic relationship between the semantic representation of the text data stored in each of the K nodes and the semantic relationship with the text generation task, and deletes the nodes with low semantic relationship among the K nodes to obtain M nodes.

[0083] In this embodiment, semantic layer filtering is performed by calculating the semantic similarity between context information and each of the N nodes; M nodes that are semantically related to the text generation task are determined by the text generation model and logical layer filtering is performed. In this way, the nodes most relevant to the text generation task are selected from the node database, thereby realizing the retrieval of global information.

[0084] Optionally, determining the prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes includes:

[0085] For any prompt word, the semantic matching degree corresponding to the prompt word is determined based on the semantic type of the blank slot in the target text template and the node type of the node corresponding to the prompt word.

[0086] The semantic relationship matching degree of the prompt word is determined based on the semantic relationship between the context information and the node corresponding to the prompt word;

[0087] Based on the context information and the text data stored in the node corresponding to the prompt word, determine the context matching degree of the prompt word;

[0088] The semantic matching degree, semantic relationship matching degree and context matching degree corresponding to the prompt word are weighted and summed to obtain the weight value corresponding to each prompt word;

[0089] A sequence of prompt words is generated based on the order of their weight values ​​from highest to lowest.

[0090] It should be noted that there are pre-set prompt words, and each prompt word corresponds to a node.

[0091] In this embodiment, the target document template includes blank slots. The node type of the node corresponding to the prompt word is matched with the semantic type of the blank slot to determine the semantic matching degree of the prompt word.

[0092] For example, when the semantic type of a blank slot represents "policy basis", the matching degree between the nodes corresponding to the "policy clauses" and "policy regulations" node types and the blank slot is relatively high. In other words, the semantic matching degree of the prompt words corresponding to the above nodes is relatively high.

[0093] As mentioned above, the nodes also store text data. In this embodiment, the semantic relationship matching degree corresponding to the prompt word is determined based on the semantics represented by the context information and the semantics represented by the text data stored in the node corresponding to the prompt word. Generally speaking, prompt words that are strongly related to the context information have a higher semantic relationship matching degree.

[0094] In this embodiment, the context matching degree corresponding to the prompt word is determined based on the relative distance between the context information and the text data stored in the node corresponding to the prompt word. Generally speaking, the closer the relative distance between the context information and the text data, the higher the context matching degree corresponding to the prompt word. Optionally, the context matching degree corresponding to the prompt word can be normalized so that the context matching degree corresponding to the prompt word is within the range of [0, 1].

[0095] After determining the semantic matching degree, semantic relation matching degree, and context matching degree for each prompt word, a weighted sum of these three degrees is performed to obtain a weight value for each prompt word. This weight value can be understood as a priority score for the prompt word.

[0096] Alternatively, the weight value corresponding to the prompt word can be calculated using the following formula:

[0097] ;

[0098] in, Indicates the weight value. Indicates semantic matching degree. Indicates the semantic relationship matching degree. Indicates the degree of context matching. , ,and Both represent weighting coefficients, and , ,and The sum is 1. Optionally, set It is 0.4. It is 0.4. It is 0.2.

[0099] After obtaining the weight value corresponding to each prompt word, a prompt word sequence is generated according to the order of the weight values ​​of multiple prompt words from high to low.

[0100] Optionally, the top L prompt words are selected according to their weight values ​​from highest to lowest to form a prompt word sequence, where L is a positive integer less than M and greater than 1.

[0101] In this embodiment, semantic type matching is performed on preset prompt words based on the semantic type of the blank slots in the target text template; logical relationship matching is performed on preset prompt words based on the semantic relationship between the context information and the nodes corresponding to the prompt words; and context distance matching is performed on preset prompt words based on the context information and the text data stored in the nodes corresponding to the prompt words. Thus, according to the requirements of the text generation task, prompt words are filtered through multi-dimensional feature matching, and the combination and presentation order of prompt words are dynamically adjusted to ensure that the global information injected through the prompt word sequence highly matches the semantic type and logical requirements of the text generation task.

[0102] For a clear explanation of the solution, please refer to [link / reference needed]. Figure 2 , Figure 2 This is a schematic diagram illustrating the model input sequence generation and application process provided in an embodiment of this application. For example... Figure 2 As shown, based on context information, target text template, and M nodes in the node database, multi-dimensional feature matching, including semantic type matching, logical relationship matching, and context distance matching, is performed on the prompt words. A priority ranking algorithm is used to determine the weight value corresponding to each prompt word based on semantic matching degree, semantic relationship matching degree, and context matching degree. The top L prompt words are selected according to the order of their weight values ​​from high to low to form a prompt word sequence.

[0103] Furthermore, contextual information is inserted into the cue word sequence through either initial insertion or adjacent insertion, and high attention weights are assigned to this contextual information to form the model input sequence. This input sequence and the target text template are then fed into the text generation model for inference to generate the target text.

[0104] Optionally, after inputting the model input sequence and the target text template into a preset text generation model to generate the target text, the method further includes:

[0105] Perform text validation on the target text to obtain the text validation value corresponding to the target text;

[0106] If the text verification value is less than the second preset threshold, the context information is retrieved again, and the process of determining N nodes that match the query vector in the node database based on the text generation task is returned.

[0107] If the text verification value is greater than or equal to the second preset threshold, the target text is input into the node database to update the node database.

[0108] In this embodiment, after generating the target text, text validation is performed on the target text to obtain the corresponding text validation value. This text validation value is used to characterize the accuracy of the target text.

[0109] If the text verification value is less than the second preset threshold, it indicates that the accuracy of the target text is low and the generated text content has conflicts or deviations. In this case, the context information is re-acquired and the target text is regenerated based on the re-acquired context information.

[0110] If the text verification value is greater than or equal to the second preset threshold, it indicates that the target text is highly accurate and passes the target text verification. Then, the target text is input into the node database to update the node database.

[0111] In this embodiment, if the text verification value corresponding to the target text is less than the second preset threshold, the context information is re-acquired and the target text is regenerated; if the text verification value corresponding to the target text is greater than or equal to the second preset threshold, the node database is updated using the target text, thereby forming an iterative closed loop of "generation-verification-writeback" to ensure that the target text with a very long paragraph meets the requirements of global information consistency and the text content realizes a logical closed loop.

[0112] Optionally, the step of performing text validation on the target text to obtain the text validation value corresponding to the target text includes:

[0113] Based on the numerical text included in the target text and the numerical text included in the node database, determine the text consistency value corresponding to the target text;

[0114] Based on the clause text included in the target text and the clause text included in the node database, determine the text accuracy value corresponding to the target text;

[0115] Based on the semantic relationships between the paragraphs included in the target text, determine the text integrity value corresponding to the target text;

[0116] The text consistency value, text accuracy value, and text integrity value corresponding to the target text are weighted and summed to obtain the text verification value corresponding to the target text.

[0117] In this embodiment, the text consistency value corresponding to the target text can be determined based on the numerical text included in the target text and the numerical text included in the node database; the text accuracy value corresponding to the target text can be determined based on the clause text included in the target text and the clause text included in the node database; the text integrity value corresponding to the target text can be determined based on the semantic relationship between the paragraphs included in the target text; and then the text consistency value, text accuracy value and text integrity value corresponding to the target text are weighted and summed to obtain the text verification value corresponding to the target text.

[0118] Specifically, it can be expressed by the following formula:

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] in, This represents the text validation value; , ,and Both represent weighting coefficients, and , ,and The sum of is 1; Indicates the numerical value of text consistency; This indicates the numerical value representing the accuracy of the text. Indicates a numerical value representing text integrity; This indicates the quantized value of the numerical text included in the target text; This represents the quantized value of the numeric text included in the node database; This indicates the preset error tolerance, which can be customized. For example, the allowed fluctuation range of the quantized value is ±5%. It is 5; This indicates the number of matching relationships between the clause text included in the target text and the clause text included in the node database; Indicates the number of clauses included in the target text; This indicates the number of paragraphs in the target text that have logical relationships. Indicates the total number of paragraphs included in the target text.

[0124] Optionally, before the text generation task determines N nodes in the node database that match the query vector, the method further includes:

[0125] Obtain the reference text;

[0126] The reference text is preprocessed to obtain a text sequence vector, which is a vector representation of the reference text;

[0127] The text sequence vector is subjected to semantic segmentation and deduplication merging operations to obtain multiple semantic units;

[0128] Information extraction and relation extraction are performed on the multiple semantic units to obtain multiple nodes, each node storing node type, text data and node number;

[0129] The node database is constructed based on the semantic relationships between the multiple nodes.

[0130] In this embodiment, reference text is obtained and preprocessed to obtain a text sequence vector.

[0131] Optionally, the above preprocessing includes metadata extraction and text normalization. Specifically:

[0132] After obtaining the reference text, the first step is to extract metadata, which can be done through a data parser and a pattern matcher. For structured data in the reference text, the data parser identifies and processes specific tags and fields within the structured data; for unstructured data in the reference text, the pattern matcher uses a predefined set of regular expressions and a rule base to extract standardized metadata fields from the beginning or specific areas (e.g., the cover page, the first page) of the reference text.

[0133] After the metadata extraction is complete, the unstructured text content can be normalized using text cleaning filters and regular expression matchers, including the following steps:

[0134] 1. Use regular expressions to batch remove redundant symbols and special characters from text.

[0135] 2. Employ a layout-based machine learning model that combines specific keywords (such as "Page X", "Total Y Pages", "Copyright Reserved") and location information (top or bottom of text area) to identify and remove headers, footers, and page numbers.

[0136] 3. Identify and remove non-core text such as table of contents, reference lists, image descriptions, and table border descriptions.

[0137] 4. Force the text content to be converted to a uniform encoding, such as UTF-8 encoding, to ensure the consistency of the character set.

[0138] After obtaining the text sequence vector, semantic segmentation and deduplication merging operations are performed on the text sequence vector to obtain multiple semantic units.

[0139] It should be noted that semantic segmentation of text sequence vectors includes the steps of identifying semantic segmentation points and generating multi-granularity units. Specifically:

[0140] A deep learning-based document structure analysis model is used to perform semantic recognition and semantic segmentation on text sequence vectors. Optionally, in the application scenario of generating official document text, the above model is trained on a dedicated corpus containing hundreds of thousands of official documents, which can identify multi-level document structure features. Furthermore, the above model combines a Transformer encoder and a conditional random field layer to identify explicit semantic segmentation points.

[0141] Based on the identified semantic segmentation points, the ultra-long text sequence is precisely divided into multi-granularity units, and each multi-granularity unit is assigned a globally unique identifier, which is used to indicate the position information in the original reference text.

[0142] It should be noted that the deduplication and merging operation for multi-granularity units includes calculating inter-unit similarity and merging redundant units. Specifically:

[0143] The segmented multi-granularity units are paired and their semantic similarity is calculated. Optionally, a general semantic vector model, such as the bge-m3 model, can be used to convert each multi-granularity unit into a 1024-dimensional semantic embedding vector, and the semantic similarity is quantified by calculating the cosine similarity between the vectors.

[0144] If the semantic similarity between two multi-granularity units exceeds a preset threshold, optionally 0.85, then it is determined whether they are duplicates. If they are duplicate multi-granularity units, one of them is directly deleted; if they are not duplicate multi-granularity units, they are merged. The merging strategy includes retaining the unit with more information and more comprehensive expression from the two multi-granularity units; or, inputting the two multi-granularity units into a text encoding model, such as the T5-base model, to merge the two multi-granularity units and generate a new multi-granularity unit. The aforementioned multi-granularity unit can be understood as a semantic unit.

[0145] In this embodiment, after obtaining multiple semantic units, a Named Entity Recognition (NER) module is applied to extract information from these units, resulting in multiple node information. In the application scenario of generating official document text, the aforementioned NER module supports the recognition of over 50 specific entity types. It should be understood that for the structured information included in multiple semantic units, rule matching and pattern recognition algorithms can be used for information extraction.

[0146] Optionally, after applying the NER module to extract information from multiple semantic units and obtain multiple node information, normalization and deduplication operations can be performed on the multiple node information to eliminate the heterogeneity between node information and ensure the uniqueness of information.

[0147] Optionally, the implementation involves performing numerical normalization, date normalization, name normalization, and clause normalization on multiple node information. Semantic embedding vectors of the normalized node information are then generated. The cosine similarity between each semantic embedding vector is calculated, and a similarity threshold is set. Optionally, the similarity threshold is 0.9. For two semantic embedding vectors with a cosine similarity higher than the similarity threshold, the semantic embedding vector with greater information and more comprehensive expression is retained, thereby achieving deduplication of multiple node information.

[0148] Optionally, after applying the NER module to extract information from multiple semantic units and obtain multiple node information, nodes are generated based on the information of each node, and the semantic relationship between each node is determined through the Relation Extraction (RE) module. The RE module is implemented using an attention-based relation classifier, which learns the contextual features, positional information, and pre-trained word embeddings between nodes to determine their semantic relationships.

[0149] It should be understood that semantic relations include logical relations and contextual co-occurrence relations.

[0150] Logical relationships include: premise-measure relationship (PREMISE_OF); task-progress relationship (TASK_PROGRESS); policy-execution relationship (POLICY_EXECUTION); subject-object relationship (AGENT_OBJECT); causal relationship (CAUSAL_RELATION); containment relationship (CONTAINS); dependency relationship (DEPENDS_ON); and temporal order relationship (TEMPORAL_ORDER).

[0151] If the text data corresponding to two nodes appear in the same paragraph or two adjacent sentences, then a contextual co-occurrence relationship is determined between the two nodes. The contextual co-occurrence relationship can be represented as CO_OCCURS_WITH.

[0152] In an optional application scenario, please refer to Figure 3 , Figure 3 This is a schematic diagram of some nodes stored in the node database provided in this application embodiment. Figure 3Seven nodes are shown. Each node stores a node type, text data, and a node number. Adjacent nodes are connected by lines, which represent the semantic relationships between the nodes.

[0153] Figure 3 The connecting lines also correspond to edge weight values, which are determined by the semantic association strength between nodes. Optionally, the edge weight values ​​can be calculated using the following formula:

[0154] ;

[0155] in, This represents the i-th node. This represents the j-th node; Represents a node With nodes The edge weight values ​​between them; Represents a node Corresponding semantic embedding vector , with nodes Corresponding semantic embedding vector Normalized cosine similarity between them; Represents a node and nodes The number of times it appears in the same paragraph or chapter; this value is a normalized value based on the number of occurrences. Represents a node With nodes Relative positional distance in the reference text; , ,and Both represent weighting coefficients, and , ,and The sum is 1, optionally, Set to 0.5. Set to 0.3, Set it to 0.2.

[0156] Optionally, the nodes stored in the node database can be hierarchically classified into top-level nodes, intermediate nodes, and bottom-level nodes. In the application scenario of generating official documents, the top-level nodes represent global core themes or high-level guiding information, such as national strategies, annual work goals, and core policies; the intermediate nodes represent specific tasks, strategies, or departmental work objectives that support the top-level nodes, such as specific implementation plans and phased task indicators; and the bottom-level nodes represent specific execution details, quantitative indicators, timelines, responsible persons, and specific operational steps.

[0157] In this embodiment, a node database with global information is constructed by performing corresponding operations on the reference text. By calling the nodes in the node database, global information is injected into the subsequent target text generation process, enabling the invocation of cross-paragraph information, thereby ensuring that the generated ultra-long text meets the requirements of accurate text content and logical closure.

[0158] For a better understanding of the overall technical solution, please refer to [link / reference]. Figure 4 ,like Figure 4 As shown, the text generation method provided in this application includes six steps: Step 1: Input data processing and semantic pre-parsing; Step 2: Global memory rewriting and structured storage; Step 3: Dynamic retrieval with enhanced ultra-long context; Step 4: Global information-driven attention optimization; Step 5: Structured content generation and global information mapping; Step 6: Content verification and write-back.

[0159] In step one, the reference text is formatted to obtain a text sequence vector. Then, the text sequence vector is subjected to chapter segmentation (equivalent to semantic segmentation in the above embodiment) and deduplication / merging operations to obtain another text sequence vector. The formatting operation includes metadata extraction and text normalization; the chapter segmentation operation includes identifying semantic segmentation points and generating multi-granularity units; and the deduplication / merging operation includes calculating inter-unit similarity and removing and merging redundant units.

[0160] In step two, key information extraction, information normalization and deduplication, edge representation, and logical relationship network construction are performed on the text sequence vectors to obtain a node database. Key information extraction includes named entity recognition, rule matching and pattern recognition, and relationship extraction; information normalization and deduplication include normalization and deduplication operations; edge representation and logical relationship network construction includes hierarchical classification and edge weight calculation.

[0161] In step three, an initial search and hierarchical summarization and filtering are performed on the nodes included in the node database, resulting in M ​​nodes. The hierarchical summarization and filtering includes semantic layer filtering, logical layer filtering, and enhanced context concatenation.

[0162] In step four, the model input sequence is determined by performing input reconstruction and cue word scheduling operations. Input reconstruction includes explicit injection and weight labeling. Explicit injection inserts the target context into the cue word sequence, and weight labeling marks the attention weights of the target context. Cue word scheduling includes a multi-dimensional feature matching algorithm and a priority ranking algorithm to determine the cue word sequence.

[0163] In step five, key information is matched with template slots through document template loading and slot recognition, generating target text segment by segment. The process of matching key information with template slots includes slot semantic type recognition and information extraction and filling.

[0164] In step six, the generated target text undergoes numerical consistency verification, cross-chapter content self-consistency verification, and citation accuracy verification. If the target text passes the text verification, it is input into the node database to update the node database; if the target text fails the text verification, the context information is retrieved again, and the process returns to step three.

[0165] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a text generation device provided in an embodiment of this application, as shown below. Figure 5 As shown, the text generation device 500 includes:

[0166] The first determining module 501 is used to determine N nodes that match the query vector in the node database based on the text generation task. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1.

[0167] The filtering module 502 is used to filter the N nodes based on the context information of the location of the text generation task to obtain M nodes, where M is a positive integer greater than 1 and less than N;

[0168] The second determining module 503 is used to determine a prompt word sequence based on the target text template corresponding to the text generation task, the context information and the M nodes, wherein the prompt word sequence includes multiple prompt word nodes;

[0169] The insertion module 504 is used to insert the target context into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes.

[0170] The generation module 505 is used to input the model input sequence and the target text template into a preset text generation model to generate the target text.

[0171] Optionally, the filtering module 502 is specifically used for:

[0172] Calculate the semantic similarity between the context information and each of the N nodes;

[0173] Delete the nodes whose semantic similarity is less than the first preset threshold from the N nodes to obtain K nodes, where K is a positive integer greater than M and less than N;

[0174] Using the text generation model, M nodes out of the K nodes are identified that have a semantic relationship with the text generation task.

[0175] Optionally, the second determining module 503 is specifically used for:

[0176] For any prompt word, the semantic matching degree corresponding to the prompt word is determined based on the semantic type of the blank slot in the target text template and the node type of the node corresponding to the prompt word.

[0177] The semantic relationship matching degree of the prompt word is determined based on the semantic relationship between the context information and the node corresponding to the prompt word;

[0178] Based on the context information and the text data stored in the node corresponding to the prompt word, determine the context matching degree of the prompt word;

[0179] The semantic matching degree, semantic relationship matching degree and context matching degree corresponding to the prompt word are weighted and summed to obtain the weight value corresponding to each prompt word;

[0180] A sequence of prompt words is generated based on the order of their weight values ​​from highest to lowest.

[0181] Optionally, the text generation device 500 further includes:

[0182] The verification module is used to perform text verification on the target text and obtain the text verification value corresponding to the target text.

[0183] The first acquisition module is used to reacquire context information when the text verification value is less than the second preset threshold, and return to the step of performing the text generation task to determine N nodes that match the query vector in the node database;

[0184] The update module is used to input the target text into the node database to update the node database when the text verification value is greater than or equal to the second preset threshold.

[0185] Optionally, the verification module is specifically used for:

[0186] Based on the numerical text included in the target text and the numerical text included in the node database, determine the text consistency value corresponding to the target text;

[0187] Based on the clause text included in the target text and the clause text included in the node database, determine the text accuracy value corresponding to the target text;

[0188] Based on the semantic relationships between the paragraphs included in the target text, determine the text integrity value corresponding to the target text;

[0189] The text consistency value, text accuracy value, and text integrity value corresponding to the target text are weighted and summed to obtain the text verification value corresponding to the target text.

[0190] Optionally, the text generation device 500 further includes:

[0191] The second acquisition module is used to acquire reference text;

[0192] A preprocessing module is used to preprocess the reference text to obtain a text sequence vector, wherein the text sequence vector is a vector representation of the reference text;

[0193] The operation module is used to perform semantic segmentation and deduplication merging operations on the text sequence vector to obtain multiple semantic units;

[0194] The extraction module is used to extract information and extract relationships from the multiple semantic units to obtain multiple nodes, wherein each node stores node type, text data and node number.

[0195] A construction module is used to construct the node database based on the semantic relationships between the multiple nodes.

[0196] The text generation device 500 is capable of implementing the various processes applied to the various embodiments of the text generation method described above. The technical features correspond one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0197] This application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described text generation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0198] For details, see Figure 6 This application also provides an electronic device, including a bus 601, a transceiver 602, an antenna 603, a bus interface 604, a processor 605, and a memory 606.

[0199] The processor 605 is configured to determine N nodes that match a query vector in a node database based on a text generation task. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1.

[0200] Based on the context information of the location of the text generation task, the N nodes are filtered to obtain M nodes, where M is a positive integer greater than 1 and less than N;

[0201] Based on the target text template corresponding to the text generation task, the context information, and the M nodes, a prompt word sequence is determined, wherein the prompt word sequence includes multiple prompt word nodes;

[0202] The target context is inserted into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes.

[0203] The transceiver 602 is used to input the model input sequence and the target text template into a preset text generation model to generate the target text.

[0204] exist Figure 6 In this document, a bus architecture (represented by bus 601) is used. Bus 601 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 605 and memory represented by memory 606. Bus 601 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 604 provides an interface between bus 601 and transceiver 602. Transceiver 602 may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 605 is transmitted over a wireless medium via antenna 603, which further receives data and transmits data to processor 605.

[0205] Processor 605 manages bus 601 and general processing, and also provides various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 606 can be used to store data used by processor 605 during operation.

[0206] Optionally, the processor 605 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0207] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described text generation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0208] This application also provides a computer program product, including computer instructions. When executed by a processor, these computer instructions implement the various processes of the above-described text generation method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0209] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0211] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A text generation method, characterized in that, The method includes: Based on the text generation task, N nodes matching the query vector are determined in the node database. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1. Based on the context information of the location of the text generation task, the N nodes are filtered to obtain M nodes, where M is a positive integer greater than 1 and less than N; Based on the target text template corresponding to the text generation task, the context information, and the M nodes, a prompt word sequence is determined, wherein the prompt word sequence includes multiple prompt word nodes; The target context is inserted into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes. The model input sequence and the target text template are input into a preset text generation model to generate the target text.

2. The method according to claim 1, characterized in that, The N nodes are filtered based on the context information of the location of the text generation task to obtain M nodes, including: Calculate the semantic similarity between the context information and each of the N nodes; Delete the nodes whose semantic similarity is less than the first preset threshold from the N nodes to obtain K nodes, where K is a positive integer greater than M and less than N; Using the text generation model, M nodes out of the K nodes are identified that have a semantic relationship with the text generation task.

3. The method according to claim 1, characterized in that, The step of determining the prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes includes: For any prompt word, the semantic matching degree corresponding to the prompt word is determined based on the semantic type of the blank slot in the target text template and the node type of the node corresponding to the prompt word. The semantic relationship matching degree of the prompt word is determined based on the semantic relationship between the context information and the node corresponding to the prompt word; Based on the context information and the text data stored in the node corresponding to the prompt word, determine the context matching degree of the prompt word; The semantic matching degree, semantic relationship matching degree and context matching degree corresponding to the prompt word are weighted and summed to obtain the weight value corresponding to each prompt word; A sequence of prompt words is generated based on the order of their weight values ​​from highest to lowest.

4. The method according to any one of claims 1 to 3, characterized in that, After inputting the model input sequence and the target text template into a preset text generation model to generate the target text, the method further includes: Perform text validation on the target text to obtain the text validation value corresponding to the target text; If the text verification value is less than the second preset threshold, the context information is retrieved again, and the process of determining N nodes that match the query vector in the node database based on the text generation task is returned. If the text verification value is greater than or equal to the second preset threshold, the target text is input into the node database to update the node database.

5. The method according to claim 4, characterized in that, The step of performing text validation on the target text to obtain the text validation value corresponding to the target text includes: Based on the numerical text included in the target text and the numerical text included in the node database, determine the text consistency value corresponding to the target text; Based on the clause text included in the target text and the clause text included in the node database, determine the text accuracy value corresponding to the target text; Based on the semantic relationships between the paragraphs included in the target text, determine the text integrity value corresponding to the target text; The text consistency value, text accuracy value, and text integrity value corresponding to the target text are weighted and summed to obtain the text verification value corresponding to the target text.

6. The method according to any one of claims 1 to 3, characterized in that, Before the text-based generation task determines N nodes in the node database that match the query vector, the method further includes: Obtain the reference text; The reference text is preprocessed to obtain a text sequence vector, which is a vector representation of the reference text; The text sequence vector is subjected to semantic segmentation and deduplication merging operations to obtain multiple semantic units; Information extraction and relation extraction are performed on the multiple semantic units to obtain multiple nodes, each node storing node type, text data and node number; The node database is constructed based on the semantic relationships between the multiple nodes.

7. A text generation device, characterized in that, The device includes: The first determining module is used to determine N nodes that match the query vector in the node database based on the text generation task. The query vector is obtained by vector transformation of the text generation task. The node database includes multiple preset nodes, where N is a positive integer greater than 1. The filtering module is used to filter the N nodes based on the context information of the location of the text generation task to obtain M nodes, where M is a positive integer greater than 1 and less than N; The second determining module is used to determine a prompt word sequence based on the target text template corresponding to the text generation task, the context information, and the M nodes, wherein the prompt word sequence includes multiple prompt word nodes; An insertion module is used to insert the target context into the prompt word sequence to obtain the model input sequence. The target context is generated by concatenating the context information and the text data stored in the M nodes. The generation module is used to input the model input sequence and the target text template into a preset text generation model to generate the target text.

8. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 6.

Images