Text generation method, device, electronic device and computer-readable storage medium

Abstract

The present application provides a text generation method, apparatus, electronic device and computer-readable storage medium. The method includes: retrieving documents and information block information related to the input information through a set model; each document includes one or more information blocks, and the information blocks are configured to be obtained by decomposing the text in the document; matching target information blocks related to the input information according to the input information; determining one or more information blocks around the target information blocks; merging the target information blocks and one or more information blocks around the target information blocks to generate target text; the target text is the output text corresponding to the input information. When the present application performs retrieval, it merges the retrieved target information blocks and other information blocks around the target information blocks. Merging the target information blocks can make the obtained target text more comprehensive and accurate, and improve the integrity and accuracy of the target text.

Description

Technical Field

[0001] This application relates to the field of information processing. Specifically, it relates to a text generation method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] In the current information retrieval field, especially in retrieval-augmented generation (RAG) systems, it usually relies on powerful embedding models or traditional information retrieval algorithms such as BM25 to extract relevant information from large-scale knowledge bases. When using an embedding model for information retrieval, although it can capture the semantic similarity between documents, due to the complexity of the high-dimensional vector space, it is often difficult to accurately locate specific and fine-grained knowledge points. This results in the retrieval results possibly containing a large number of fragmentary information that does not fully match or is only partially relevant to the user's query intent, affecting the accuracy and integrity of the text. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of this application is to provide a text generation method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy and integrity of the text.

[0004] In a first aspect, the embodiments of this application provide a text generation method, including: retrieving documents and information block information related to the input information through a set model; where each document includes one or more information blocks, and the information block is configured to be obtained by decomposing the text in the document; matching a target information block related to the input information according to the input information; determining one or more information blocks around the target information block; merging the target information block and one or more information blocks around the target information block to generate a target text; where the target text is the output text corresponding to the input information.

[0005] In the above implementation process, during retrieval, the retrieved target information block and other information blocks around the target information block are merged. Merging the target information block can make the obtained target text more comprehensive and accurate, improving the integrity and accuracy of the target text.

[0006] In one embodiment, the information block information includes: document meta-information to which the information block belongs; before matching the target information block related to the input information according to the input information, the method further includes: determining the document to which each information block belongs according to the document meta-information; performing flag bit recognition on the content in the document, and classifying the information blocks according to the flag bit recognition result; wherein, the information blocks in one document are of the same type; the matching of the target information block related to the input information according to the input information includes: matching the target information block related to the input information according to the input information and the corresponding matching method of the type to which the information block belongs.

[0007] In the above implementation process, by classifying the information blocks and performing target information block matching on each type of information block through the corresponding matching method, the accuracy of the matched target information block can be improved.

[0008] In one embodiment, the type to which the information block belongs includes title enhancement type; the matching of the target information block related to the input information according to the input information and the corresponding matching method of the type to which the information block belongs includes: if it is recognized that the document includes a flag bit and the flag bit is a title flag bit, determining that the information block is of the title enhancement type; taking out all the information blocks in the document; matching the title flag bit corresponding to the input information, and determining the information block corresponding to the matched title flag bit as the target information block; the merging of the target information block and one or more information blocks around the target information block to generate the target text includes: obtaining all the text contents under the target information block; determining the information blocks before and after the target information block, and obtaining all the text contents in the information blocks before and after the target information block; merging all the text contents under the target information block and all the text contents in the information blocks before and after the target information block to generate the target text.

[0009] In the above implementation process, when performing target information block matching, matching according to the corresponding matching method of the type to which the target information block belongs can improve the accuracy of the target information block. In addition, when the information block is of the title enhancement type, directly determining the corresponding target information block by matching the title flag bit can reduce the matching difficulty of the target information block and improve the matching efficiency.

[0010] In one embodiment, the type to which the information block belongs includes the large model generalization type; the matching of the target information block related to the input information according to the corresponding matching method of the input information and the type to which the information block belongs includes: if it is recognized that the document includes a flag bit and the flag bit is the large model generalization flag bit, determining that the information block is of the large model generalization type; taking out all the information blocks in the document; matching the large model generalization paragraph overview corresponding to the input information, and determining the information block corresponding to the matched large model generalization paragraph overview as the target information block; the merging of the target information block and one or more information blocks around the target information block to generate the target text includes: obtaining all the text contents under the large model generalization paragraph overview; determining the information blocks before and after the large model generalization paragraph overview, and obtaining all the text contents in the information blocks before and after the large model generalization paragraph overview; merging all the text contents under the large model generalization paragraph overview and all the text contents in the information blocks before and after the large model generalization paragraph overview to generate the target text.

[0011] In the above implementation process, when the information block is of the large model generalization type, the corresponding target information block can be determined by matching the large model generalization type, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0012] In one embodiment, the type to which the information block belongs includes the general type; the matching of the target information block related to the input information according to the corresponding matching method of the input information and the type to which the information block belongs includes: if it is recognized that the document does not include a flag bit, determining that the information block is of the general type; taking out all the information blocks in the document; determining the target information block according to the relevance score between the information block and the input information; the merging of the target information block and one or more information blocks around the target information block to generate the target text includes: determining the target information blocks within a set distance range, and obtaining all the text contents in the target information blocks within the set distance range, as well as all the text contents in the information blocks between the target information blocks within the set distance range; merging all the text contents in the target information blocks within the set distance range and all the text contents in the information blocks between the target information blocks within the set distance range to generate the target text.

[0013] In the above implementation process, by setting the corresponding merging method in the case where the information block is of the general type, the application scenarios of this text generation method can be increased. In addition, for general information blocks, when matching the target information block, matching according to the relevance score between the information block and the input information can match the information block strongly associated with the input information, improving the accuracy of the target information block.

[0014] In one embodiment, before retrieving documents and information block information related to the input information through the set model, the method further includes: extracting relevant title levels in the document; dividing the information in the document by the title; isolating each title and the information content corresponding to the title through a title flag bit to obtain a plurality of shards; if the number of flags of a certain shard is greater than the set number of flags, then sharding the information content; wherein, each shard includes a title string.

[0015] In the above implementation process, by using the title flag bit to shard the information content in the document according to the title, the sharding method is simple and easy to implement. In addition, after sharding by the title, when matching the target information block, the corresponding target information block can be directly determined by matching the title flag bit, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0016] In one embodiment, before retrieving documents and information block information related to the input information through the set model, the method further includes: dividing the information content in the document by paragraph levels; obtaining the corresponding content summaries of the information content in each block; isolating between each content summary and the corresponding information content through a large model summary flag bit.

[0017] In the above implementation process, by using the large model summary flag bit to shard the information content in the document according to paragraphs, the sharding effect is good and the content in the information block after sharding can be reduced. Furthermore, when matching the target information block, the corresponding target information block can be directly determined by the large model summary flag bit, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0018] In a second aspect, an embodiment of the present application further provides a text generation device, including: a retrieval module, configured to retrieve documents and information block information related to the input information through a set model; wherein, each document includes one or more information blocks, and the information block is configured to be obtained by decomposing the text in the document; a matching module, configured to match a target information block related to the input information according to the input information; a determination module, configured to determine one or more information blocks around the target information block; a merging module, configured to merge the target information block and one or more information blocks around the target information block to generate a target text; wherein, the target text is the output text corresponding to the input information.

[0019] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory, the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, when the machine-readable instructions are executed by the processor, the steps of the method in the first aspect, or any possible implementation manner of the first aspect are executed.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the text generation method in the first aspect or any possible implementation manner of the first aspect.

[0021] To make the above objects, features, and advantages of the present application more obvious and understandable, specific embodiments are hereinafter given and described in detail in conjunction with the accompanying drawings. Description of the Drawings

[0022] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a block diagram of an electronic device provided by an embodiment of the present application;

[0024] Figure 2 It is a flowchart of a text generation method provided by an embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of functional modules of a text generation device provided by an embodiment of the present application. Detailed Embodiments

[0026] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0027] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0028] In an existing Retrieval-Augmented Generation (RAG) system, knowledge base information retrieval is a core link, which directly affects whether the system can accurately and efficiently provide users with the required information. To achieve this goal, the system generally uses an Embedding model or a BM25 scheme for retrieval.

[0029] The Embedding model is a natural language processing technology that can convert text into numerical vectors, which can capture the semantic meaning and context information of the text. Therefore, in the RAG system, the Embedding model can be used to vectorize query statements and documents in the knowledge base, and then retrieve relevant information by calculating the similarity between vectors. The advantage of this method is that it can capture the deep semantic information of the text, thus improving the accuracy and relevance of the retrieval. However, since the Embedding model retrieves based on the vector representation of the text, it may be affected by issues such as the diversity of text expressions and synonyms, resulting in some information in the retrieval results that is not exactly a perfect match with the query but is semantically similar, thus causing information fragmentation.

[0030] The BM25 scheme is a classic information retrieval algorithm based on term frequency and inverse document frequency. It calculates the importance of each term in the document by statistically analyzing the terms in the document, and sorts and filters the documents according to these importance levels. The BM25 scheme performs well in processing long documents and short queries and can quickly find the documents most relevant to the query. However, since it retrieves based on lexical-level matching, it may ignore some information that is semantically related to the query but not directly lexically matched, which also leads to information fragmentation in the retrieval results.

[0031] In view of this, this application proposes a text generation method. When performing retrieval, it merges the retrieved target information chunks and other information chunks around the target information chunk. Merging the target information chunk can make the resulting target text more comprehensive and accurate, improving the integrity and accuracy of the target text.

[0032] To facilitate the understanding of this embodiment, the electronic device that executes the text generation method disclosed in the embodiments of this application will be introduced in detail first.

[0033] As Figure 1 shown, it is a block diagram of the electronic device. The electronic device 100 may include a memory 111 and a processor 113. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only illustrative and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0034] The above-mentioned memory 111 and processor 113 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The above-mentioned processor 113 is used to execute the executable module stored in the memory.

[0035] Among them, the memory 111 can be, but is not limited to, random access memory (Random Access Memory, abbreviated as RAM), read-only memory (Read Only Memory, abbreviated as ROM), programmable read-only memory (Programmable Read-Only Memory, abbreviated as PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, abbreviated as EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, abbreviated as EEPROM), etc. Among them, the memory 111 is used to store a program, and after receiving an execution instruction, the processor 113 executes the program. The method executed by the electronic device 100 defined by the process disclosed in any embodiment of the present application can be applied to the processor 113 or implemented by the processor 113.

[0036] The above-mentioned processor 113 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 113 can be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; it can also be a digital signal processor (digital signal processor, abbreviated as DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0037] The electronic device 100 in this embodiment can be used to execute each step in the various methods provided in the embodiments of the present application. The implementation process of the text generation method will be described in detail below through several embodiments.

[0038] Please refer to Figure 2 , which is a flowchart of the text generation method provided by the embodiments of the present application. Next,Figure 2 The specific process shown will be elaborated in detail.

[0039] Step 201: Retrieve documents and information block information related to the input information through a set model.

[0040] The set model here is a pre-set query model for document and information block retrieval. For example, an Embedding vectorization model, a BM25 retrieval model, etc. This set model can be selected according to the actual situation.

[0041] Among them, the BM25 retrieval model is a traditional retrieval model where BM25 belongs to the term frequency-inverse document frequency (TF-IDF) and is used to measure the relevance between a query and a document. The Embedding model uses a neural network to generate vector representations of words or sentences and retrieves semantically relevant content by measuring distances in the vector space.

[0042] The above input information refers to information such as words, sentences, pictures, etc. used to obtain retrieval results, and this input information can be selected according to the actual situation.

[0043] It can be understood that in some AI large model application scenarios, users input corresponding questions, instructions, etc. into the window of the AI large model. The AI large model retrieves in the knowledge base, slices and merges the retrieved results to obtain corresponding results, and displays these results in the window of the AI large model.

[0044] The documents here can be documents in one or more knowledge bases associated with the set model. The information block information refers to one or more information blocks in the documents.

[0045] Among them, each document includes one or more information blocks, and the information blocks are configured to be obtained by decomposing the text in the document.

[0046] In one embodiment, the information block information may include: document meta-information to which the information block belongs, document meta-information to which the information block belongs, information block content text, etc. Among them, the document meta-information to which the information block belongs may include information such as document ID, knowledge base ID to which it belongs, information block index, etc. The specific content of this information block information can be selected according to the actual situation.

[0047] Step 202: Match the target information block related to the input information according to the input information.

[0048] The information blocks here are obtained by matching according to the relevance to the input information.

[0049] Optionally, the target information block can be one or more.

[0050] The above-mentioned target information block can be determined according to the relevance to the input information. For example, an information block with a relevance greater than the relevance threshold is determined as the target information block.

[0051] Step 203: Determine one or more information blocks around the target information block.

[0052] One or more information blocks around the target information here refer to the information blocks on the upper side or the lower side adjacent to the target information. Of course, the information blocks around the target information can also be the information blocks within a set range from the target information. One or more information blocks around the target information can be selected according to the actual situation.

[0053] Step 204: Merge the target information block and one or more information blocks around the target information block to generate the target text.

[0054] Among them, the target text is the output text corresponding to the input information.

[0055] In one embodiment, when merging information blocks, the information blocks in the same document are merged.

[0056] It can be understood that one or more target information blocks matched by the input information are very likely to be scattered. When merging the information blocks in the same document, if these scattered target information blocks are directly merged, the coherence and accuracy of the merged target text may be low. The present application proposes that when merging information blocks, the target information block is merged with the information blocks around it, and then the corresponding target text is generated, which can improve the coherence and accuracy of the target text.

[0057] Among them, after determining the target information block, all the text contents corresponding to the target information can be determined. After determining the information blocks around the target information block, all the text contents corresponding to the information around the target information block can also be determined. Furthermore, when merging the target information block and one or more information blocks around the target information block, all the text contents corresponding to the target information and all the text contents corresponding to the information around the target information block can be merged to obtain the target text.

[0058] In the above implementation process, when performing retrieval, the retrieved target information block is merged with other information blocks around the target information block. Merging the target information block can make the obtained target text more comprehensive and accurate, and improve the integrity and accuracy of the target text.

[0059] In a possible implementation manner, before step 202, the method further includes: determining the document to which each information block belongs according to the document meta-information; performing flag bit recognition on the content in the document, and classifying the information blocks according to the flag bit recognition result.

[0060] The document to which each information block here belongs can be determined by the document ID. Among them, the information blocks in a document are of the same type.

[0061] In one embodiment, the information blocks in a document can be isolated by flag bits. For example, a title flag bit, a large model summary flag bit, etc.

[0062] It can be understood that by identifying the specific string structure of the flag bits in the document, the type to which the information block belongs can be determined.

[0063] Optionally, the types to which the information blocks belong include types such as title enhanced type, large model summary type, and general type, and the type to which the information block belongs can be selected according to the actual situation.

[0064] In one embodiment, step 202 includes: matching the target information block related to the input information according to the matching method corresponding to the input information and the type to which the information block belongs.

[0065] It can be understood that a corresponding matching method can be set for each type of information block. When matching the target information block, match according to the matching method corresponding to the type to which the information block belongs.

[0066] In the above implementation process, by classifying the information blocks and matching the target information blocks for each type of information block through the corresponding matching method, the accuracy of the matched target information block can be improved.

[0067] In one possible implementation manner, matching the target information block related to the input information according to the matching method corresponding to the input information and the type to which the information block belongs includes: if it is recognized that the document includes a flag bit and the flag bit is a title flag bit, determining that the information block is of the title enhanced type; taking out all the information blocks in the document; matching the title flag bit corresponding to the input information, and determining the information block corresponding to the matched title flag bit as the target information block.

[0068] In one embodiment, before determining that the information block is of the title enhanced type, the method further includes: performing flag bit recognition on the classified information blocks in the document. Among them, each information block is isolated by a flag bit.

[0069] The flag bit here can be a string of special characters. For example, the title flag bit can be: ##title_split##, and the large model summary flag bit can be: ##model_split##.

[0070] The representation form of the above-mentioned title enhanced type information block can be: the information block corresponds to the hierarchical title + flag bit + information block content.

[0071] Understandably, the document ID can be determined first, and then all the information blocks contained in the document can be retrieved.

[0072] It should be understood that after obtaining the input information, the title can be matched through information such as keywords and key phrases in the input information to match the corresponding title flag bit. After determining the information block corresponding to the matched title flag bit, the information block corresponding to the title flag bit is determined as the target information block.

[0073] In one embodiment, step 204 includes: obtaining all the text content under the target information block; determining the information blocks before and after the target information block, and obtaining all the text content in the information blocks before and after the target information block; merging all the text content under the target information block and all the text content in the information blocks before and after the target information block to generate the target text.

[0074] Optionally, the information blocks before and after the target information block can be one information block before and after the target information block, two information blocks before and after the target information block, three information blocks before and after the target information block, etc. The number of information blocks before and after the target information block can be selected according to the actual situation.

[0075] It should be understood that when merging the text content, the merging can be performed in sequence according to the order of the text content in the document.

[0076] In the above implementation process, when matching the target information block, matching according to the matching method corresponding to the type of the target information block can improve the accuracy of the target information block. In addition, when the information block is title-enhanced, directly determining the corresponding target information block by matching the title flag bit can reduce the matching difficulty of the target information block and improve the matching efficiency.

[0077] In one possible implementation manner, matching the target information block related to the input information according to the input information and the matching method corresponding to the type of the information block includes: if it is recognized that the document includes a flag bit and the flag bit is a large model summary flag bit, determining that the information block is a large model summary type; retrieving all the information blocks in the document; matching the large model summary paragraph overview corresponding to the input information, and determining the information block corresponding to the matched large model summary paragraph overview as the target information block.

[0078] In one embodiment, before determining that the information block is a large model summary type, the method further includes: performing flag bit recognition on the classified information blocks in the document. Among them, each information block is isolated by a flag bit.

[0079] The representation form of the above-mentioned information block of the large model summary type can be: large model paragraph overview + flag bit + information block content.

[0080] The large model generalization type here refers to the type that summarizes a passage, a sentence, or a paragraph.

[0081] Understandably, when there is one or more paragraphs in a document and the content in the paragraphs is relatively large, a paragraph or a sentence can be used to summarize the paragraph. When performing target information block matching, by matching the input information with the large model generalization flag bit, the matching efficiency can be improved.

[0082] After obtaining the input information, the large model generalization flag bit can be matched through information such as keywords and key phrases in the input information to match the corresponding large model generalization flag bit. After determining the information block corresponding to the matched large model generalization flag bit, determine the information block corresponding to the large model generalization flag bit as the target information block.

[0083] In one embodiment, step 204 includes: obtaining all the text content under the large model generalization paragraph overview; determining the information blocks before and after the large model generalization paragraph overview, and obtaining all the text content in the information blocks before and after the large model generalization paragraph overview; merging all the text content under the large model generalization paragraph overview and all the text content in the information blocks before and after the large model generalization paragraph overview to generate the target text.

[0084] In the above implementation process, when the information block is of the large model generalization type, the corresponding target information block can be determined by matching the large model generalization type, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0085] In one possible implementation manner, according to the input information and the corresponding matching method of the information block type, matching the target information block related to the input information includes: if it is recognized that the document does not include a flag bit, determining that the information block is of the general type; taking out all the information blocks in the document; determining the target information block according to the relevance score between the information block and the input information.

[0086] The general type here refers to the information block that is not segmented by a flag bit.

[0087] In one embodiment, the information blocks can be grouped according to the information of the information blocks, and the information blocks associated with the document information can be taken out from the database.

[0088] The above-mentioned relevance score refers to the relevance between each information block and the input information.

[0089] Optionally, the information block with a relevance score greater than the relevance threshold can be determined as the target information block.

[0090] In one embodiment, step 204 includes: determining target information blocks within a set distance range, obtaining all the text content in the target information blocks within the set distance range, and all the text content in the information blocks between the target information blocks within the set distance range; merging all the text content in the target information blocks within the set distance range and all the text content in the information blocks between the target information blocks within the set distance range to generate target text.

[0091] Wherein, the set distance range refers to a pre-set distance range.

[0092] Optionally, the set distance range may refer to: the range where a set number of information blocks are located around the target information block; the set distance range may also refer to: the range between two adjacent target information blocks when the number of information blocks between two adjacent target information blocks is within a set number, etc. Of course, the set distance can also be a combination of the above two methods. The set distance range can be adjusted according to the actual situation.

[0093] Exemplarily, if the set distance range refers to: the range between two adjacent target information blocks when the number of information blocks between two adjacent target information blocks is less than or equal to 5. And the determined target information blocks include: target information block 1, target information block 5, and target information block 13. Among them, the number of information blocks between target information block 1 and target information block 5 is 3, and the number of information blocks between target information block 5 and target information block 13 is 7. Then, obtain all the text content in all the information blocks from target information block 1 to target information block 5 and merge these text contents. In addition, since the number of information blocks between target information block 13 and target information block 5 exceeds 5, obtain the text content of target information block 13 separately and output the text content of target information block 13 separately.

[0094] If the set distance range refers to: the range between two adjacent target information blocks when the number of information blocks between two adjacent target information blocks is less than or equal to 5, and the range where one information block is located around the target information block. And the determined target information blocks include: target information block 1, target information block 5, and target information block 13. Among them, the number of information blocks between target information block 1 and target information block 5 is 3, and the number of information blocks between target information block 5 and target information block 13 is 7. Then, obtain all the text content in all the information blocks from target information block 1 to target information block 5 and merge these text contents. In addition, since the number of information blocks between target information block 13 and target information block 5 exceeds 5, obtain the text content of the information block adjacent above target information block 13 and the text content of the information block adjacent below target information block 13 respectively, and merge the text content of target information block 13, the information block adjacent above target information block 13, and the information block adjacent below target information block 13.

[0095] In the above implementation process, when the information block is set to be general-purpose, the corresponding merging method can increase the application scenarios of this text generation method. In addition, for general-purpose information blocks, when matching target information blocks, matching according to the correlation score between the information block and the input information can match information blocks that are strongly correlated with the input information, improving the accuracy of the target information block.

[0096] In a possible implementation manner, before step 201, the method further includes: extracting relevant title levels in the document; dividing the information in the document by titles; isolating each title and the information content corresponding to the title through a title flag bit to obtain multiple slices; if the number of flags of a certain slice is greater than the set number of flags, then slice the information content.

[0097] In an embodiment, after extracting relevant title levels in the document, a parent-child structure can be formed.

[0098] Each of these title levels has its corresponding information content. Among them, the parent-child structure corresponding to the information content can be flattened into a title string. For example, Title 1 -> Title 2 -> Title 3, this structure will be flattened into Title 1 - Title 2 - Title 3.

[0099] It can be understood that after flattening the parent-child structure corresponding to the information content into a title string, the flattened title string and the corresponding information content can be merged, and isolated by the title flag bit in the middle.

[0100] Among them, if the number of flags of the information content is greater than the maximum number of flags supported by the slice where it is located, then the information content can be sliced.

[0101] Each of these slices includes a title string and is isolated by a flag bit string.

[0102] In the above implementation process, by using the title flag bit to slice the information content in the document according to the title, the slicing method is simple and easy to implement. In addition, after slicing by the title, when matching the target information block, the corresponding target information block can be directly determined by matching the title flag bit, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0103] In a possible implementation manner, before step 201, the method further includes: dividing the information content in the document by paragraph levels; obtaining the corresponding content summaries of the information content in each block; isolating between each content summary and the corresponding information content through a large model generalization flag bit.

[0104] Among them, the content summary refers to the summary of the content in the paragraph. The content summary can be controlled within a certain number of characters.

[0105] It should be understood that after obtaining the corresponding content summaries of the information content in each block, the summary and the information content can be merged, and then isolated by the large model summary flag bit in the middle.

[0106] In the above implementation process, by using the large model summary flag bit to slice the information content in the document by paragraph, the slicing effect is good, and the content in the information blocks after slicing can be reduced. Furthermore, when matching the target information block, the corresponding target information block can be directly determined through the large model summary flag bit, reducing the matching difficulty of the target information block and improving the matching efficiency.

[0107] Based on the same application concept, the embodiments of the present application also provide a text generation device corresponding to the text generation method. Since the principle of solving problems by the device in the embodiments of the present application is similar to that of the foregoing text generation method embodiments, the implementation of the device in this embodiment can refer to the description in the embodiments of the above method, and the repeated parts will not be elaborated.

[0108] Please refer to Figure 3 , which is a schematic diagram of the functional modules of the text generation device provided by the embodiments of the present application. Each module in the text generation device in this embodiment is used to execute each step in the above method embodiments. The text generation device includes a retrieval module 301, a matching module 302, a determination module 303, and a merging module 304; wherein,

[0109] The retrieval module 301 is used to retrieve documents and information block information related to the input information through a set model; wherein, each document includes one or more information blocks, and the information blocks are configured to be obtained by decomposing the text in the document.

[0110] The matching module 302 is used to match the target information block related to the input information according to the input information.

[0111] The determination module 303 is used to determine one or more information blocks around the target information block.

[0112] The merging module 304 is used to merge the target information block and one or more information blocks around the target information block to generate a target text; wherein, the target text is the output text corresponding to the input information.

[0113] In a possible implementation manner, the text generation device further includes a classification module, which is used to: determine the documents to which each information block belongs according to the document meta information; perform flag bit recognition on the content in the document, and classify the information blocks according to the flag bit recognition result; wherein, the information blocks in one document are of the same type.

[0114] In a possible implementation, the matching module 302 is further configured to: match a target information block related to the input information according to the input information and the corresponding matching method of the type to which the information block belongs.

[0115] In a possible implementation, the matching module 302 is specifically configured to: if it is recognized that the document includes a flag bit and the flag bit is a title flag bit, determine that the information block is a title enhanced type; extract all information blocks in the document; match the title flag bit corresponding to the input information, and determine the information block corresponding to the matched title flag bit as the target information block.

[0116] In a possible implementation, the merging module 304 is further configured to: obtain all text contents under the target information block; determine the information blocks before and after the target information block, and obtain all text contents in the information blocks before and after the target information block; merge all text contents under the target information block and all text contents in the information blocks before and after the target information block to generate the target text.

[0117] In a possible implementation, the matching module 302 is specifically configured to: if it is recognized that the document includes a flag bit and the flag bit is a large model summary flag bit, determine that the information block is a large model summary type; extract all information blocks in the document; match the large model summary paragraph overview corresponding to the input information, and determine the information block corresponding to the matched large model summary paragraph overview as the target information block.

[0118] In a possible implementation, the merging module 304 is further configured to: obtain all text contents under the large model summary paragraph overview; determine the information blocks before and after the large model summary paragraph overview, and obtain all text contents in the information blocks before and after the large model summary paragraph overview; merge all text contents under the large model summary paragraph overview and all text contents in the information blocks before and after the large model summary paragraph overview to generate the target text.

[0119] In a possible implementation, the matching module 302 is specifically configured to: if it is recognized that the document does not include a flag bit, determine that the information block is a general type; extract all information blocks in the document; determine the target information block according to the correlation score between the information block and the input information.

[0120] In a possible implementation, the merging module 304 is further configured to: determine target information blocks within a set distance range, obtain all the text contents in the target information blocks within the set distance range, and all the text contents in the information blocks between the target information blocks within the set distance range; merge all the text contents in the target information blocks within the set distance range and all the text contents in the information blocks between the target information blocks within the set distance range to generate the target text.

[0121] In a possible implementation, the text generation device further includes a sharding module, configured to: extract relevant title levels in the document; divide the information in the document by the titles; isolate each title and the information content corresponding to the title through a title flag bit to obtain multiple shards; if the number of flags of a certain shard is greater than a set number of flags, then shard the information content; wherein each shard includes a title string.

[0122] In a possible implementation, the text generation device further includes an isolation module, configured to: extract relevant title levels in the document; divide the information in the document by the titles; isolate each title and the information content corresponding to the title through a title flag bit to obtain multiple shards; if the number of flags of a certain shard is greater than a set number of flags, then shard the information content; wherein each shard includes a title string.

[0123] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, it executes the steps of the text generation method described in the above method embodiment.

[0124] The computer program product of the text generation method provided by the embodiment of the present application includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the text generation method described in the above method embodiment. For details, reference can be made to the above method embodiment, which will not be elaborated here.

[0125] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules in each embodiment of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0127] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes. It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitations, elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article, or device comprising the said elements.

[0128] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the protection scope of this application. It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0129] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or replacements, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

1. A text generation method, characterized in that: include: Retrieving documents and information blocks related to the input information by setting a model; wherein each document includes one or more information blocks, and the information blocks are configured to be obtained by decomposing text in the document; According to the input information, matching a target information block related to the input information; determining one or more information blocks around the target information block; Merging the target information block and one or more information blocks around the target information block to generate a target text; Wherein, the target text is the output text corresponding to the input information; Wherein, the information block information includes: document meta information to which the information block belongs; Before matching the target information block related to the input information according to the input information, the method further includes: Determine the document to which each information block belongs according to the document meta information; Performing flag bit recognition on the content in the document, and classifying the information blocks according to the flag bit recognition result; wherein the information blocks in one document are of the same type; The step of matching a target information block related to the input information according to the input information includes: According to the matching modes corresponding to the input information and the types of the information blocks, the target information blocks related to the input information are matched.

2. The method according to claim 1, characterized in that in, The information block belongs to a type including a title enhancement type; The matching of the target information block related to the input information according to the matching mode corresponding to the input information and the type of the information block includes: If it is identified that the document includes a flag bit, and the flag bit is a title flag bit, determining that the information block is a title enhanced type; Retrieving all information blocks from the document; Matching the title flag corresponding to the input information, and determining the information block corresponding to the matched title flag as the target information block; The step of merging the target information block with one or more information blocks around the target information block to generate a target text includes: Obtain all text content under the target information block; Determine the information blocks before and after the target information block, and obtain all text contents in the information blocks before and after the target information block; All text contents under the target information block and all text contents in the information blocks before and after the target information block are merged to generate the target text.

3. The method according to claim 1, characterized in that in, The information block belongs to a type including a large model summary type; The matching of the target information block related to the input information according to the matching mode corresponding to the input information and the type of the information block includes: If it is identified that the document includes a flag bit, and the flag bit is a large model summary flag bit, then determining that the information block is a large model summary type; Retrieving all information blocks from the document; Matching the large model summary paragraph overview corresponding to the input information, and determining the information block corresponding to the matched large model summary paragraph overview as the target information block; The step of merging the target information block with one or more information blocks around the target information block to generate a target text includes: Obtain all text content under the summary paragraph of the large model; Determine the information blocks before and after the summary of the large model summary paragraph, and obtain all text contents in the information blocks before and after the summary of the large model summary paragraph; All text contents under the summary paragraph of the large model and all text contents in the information blocks before and after the summary paragraph of the large model are merged to generate the target text.

4. The method according to claim 1, characterized in that: in, The information block belongs to a type including a general type; The matching of the target information block related to the input information according to the matching mode corresponding to the input information and the type of the information block includes: If it is determined that the document does not include a flag bit, then the information block is determined to be a universal type; Retrieving all information blocks from the document; Determining a target information block according to a relevance score between the information block and the input information; The step of merging the target information block with one or more information blocks around the target information block to generate a target text includes: Determine the target information block within the set distance range, and obtain all text contents in the target information block within the set distance range, and all text contents in the information blocks between the target information blocks within the set distance range; All text contents in the target information blocks within the set distance range and all text contents in the information blocks between the target information blocks within the set distance range are merged to generate the target text.

5. The method according to any one of claims 1 to 4, characterized in that: Before retrieving documents and information blocks related to the input information by setting the model, the method further includes: Extract relevant heading levels in the document; Dividing the information in the document by the title; Isolate each title and the information content corresponding to the title through the title flag bit to obtain multiple fragments; If the number of flags of a certain fragment is greater than the set number of flags, the information content is fragmented; Each fragment includes a title string.

6. The method according to any one of claims 1 to 4, characterized in that: Before retrieving documents and information blocks related to the input information by setting the model, the method further includes: Divide the information content in the document into chunks using paragraph levels; Get the corresponding content overview of the information content in each block; Each content summary and the corresponding information content are isolated by the large model summary flag.

7. A text generation device, characterized in that: include: A retrieval module, used to retrieve documents and information blocks related to the input information by setting a model; wherein each document includes one or more information blocks, and the information blocks are configured to be obtained by decomposing the text in the document; A matching module, used for matching a target information block related to the input information according to the input information; A determination module, used to determine one or more information blocks around the target information block; A merging module, used for merging the target information block with one or more information blocks around the target information block to generate a target text; Wherein, the target text is the output text corresponding to the input information; the information block information includes: document meta information to which the information block belongs; A classification module is used to determine the document to which each information block belongs according to the document meta information; perform flag bit recognition on the content in the document, and classify the information block according to the flag bit recognition result; wherein the information blocks in one document are of the same type; The matching module is further used to match the target information block related to the input information according to the matching mode corresponding to the type of the input information and the information block.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of any method according to claim 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the methods described in claims 1 to 6 are executed.

Images