Entry generation method and apparatus, and electronic device

Abstract

The embodiment of the invention provides an entry generation method and device and electronic equipment, and the method comprises the steps: detecting a first document of a to-be-generated entry from a plurality of documents of a preset knowledge base, constructing a task queue according to the first document, and enabling each task in the task queue to correspond to one first document; a task processor is utilized to asynchronously read task data of a target task from the task queue, the task data of the target task comprises a second document, the second document is obtained by performing structured processing on a target first document, and the target first document is a first document corresponding to the target task; in response to determining that the second document can be converted into an entry according to the first abstract, extracting entry elements of a to-be-generated entry based on the first abstract and the second document; the first abstract is generated by abstracting the second document; and generating the entry to be generated according to the entry element. The accuracy of the generated entry content can be improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of Internet technology, and in particular to a method, device and electronic device for generating a term. Background Art

[0002] When presenting content to users (such as document content, chat message content), in order to facilitate the user's understanding of the content, for some difficult-to-understand or special-meaning words and phrases, a corresponding dictionary can be provided for the user to consult. In some application scenarios, these words and phrases can be associated with corresponding entries in the dictionary in the content, so that when the user reads these words and phrases, he can view the above entries through the above association to quickly understand the content. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device and electronic device for generating a term.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for generating a term, the method comprising: detecting a first document for a term to be generated from multiple documents of a preset knowledge base, constructing a task queue based on the first document, each task in the task queue corresponding to a first document; reading task data of a target task from the task queue, the task data of the target task including a second document, the second document being obtained by structured processing of a target first document, the target first document being the first document corresponding to the target task; in response to determining that the second document can be converted into a term based on a first summary, extracting term elements of the term to be generated based on the first summary and the second document; the first summary being obtained by summarizing the second document; and generating the term to be generated based on the term elements.

[0005] In a second aspect, an embodiment of the present disclosure provides a term generation device, which includes: a task queue construction unit, used to detect a first document of a term to be generated from multiple documents of a preset knowledge base, and to construct a task queue based on the first document, wherein each task in the task queue corresponds to a first document; a task data reading unit, used to read task data of a target task from the task queue, wherein the task data of the target task includes a second document, wherein the second document is obtained by structured processing of a target first document, and the target first document is the first document corresponding to the target task; a term element extraction unit, used to extract term elements of the term to be generated based on the first summary and the second document in response to determining that the second document can be converted into a term based on a first summary; the first summary is obtained by summarizing the second document; and a term generation unit, used to generate the term to be generated based on the term elements.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the first aspect above and various possible entry generation methods of the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the first aspect and various possible entry generation methods of the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the first aspect and various possible entry generation methods of the first aspect.

[0009] The entry generation method, device and electronic device provided in the present embodiment detect a first document of the entry to be generated from multiple documents of a preset knowledge base, build a task queue according to the first document, each task in the task queue corresponds to a first document; read task data of a target task from the task queue, the task data of the target task includes a second document, the second document is obtained by structured processing of the target first document, and the target first document is the first document corresponding to the target task; in response to determining that the second document can be converted into an entry according to a first summary, extract the entry elements of the entry to be generated based on the first summary and the second document; the first summary is obtained by summarizing the second document; and the entry to be generated is generated according to the entry elements. Since a task queue is constructed according to the first document detected from the preset knowledge base, task data is extracted from the task queue, and term elements are extracted according to the task data and terms are generated according to the term elements for asynchronous processing, the system load can be reduced; in addition, when it is determined that a document can generate a term according to the summary of the document, the term elements are extracted to generate the term. Since the document summary is obtained from different key sentences of the document, the accuracy of determining that the document can be converted into a term by the summary is high, which can improve the success rate of generating terms according to the document. In addition, when extracting term elements, if the document content is used to extract the term elements, for longer documents, the accuracy of the extracted term elements is low. After obtaining the summary of the document, the term elements are extracted based on the summary and the document, and more accurate term elements can be obtained. Therefore, extracting term elements according to the summary and the document can improve the accuracy of the generated term content. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 Schematic diagram of the process of the entry generation method provided in the embodiment of the present disclosure Figure 1 ;

[0012] Figure 2 Schematic diagram of the process of the entry generation method provided in the embodiment of the present disclosure Figure 2 ;

[0013] Figure 3 A structural block diagram of a device for generating a term provided in an embodiment of the present disclosure;

[0014] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0016] For a user group, a knowledge base of the user group is usually created and maintained so that each user in the user group can perform knowledge query. Many knowledge points in the user group knowledge base are manually created and maintained.

[0017] In the content for internal information exchange among the user group, for characters and words that match the entries in the dictionary, the definitions of the corresponding dictionary entries can be presented in the above content display interface according to user operations.

[0018] Currently, the dictionary entries are usually maintained by users manually creating and updating them. Therefore, for a new knowledge point of a user group, the user must manually add the knowledge point to the user group knowledge base and manually create the entry corresponding to the new knowledge in the dictionary. When an existing knowledge point in the knowledge base is updated, it is necessary to manually update the knowledge point in the knowledge base and manually maintain the entry corresponding to the knowledge point in the dictionary. Therefore, the maintenance cost of the knowledge base and dictionary entries within the user group is high.

[0019] The entry generation method, device and electronic device provided by the present disclosure obtain a target document in a knowledge base, format the target document to obtain a formatted target document, call an entry extraction model to extract a summary of the formatted target document, determine whether to generate an entry based on the summary extraction result, extract entry elements from the summary of the entry to be generated, and construct the entry based on the entry elements. The maintenance cost of the knowledge base and dictionary entries within the user group can be reduced, and the accuracy success rate of document generation can be improved.

[0020] Please refer to Figure 1 , Figure 1 A schematic flow chart of the term generation method provided by the present disclosure, such as Figure 1 As shown, the method comprises the following steps:

[0021] S101: Detecting a first document to generate a term from a plurality of documents in a preset knowledge base, and constructing a task queue according to the first document, wherein each task in the task queue corresponds to a first document.

[0022] S102: Reading task data of a target task from a task queue, the task data of the target task including a second document, the second document being obtained by structurally processing a target first document, the target first document being a first document corresponding to the target task.

[0023] In this embodiment, the execution subject (eg, server) of the term generation method may traverse the preset knowledge base once every preset period. During a traversal of the preset knowledge base, the first document for generating a term to be generated is detected from multiple documents in the preset knowledge base.

[0024] The preset knowledge base here may be, for example, a knowledge base corresponding to a user group. The knowledge base corresponding to a user group may store online documents generated by users in the user group.

[0025] For any document in the preset knowledge base, determine whether the document meets the preset conditions. If the preset conditions are met, the document is the first document to generate the entry. The above preset conditions include:

[0026] History did not convert this document into an entry;

[0027] The document was updated after history converted it to a term.

[0028] Specifically, the document unique identifier and creation time of the document can be obtained, the document entry conversion record can be searched, and the document unique identifier and creation time can be matched with the document entry conversion record. If the match is unsuccessful, it can be determined that the document has not been converted into an entry in history. The document entry conversion record can include the document unique identifier of each document that has been converted into an entry, and the time when each document was converted into an entry.

[0029] As an implementation method, if the creation time of the document is after the end time of the previous traversal and before the current traversal, then the document is the first document.

[0030] In this traversal, for each document, the document unique identifier, creation time and update time can be extracted. If the document is converted into an entry according to the history determined by the document unique identifier, and the update time is after the document is converted into an entry, then the document is the first document.

[0031] For each first document, the first document can be structured, for example, the first document is pre-processed by removing style information, punctuation marks, etc., retaining text content, links and structured information (such as hierarchical information such as title and body), and obtaining a structured first document. Each structured first document is saved as task data in a task queue.

[0032] The task processor may be used to asynchronously extract tasks from the task queue and send the extracted tasks to the execution body of the entry generation method.

[0033] Here, the first document will be extracted from the preset knowledge base, a task queue will be constructed according to the first document, and task data will be read from the task queue to extract the entry elements for asynchronous processing, so that the operations of detecting the first document from the preset knowledge base, constructing the task queue according to the first document, extracting the entry elements from the task queue, and generating entries according to the entry elements can be processed in different time periods. The load of the execution subject of the entry generation method can be reduced.

[0034] One or more task processors may be used to read one or more task data from the task queue, and then the one or more task processors may send the one or more task data to the execution subject in parallel for extracting entry elements.

[0035] S103: In response to determining that the second document can be converted into a term according to the first summary, extracting term elements of the term to be generated based on the first summary and the second document; the first summary is obtained by summarizing the second document.

[0036] In some implementations, the above-mentioned entry generation method further includes the following steps:

[0037] First, a first summary of the second document is generated.

[0038] Secondly, it is determined whether the second document can be converted into a term according to the first summary.

[0039] For example, the second document is segmented, and stop words are removed from the segmentation results to obtain multiple words or sentences. The importance of each word in the document is calculated based on word frequency-inverse document frequency. Important sentences or paragraphs are extracted from the second document as summary sentences, and multiple summary sentences are reorganized to make them more coherent and natural, thereby obtaining a first summary.

[0040] In some application scenarios, the second document may be input into a pre-trained summary generation model, and the summary generation model may generate a first summary of the second document.

[0041] The summary generation model can be obtained by training using multiple training samples, wherein the training samples include documents and annotated summaries. The summary generation model is trained by taking the documents as input and the annotated summaries as output truth values ​​to obtain the summary generation model.

[0042] After obtaining the first summary, it can be determined whether the second document can be converted into a term according to the first summary.

[0043] Illustratively, whether the second document meets a preset condition may be determined based on the first summary. If the second document meets the preset condition, it may be determined that the second document can be converted into a term.

[0044] The above-mentioned preset conditions may include, for example, one or more of the following:

[0045] The second document mainly focuses on the same topic or entity.

[0046] The subject or entity of the second document is a descriptive document for product services, processes, related systems, department teams, etc.

[0047] In response to determining that the second document can be converted into a term, term elements can be extracted based on the first summary and the second document.

[0048] Specifically, the term name may be generated according to the first summary, and then other term elements may be extracted from the second document.

[0049] S104: Generate the entry to be generated according to the entry elements.

[0050] After obtaining the term elements, the term elements can be combined according to a preset term structure to obtain the term.

[0051] In addition, the above method can also include updating the dictionary according to the above entry to be generated. For example, if the entry to be generated is a new entry, the new entry can be added to the dictionary; if the entry to be generated is an entry update corresponding to an existing entry, the entry update replaces the original entry, or serves as additional content to the original entry.

[0052] In this embodiment, a first document for generating an entry is detected from multiple documents of a preset knowledge base, a task queue is constructed based on the first document, and each task in the task queue corresponds to a first document; task data of a target task is read from the task queue, and the task data of the target task includes a second document, and the second document is obtained by structurally processing the target first document, and the target first document is the first document corresponding to the target task; in response to determining that the second document can be converted into an entry based on the first summary, the entry elements of the entry to be generated are extracted based on the first summary and the second document; the first summary is obtained by summarizing the second document; and the entry to be generated is generated based on the entry elements. A task queue is constructed based on the first document detected from the preset knowledge base, and the extraction of task data from the task queue, the extraction of entry elements based on the task data, and the generation of entries based on the entry elements can be processed asynchronously, which can reduce the system load; there is no need for users to manually maintain the knowledge base content and dictionary entries separately, which can reduce the maintenance cost of the knowledge base and dictionary entries within the user group. In addition, when determining that a document can generate an entry based on the document summary, the entry elements are extracted to generate the entry. Since the document summary is obtained from different key sentences in the document, the accuracy of determining that the document can be converted into an entry based on the summary is high, which can improve the success rate of generating entries based on the document. In addition, when extracting entry elements, if the document content is used to extract the entry elements, for longer documents, the accuracy of the extracted entry elements is low. After obtaining the summary of the document, the entry elements are extracted based on the summary and the document, and more accurate entry elements can be obtained. Therefore, extracting entry elements based on the summary and the document can improve the accuracy of the generated entry content.

[0053] In some embodiments, determining whether the second document can be converted into a term according to the first summary includes:

[0054] First, the first summary is input into a pre-trained first classification model, and the first classification model outputs a target category corresponding to the target document. The target category includes a first category or a second category. The first category indicates that the document can be converted into the first category of terms, and the second category indicates that the document cannot be converted into terms.

[0055] Secondly, if the target category is the first category, it is determined that the second document can be converted into a term.

[0056] The first classification model here can be any model that can classify text content. The first classification model can be pre-trained using multiple training samples so that the first classification model has the ability to classify text into the first category and the second category.

[0057] Specifically, a plurality of training samples may be used to train the first classification model, thereby obtaining the ability to classify text into the first category and the second category.

[0058] The above-mentioned multiple training samples may include text content and annotated categories of the text content. The annotated categories of the text content may include a first category or a second category. Multiple training samples may include text content annotated as the first category, and text content annotated as the second category. During training, the text content in the training samples may be used as the input of the first classification model, and the annotated categories corresponding to the training samples may be used as the output true value of the first classification model to train the first classification model, thereby obtaining a trained first classification model.

[0059] If the target category of the first summary output by the first classification model is the first category, it can be determined that the second document can be converted into a term.

[0060] In this embodiment, by inputting the first summary corresponding to the second document into the first classification model and determining whether the second document can generate an entry based on the output of the first classification model, the time used to determine whether the document can be converted into an entry can be reduced, thereby improving the efficiency of generating entries based on the document.

[0061] Please refer to Figure 2 , Figure 2 Schematic diagram of the process of the entry generation method provided by the present disclosure Figure 2 ,like Figure 2 As shown, the method comprises the following steps:

[0062] S201: Detecting a first document to generate a term from a plurality of documents in a preset knowledge base, and constructing a task queue according to the first document, wherein each task in the task queue corresponds to a first document.

[0063] S202: Reading task data of a target task from a task queue, the task data of the target task including a second document, the second document being obtained by structurally processing a target first document, the target first document being a first document corresponding to the target task.

[0064] The specific implementation of step S201 and step S202 can refer to Figure 1 The description of steps S101 and S102 of the illustrated embodiment will not be repeated here.

[0065] S203: In response to determining that the second document can be converted into a term according to the first summary, extracting a term name of the term to be generated from the first summary.

[0066] In this embodiment, the execution subject (e.g., server) of the term generation method can extract the first abstract from the second document. Then, it is determined whether the second document can be converted into a term based on the first abstract. For example, if it is determined that the second document can be converted into a term based on the first abstract, the term name of the term to be generated can be extracted from the first abstract.

[0067] The term to be generated here may be the term to which the second document is to be converted.

[0068] The entry name can be extracted from the first abstract. For example, the target word or target sentence with the highest word frequency-inverse document frequency can be extracted from multiple words and sentences based on the word frequency-inverse document frequency of each word and sentence in the first abstract, and the target word or target sentence can be used as the entry name.

[0069] In some embodiments, the above step S203 includes:

[0070] Generate first prompt information including a document title and a first summary, input the first prompt information into a text processing model, and the text processing model generates an entry name according to the first prompt information. The first prompt information is used to prompt the text processing model to generate an entry name according to the document title and the first summary.

[0071] In some application scenarios, the first prompt information may also include a rule for generating an entry name according to the document title and the first abstract. The rule may, for example, select the most frequently appearing word or sentence as the entry name.

[0072] The text processing model here can be any model that can process text content. In some application scenarios, the above text processing model can be a pre-trained model. The pre-trained training samples can include training texts and annotated term names corresponding to the training texts. The annotated term names are term names manually extracted from the training texts. After the text processing model is pre-trained using the above training texts, the text processing model has the ability to extract term names from the text.

[0073] After the first prompt information is input into the text processing model, the text processing model may output the term name extracted from the document title of the second document and the first abstract.

[0074] By extracting term names from the first abstract using a text processing model, the efficiency of extracting term names from the document title of the second document and the first abstract can be improved.

[0075] S204: Extracting the term definition of the term to be generated and resource information associated with the term to be generated from the second document.

[0076] After obtaining the term name, in order to obtain a more accurate term interpretation of the term to be generated, the term interpretation of the term to be generated may be determined from the entire second document.

[0077] For example, text contents associated with the term name may be obtained from different paragraphs of the second document, and then these text contents may be integrated, and the integration result may be used as the term interpretation of the term to be generated.

[0078] In some embodiments, the above step S204 includes the following steps:

[0079] The second document and the term name are input into the text processing model, and the term interpretation of the term to be generated is determined by the output content of the text processing model.

[0080] The text processing model can be any model with text processing capabilities, and the text processing model can obtain multiple text contents associated with the entry name from the second document. After obtaining the multiple text contents associated with the entry name, the entry definition can be generated based on the multiple text contents. For example, the multiple text contents can be abbreviated, rewritten, etc. to obtain the entry definition of the entry to be generated.

[0081] The text processing model can be any model with text processing capabilities. In some embodiments, the text processing model can determine the sentences associated with the entry name in each paragraph based on the text relevance between the entry name and each sentence. The sentences associated with the entry name in each paragraph are used as the text content associated with the entry name in the paragraph, thereby obtaining multiple text contents.

[0082] In some embodiments, the above-mentioned inputting the second document and the term name into the text processing model, and determining the term interpretation of the term to be generated from the output content of the text processing model, includes the following steps:

[0083] First, if the length of the second document is greater than a preset length threshold, the second document is split into a plurality of first text segments.

[0084] In these embodiments, the preset length threshold may be, for example, the maximum number of tokens that can be processed at one time by the text processing model. A token here is a basic unit of text in the field of text processing, and a token may be a word or a character.

[0085] When the length of the second document is greater than the above-mentioned preset length threshold, the second document can be divided into multiple first text segments. The length of each of the multiple first text segments is less than the above-mentioned preset length threshold. As an implementation method, the second document can be divided into multiple first text segments of equal length according to the preset length.

[0086] Secondly, a plurality of first text segments are respectively input into the text processing model, and the text processing model outputs the first text content associated with the above-mentioned entry name corresponding to each text segment.

[0087] A plurality of first text segments may be input into the text processing model at one time, and the text processing model outputs the first text content corresponding to each first text segment.

[0088] Alternatively, a plurality of first text segments may be input into the text processing model in different batches, and the text processing model may output the first text contents corresponding to the first text segments.

[0089] Finally, second prompt information including the contents of each first text is generated, and the second prompt information is input into a text processing model, and the text processing model generates a term definition according to the second prompt information.

[0090] After obtaining the first text contents corresponding to each first text segment, a second prompt information including a plurality of first text contents can be generated. The second prompt information is input into a text processing model. The text processing model can perform text processing such as rewriting, abbreviation or expansion on the plurality of first text contents to obtain the definition of the term.

[0091] The above-mentioned text processing model can also be various models with text processing capabilities. In some embodiments, the above-mentioned text processing model can be a pre-trained text processing model. The training process of the text processing model can include, for example, constructing a training sample, and using the training sample to train the text processing model. Each training sample can include a second prompt information and an annotation term interpretation. Each second prompt information includes a plurality of first text contents associated with the term name extracted from the sample document. The annotation term interpretation is manually generated based on the plurality of first text contents. The second prompt information in the training sample is used as the input of the text processing model, and the annotation term interpretation is used as the true value output of the text processing model, and the text processing model is trained to obtain a trained text processing model.

[0092] In these embodiments, for a long document, the document is divided into a plurality of short first text segments, and the first text content associated with the entry name is extracted from each of the plurality of first text segments; then, second prompt information including the plurality of first text contents is generated, and the second prompt information is input into a text processing model, and the text processing model processes the plurality of first text contents to obtain the entry interpretation of the entry to be generated. Thus, the interpretation of the entry to be generated can be effectively extracted from a long document.

[0093] In some embodiments, the above-mentioned inputting the second document and the term name into the text processing model, and determining the term interpretation of the term to be generated from the output content of the text processing model, includes the following steps:

[0094] First, if the length of the second document is greater than a preset length threshold, the second document is split into a plurality of second text segments; and the plurality of second text segments are arranged in the order in which they appear from front to back in the second document.

[0095] Secondly, starting from the first second text segment, the following loop operation is performed until the last second text segment: the second text content corresponding to the N-1th second text segment is obtained, and the second text content is merged with the Nth second text segment to obtain a merged text content; a third prompt information including the merged text content and the entry name is generated; the third prompt information is input into the text processing model, and the text processing model generates the second text content corresponding to the Nth second text segment according to the third prompt information; wherein N is greater than or equal to 1, and the third prompt information is used to prompt the generation of text content related to the entry name according to the merged text content.

[0096] Finally, the second text content output by the text processing model for the last second text segment is used as the term interpretation.

[0097] In these embodiments, the preset length threshold may be, for example, the maximum number of tokens that can be processed at one time by the text processing model. A token here is a basic unit of text in the field of text processing, and a token may be a word or a character.

[0098] When the length of the second document is greater than the preset length threshold, the second document may be divided into a plurality of second text segments, and the length of each of the plurality of second text segments may be less than the preset length threshold.

[0099] For each second text segment, determine the order in which the second text segment appears in the second document. Sort the plurality of second text segments according to the order in which the plurality of second text segments appear from front to back in the second document. In this way, each second text segment corresponds to a sorting number. The second text segment with a sorting number of 1 is the first text segment to appear in the second document.

[0100] Starting from the first second text segment, the second text content of the N-1th second text segment is obtained, where the second text content of the 0th second text segment may be empty. That is, for the first second text segment, the third prompt information including the first text segment itself and the entry name is generated. The third prompt information corresponding to the first text segment is input into the text processing model, and the above text processing model generates the second text content associated with the entry name from the first second text segment according to the third prompt information.

[0101] After obtaining the second text content corresponding to the first second text segment, the second text content corresponding to the first second text segment can be fused with the second second text segment to obtain the fused text content corresponding to the second second text segment. Generate a third prompt information including the above fused text content and the entry name, and input the above third prompt information into the text processing model. According to the third prompt information, the above text processing model generates the second text content associated with the entry name from the fused text content corresponding to the second second text segment. The second text content corresponds to the second second text segment. In this way, for the Kth (K is an integer greater than or equal to 2) second text segment, the fused text content corresponding to the Kth text segment carries the information related to the entry name of the first K-1 second text segments. Therefore, when the second text content of the Kth text segment is generated from the fused text content corresponding to the Kth second text segment, the text processing model can combine the first K-1 information related to the entry name to generate the second text content of the Kth second text segment with a higher relevance to the entry. The second text content corresponding to the last second text segment is used as the entry interpretation, so that the above interpretation can fully and accurately cover the content related to the entry in the second document. Therefore, the accuracy of the obtained interpretation is also high.

[0102] In some embodiments, the above step S204 includes the following steps:

[0103] First, candidate resource information in the second document is determined based on preset rules and a named entity recognition model.

[0104] Secondly, according to the context of the candidate resource information, target resource information related to the entry to be generated is determined from the candidate resource information.

[0105] Finally, extract the target resource information.

[0106] The user name may be extracted from the second document using a named entity recognition (NER) method.

[0107] The preset rules include, for example, link identification rules, image identification rules, etc. The link identification rules may include, for example, regular expressions for identifying links.

[0108] The above-mentioned image recognition rule, for example, may be a rule for using regular expressions to search for image links corresponding to one or more image formats (eg, uniform resource locators ending with ".jpg", ".png", ".gif", etc.).

[0109] For each candidate resource, the relevance of the candidate resource to the entry to be generated is determined in combination with the context in which the candidate resource is located, such as the L sentences preceding the candidate resource, and / or the L sentences following the candidate resource. Specifically, the relevance of the L sentences preceding the candidate resource, and / or the L sentences following the candidate resource, and the entry name (or entry topic) of the entry to be generated can be calculated. If the relevance is greater than a preset relevance threshold, the candidate resource can be considered to be related to the entry to be generated. The candidate resource can be used as a target resource.

[0110] If the calculated relevance of the L sentences before the candidate resource and the L sentences after the candidate resource to the entry name (or entry topic) to be generated is less than the preset relevance threshold, it can be considered that the candidate resource is irrelevant to the entry to be generated.

[0111] The target resource may be extracted from the second document.

[0112] The target resources include one or more of the following:

[0113] Users associated with the entry to be generated, groups associated with the entry to be generated, pictures associated with the entry to be generated, associated documents or related links.

[0114] S205: Generate the above-mentioned entry to be generated according to the entry name, the entry definition and the resource information.

[0115] The entry name, the entry definition and the resource information of the target resource can be filled into the corresponding position of the entry template to obtain the entry to be generated.

[0116] In this embodiment, it is described that the term name is extracted according to the document synchronization of the second document and the first summary, the term interpretation and resource information are extracted from the second document, and the content of the term is generated according to the above term name, term interpretation and resource information. The above scheme can be applied to converting documents of different formats into terms, and can improve the efficiency of converting documents of different formats into terms.

[0117] exist Figure 2 In some implementations of the entry generation method shown, the method further includes:

[0118] Find the alias of the term to be generated from the second document; or,

[0119] Generate an entry alias for the entry to be generated according to the entry name.

[0120] In some application scenarios, fourth prompt information including the term name and the second document may be generated, and the fourth prompt information may also include an alias extraction rule for extracting the alias from the second document. The alias extraction rule may, for example, include taking a word in the second document that is related to the term name and has a token quantity less than a preset quantity threshold as the term alias.

[0121] The fourth prompt information is input into the text processing model, and the text processing model can output the term alias.

[0122] In some application scenarios, the entry name can be compressed to obtain the entry alias of the entry to be generated. As an implementation method, a fifth prompt information including the entry name can be generated, and the fifth prompt information can be input into the text processing model. The fourth text processing model compresses the entry name according to the fifth prompt information to obtain the alias of the entry to be generated. The fifth prompt information can include compression rules. The compression rules can include, for example, extracting keywords from the entry name and using the keywords as aliases; or extracting the first letters corresponding to the first T words in the entry name, and concatenating the first letters corresponding to the first T words to obtain the alias of the entry to be generated, etc.

[0123] The above step S205 also includes generating the above entry to be generated according to the entry name, entry definition, resource information and alias.

[0124] In some application scenarios, the length of the entry name is long, and users may use aliases to interact when exchanging information. When the document content or interactive information includes an alias, the above entry can be triggered. Therefore, the usage rate of the entry can be improved.

[0125] The following is an illustration of an application scenario. For user group A, the knowledge base of user group A can be traversed at preset intervals, and multiple first documents A1 to A10 for entries to be generated are detected during the traversal process. The first documents A1 to A10 are structured to obtain structured first documents corresponding to the first documents A1 to A10. The structured first documents corresponding to the above-mentioned first documents A1 to A10 are stored in a task queue. The task queue may include 10 tasks. Each task corresponds to one of the above-mentioned structured first documents.

[0126] The task processor is called asynchronously, and when the text processing model is idle, for example, the task processor extracts the second document A1' corresponding to the target first document A1 from the task queue. The second document A1' is input into the text processing model to obtain the first summary corresponding to the second document A1'. The above-mentioned first summary is then input into the first classification model, and the first classification model outputs the target category of the second document. When the target category is the first category, the document title and the first summary of the second document are input into the text processing model, and the text processing model outputs the entry name of the entry to be generated. Then, the entry name and the second document are input into the text processing model, and the text processing model outputs the entry interpretation of the entry to be generated. Then, according to the NER method or preset rules, resource information such as associated users, associated documents, and associated pictures are extracted from the second document A1'. The entry W1 to be generated is generated according to the entry name, entry, and resource information.

[0127] Corresponding to the entry generation method of the above embodiment, Figure 3 This is a structural block diagram of a term generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 3 The device 30 includes: a task queue construction unit 301, a task data reading unit 302, a term element extraction unit 303 and a term generation unit 304.

[0128] The task queue construction unit 301 is used to detect a first document to generate a term from a plurality of documents in a preset knowledge base, and to construct a task queue according to the first document, wherein each task in the task queue corresponds to a first document;

[0129] The task data reading unit 302 is used to read the task data of the target task from the task queue, where the task data of the target task includes a second document, where the second document is obtained by performing structured processing on the target first document, where the target first document is the first document corresponding to the target task;

[0130] The term element extraction unit 303 is used to extract the term elements of the term to be generated based on the first summary and the second document in response to determining that the second document can be converted into a term according to the first summary; the first summary is obtained by summarizing the second document;

[0131] The term generation unit 304 is used to generate a term to be generated according to the term elements.

[0132] In some embodiments, the apparatus 30 further includes a summary generating unit (not shown in the figure) and a determining unit (not shown in the figure).

[0133] A summary generating unit, configured to generate a first summary of the second document;

[0134] A determination unit is used to determine whether the second document can be converted into a term according to the first summary.

[0135] In some embodiments, the determining unit is further used to input the first summary into a pre-trained first classification model, and the first classification model outputs a target category corresponding to the target document, the target category includes a first category or a second category, the first category indicates that the document can be converted into a first category of a term, and the second category indicates that the document cannot be converted into a term;

[0136] If the target category is the first category, it is determined that the second document can be converted into a term.

[0137] In some embodiments, the term element extraction unit 303 is further configured to:

[0138] Extracting the entry name of the entry to be generated according to the document title of the second document and the first abstract;

[0139] Extracting the term definition of the term to be generated and resource information associated with the term to be generated from the second document;

[0140] The term generation unit 304 is further used to generate a term to be generated according to the term name, term definition and resource information.

[0141] In some embodiments, the term element extraction unit 303 is further configured to:

[0142] Generate first prompt information including a document title and a first summary, input the first prompt information into a first text processing model, and generate an entry name according to the first prompt information by the first text processing model. The first prompt information is used to prompt the first text processing model to generate an entry name according to the document title and the first summary.

[0143] In some embodiments, the term element extraction unit 303 is further configured to:

[0144] The second document and the term name are input into the text processing model, and the term interpretation of the term to be generated is determined by the output content of the text processing model.

[0145] In some embodiments, the term element extraction unit 303 is further configured to:

[0146] If the length of the second document is greater than a preset length threshold, splitting the second document into a plurality of first text segments;

[0147] Inputting a plurality of first text segments into a text processing model respectively, and outputting from the text processing model first text contents associated with the entry names corresponding to the respective text segments;

[0148] Second prompt information including the contents of each first text is generated, the second prompt information is input into a text processing model, and the text processing model generates a term definition according to the second prompt information.

[0149] In some embodiments, the term element extraction unit 303 is further configured to:

[0150] If the length of the second document is greater than a preset length threshold, splitting the second document into a plurality of second text segments; arranging the plurality of second text segments in the order in which the plurality of second text segments appear from front to back in the second document;

[0151] Starting from the first second text segment, perform the following loop operations until the last second text segment:

[0152] Obtain the second text content corresponding to the N-1th second text segment, and fuse the second text content with the Nth second text segment to obtain fused text content; generate third prompt information including the fused text content and the entry name; input the third prompt information into the text processing model, and the text processing model generates the second text content corresponding to the Nth second text segment according to the third prompt information; wherein N is an integer greater than or equal to 1, and the third prompt information is used to prompt the generation of text content related to the entry name according to the fused text content;

[0153] The second text content output by the text processing model for the last second text segment is used as the term definition.

[0154] In some embodiments, the term element extraction unit 303 is further used to: determine candidate resource information in the target first document based on preset rules and a named entity recognition method;

[0155] Determining target resource information related to the entry to be generated from the candidate resource information according to the context of the candidate resource information;

[0156] Extract target resource information;

[0157] In some embodiments, the target resource information includes one or more of the following:

[0158] Users associated with the entry to be generated, groups associated with the entry to be generated, pictures associated with the entry to be generated, associated documents or related links.

[0159] In some embodiments, the term element extraction unit 303 is further configured to:

[0160] Searching the second document for an alias of the term to be generated; or

[0161] Generate an entry alias for the entry to be generated according to the entry name.

[0162] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0163] refer to Figure 4 , which shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present disclosure, and the electronic device 400 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0164] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 to a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0165] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0166] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program (computer execution instruction) is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0167] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0168] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0169] The computer-readable medium carries one or more programs (computer-executable instructions). When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0170] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0171] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0173] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0174] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0176] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0177] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for generating a term, include: Detecting a first document to generate a term from a plurality of documents in a preset knowledge base, and constructing a task queue according to the first document, wherein each task in the task queue corresponds to a first document; Reading task data of a target task from the task queue, wherein the task data of the target task includes a second document, wherein the second document is obtained by performing structured processing on a target first document, and the target first document is a first document corresponding to the target task; In response to determining that the second document can be converted into a term according to the first summary, extracting term elements of a term to be generated based on the first summary and the second document; The first summary is obtained by summarizing the second document; The entry to be generated is generated according to the entry element.

2. The method according to claim 1, It is characterized in that The method further comprises: generating a first summary of the second document; It is determined whether the second document can be converted into a term according to the first summary.

3. The method according to claim 2, It is characterized in that The step of determining whether the second document can be converted into a term according to the first summary comprises: Inputting the first summary into a pre-trained first classification model, and having the first classification model output a target category corresponding to the target document, wherein the target category includes a first category or a second category, wherein the first category indicates that the document can be converted into a first category of a term, and the second category indicates that the document cannot be converted into a term; If the target category is the first category, it is determined that the second document can be converted into a term.

4. The method according to claim 1, It is characterized in that In response to determining that the second document can be converted into a term according to the first summary, extracting term elements of the term to be generated based on the first summary and the second document includes: Extracting the entry name of the entry to be generated according to the document title of the second document and the first abstract; Extracting the term definition of the term to be generated and resource information associated with the term to be generated from the second document; The step of generating a term according to the term element comprises: The entry to be generated is generated according to the entry name, the entry definition and the resource information.

5. The method according to claim 4, It is characterized in that The step of extracting the entry name of the entry to be generated according to the document title of the second document and the first abstract includes: Generate first prompt information including the document title and the first summary, input the first prompt information into a text processing model, and allow the text processing model to generate an entry name according to the first prompt information, wherein the first prompt information is used to prompt the first text processing model to generate an entry name according to the document title and the first summary.

6. The method according to claim 5, It is characterized in that The step of extracting the term interpretation of the term to be generated from the second document includes: The second document and the term name are input into the text processing model, and the term interpretation of the term to be generated is determined based on the output content of the text processing model.

7. The method according to claim 6, It is characterized in that The step of inputting the second document and the term name into the text processing model, and determining the term interpretation of the term to be generated based on the output content of the text processing model, includes: If the length of the second document is greater than a preset length threshold, splitting the second document into a plurality of first text segments; Inputting a plurality of first text segments into the text processing model respectively, and outputting from the text processing model first text contents associated with the entry name corresponding to each text segment; Second prompt information including each of the first text contents is generated, the second prompt information is input into the text processing model, and the text processing model generates the term definition according to the second prompt information.

8. The method according to claim 6, It is characterized in that The step of inputting the second document and the term name into the text processing model, and determining the term interpretation of the term to be generated based on the output content of the text processing model, includes: If the length of the second document is greater than a preset length threshold, splitting the second document into a plurality of second text segments; arranging the plurality of second text segments in the order in which the plurality of second text segments appear from front to back in the second document; Starting from the first second text segment, perform the following loop operations until the last second text segment: Obtain the second text content corresponding to the N-1th second text segment, and fuse the second text content with the Nth second text segment to obtain fused text content; generate third prompt information including the fused text content and the entry name; input the third prompt information into the text processing model, and the text processing model generates the second text content corresponding to the Nth second text segment according to the third prompt information; wherein N is an integer greater than or equal to 1, and the third prompt information is used to prompt the generation of text content related to the entry name according to the fused text content; The second text content output by the text processing model for the last second text segment is used as the term definition.

9. The method according to claim 4, It is characterized in that The step of extracting the definition of the term to be generated and resource information associated with the term to be generated from the second document includes: Determine candidate resource information in the target first document based on preset rules and a named entity recognition method; Determining target resource information related to the entry to be generated from the candidate resource information according to the context of the candidate resource information; The target resource information is extracted.

10. The method according to claim 9, It is characterized in that The target resource information includes one or more of the following: A user associated with the entry to be generated, a group associated with the entry to be generated, a picture associated with the entry to be generated, an associated document or a related link.

11. The method according to any one of claims 4 to 9, It is characterized in that The method further comprises: Searching the second document for an alias of the entry to be generated; or Generate an entry alias for the entry to be generated according to the entry name.

12. A device for generating a term, include: A task queue construction unit, used to detect a first document to generate a term from a plurality of documents in a preset knowledge base, and to construct a task queue according to the first document, wherein each task in the task queue corresponds to a first document; A task data reading unit, used to asynchronously read the task data of the target task from the task queue using a task processor, wherein the task data of the target task includes a second document, wherein the second document is obtained by structurally processing the target first document, and the target first document is a first document corresponding to the target task; a term element extraction unit, configured to extract term elements of a term to be generated based on the first summary and the second document in response to determining that the second document can be converted into a term according to the first summary; The first summary is obtained by summarizing the second document; A term generation unit is used to generate the term to be generated according to the term element.

13. An electronic device, It is characterized in that include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images