Document tagging method and system based on entity enhancement and multi-granularity fusion

Abstract

The application provides a document labeling method and system based on entity enhancement and multi-granularity fusion, and relates to the technical field of artificial intelligence. The method comprises the following steps: cutting a to-be-processed document into multiple text blocks, and performing entity extraction on each text block to obtain a first entity list of each text block; based on the text blocks and the first entity list, text block-level labeling, text cluster-level labeling and full-text abstract-level labeling are respectively performed to generate text block labels, text cluster labels and full-text abstract labels; and the text block labels, the text cluster labels and the full-text abstract labels are fused to obtain labels of the to-be-processed document. The application improves the comprehensiveness and accuracy of document label generation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a document labeling method and system based on entity enhancement and multi-granularity fusion. BACKGROUND

[0002] With the rapid development of big data technology, the data of information is growing explosively. With the continuous and extensive application of information technology in work, a large amount of document data is accumulated. In order to quickly search and analyze massive data, or use document data for model training, it is usually necessary to label the document to generate the label of the document. The existing document labeling method mostly depends on a single method to generate local or coarse-grained labels, and the label generation credibility is insufficient. SUMMARY

[0003] Therefore, the present application aims to provide a document labeling method and system based on entity enhancement and multi-granularity fusion, so as to improve the comprehensiveness and accuracy of document label generation.

[0004] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0005] In a first aspect, the present application provides a document labeling method based on entity enhancement and multi-granularity fusion, comprising: dividing a to-be-processed document into a plurality of text blocks, and performing entity extraction on each text block to obtain a first entity list of each text block; based on the text blocks and the first entity list, respectively performing text block level labeling, text cluster level labeling and full-text abstract level labeling to generate text block labels, text cluster labels and full-text abstract labels; and fusing the text block labels, the text cluster labels and the full-text abstract labels to obtain labels of the to-be-processed document.

[0006] Optionally, based on the text block and the first entity list, text block level tagging, text cluster level tagging and full text summary level tagging are respectively performed to generate text block labels, text cluster labels and full text summary labels, including: based on the text block and the first entity list, generating a first system label and a first LLM label of each text block, and fusing the first system label and the first LLM label to obtain a text block label of each text block and a corresponding first label score; clustering the text blocks to obtain a plurality of text clusters, and based on the first entity list, generating a second entity list corresponding to the text clusters, and based on the text clusters and the second entity list, generating a second system label and a second LLM label of each text cluster, and fusing the second system label and the second LLM label to obtain a text cluster label of each text cluster and a corresponding second label score; based on the text block, generating a full text summary, and merging and deduplicating the first entity list to generate a full text entity list, and based on the full text summary and the full text entity list, generating a third system label and a third LLM label corresponding to the full text summary, and fusing the third system label and the third LLM label to obtain a full text summary label and a corresponding third label score.

[0007] Optionally, based on the text block and the first entity list, a first system label and a first LLM label of each text block are generated, and the first system label and the first LLM label are fused to obtain a text block label of each text block and a corresponding first label score, including: based on the similarity between the vector of each text block and the label vector in the pre-constructed label library, a first system label of each text block is obtained; based on the text block and the corresponding first entity list, a prompt word of a large language model is generated, and the prompt word is input into the large language model to obtain a first LLM label of the text block and a confidence of each first LLM label; the normalization scores of the first system label and the first LLM label are calculated, and the first system label and the first LLM label are deduplicated based on the normalization scores to obtain the text block label of each text block; wherein the normalization score of the first system label is the similarity between the first system label and the label vector in the label library multiplied by a preset adaptation coefficient; the normalization score of the first LLM label is the confidence of the first LLM label multiplied by the accuracy coefficient of the large language model; based on the position of the text block in the document to be processed, a position weight corresponding to each text block is determined; the normalization score of the text block label of each text block is multiplied by the corresponding position weight to obtain a first label score of each text block label.

[0008] Optionally, the text blocks are clustered to obtain a plurality of text clusters, and a second entity list corresponding to the text clusters is generated based on the first entity list, including: processing the vector of each text block by using an HDBSCAN algorithm to obtain a cluster label of each text block; grouping the text blocks based on the cluster label to obtain a plurality of text clusters; determining a cluster center of the text cluster based on the average value of the vector of each text block in the text cluster, and selecting a first preset number of text blocks based on the similarity between each text block and the cluster center to obtain a cluster-level text of the text cluster; and merging and deduplicating the first entity list of the text block corresponding to the cluster-level text to obtain the second entity list corresponding to the text cluster.

[0009] Optionally, the second system label and the second LLM label are fused to obtain a text cluster label of each text cluster and a corresponding second label score, including: calculating the normalized score of the second system label and the second LLM label, and deduplicating the second system label and the second LLM label based on the normalized score to obtain the text cluster label of each text cluster; determining a cluster weight of each text cluster based on the cluster size of the text cluster and the average similarity between the text block in the text cluster and the cluster center; and multiplying the normalized score of the text cluster label of each text cluster by the corresponding cluster weight to obtain the second label score of each text cluster label.

[0010] Optionally, a full-text summary is generated based on the text blocks, including: dividing the text blocks into a plurality of dynamic semantic groups based on the similarity between the text blocks; splicing the text blocks in the dynamic semantic group, and inputting the spliced text into a large language model to generate a group-level summary and an intra-group core topic of each dynamic semantic group; merging the group-level summaries of each dynamic semantic group, and inputting the merged text and the intra-group core topic of each dynamic semantic group into the large language model to generate a full-text summary.

[0011] Optionally, a third system label and a third LLM label corresponding to the full-text summary are generated based on the full-text summary and the full-text entity list, and the third system label and the third LLM label are fused to obtain a full-text summary label and a corresponding third label score, including: obtaining the third system label corresponding to the full-text summary based on the similarity between the vector of the full-text summary and the label vector in the pre-constructed label library; inputting the full-text summary, the intra-group core topic of each dynamic semantic group, and the full-text entity list into the large language model to obtain the third LLM label corresponding to the full-text summary and the confidence of each third LLM label; calculating the normalized score of the third system label and the third LLM label, and deduplicating the third system label and the third LLM label based on the normalized score to obtain the full-text summary label; and multiplying the normalized score of each full-text summary label by a preset coefficient to obtain the third label score of each full-text summary label.

[0012] Optionally, the text block label, the text cluster label and the full-text summary label are fused to obtain the label of the document to be processed, including: obtaining the label text, score information and source information of the text block label, the text cluster label and the full-text summary label, and performing standardization processing on the text block label, the text cluster label and the full-text summary label; grouping the text block label, the text cluster label and the full-text summary label based on the label text to obtain a label group; wherein the label group includes one or more labels with the same label text, and the label is one or more of the text block label, the text cluster label and the full-text summary label; for each label group, calculating the weighted average score of the labels in the label group based on the preset weight coefficient, and determining the diversity bonus of the labels in the label group based on the source information of the labels; determining the comprehensive score of the labels in the label group based on the weighted average score and the diversity bonus of the labels in the label group, and merging the labels in the label group to obtain a merged label; selecting a second preset number of merged labels as preliminary screening labels based on the comprehensive score of the merged labels; eliminating synonymous labels based on the similarity between the preliminary screening labels to obtain a third number of labels as the label of the document to be processed.

[0013] In a second aspect, the present application provides a document labeling system based on entity enhancement and multi-granularity fusion, comprising: an entity extraction module, configured to split a document to be processed into a plurality of text blocks, and perform entity extraction on each text block to obtain a first entity list of each text block; a label generation module, configured to perform text block level labeling, text cluster level labeling and full-text summary level labeling based on the text blocks and the first entity list, to generate text block labels, text cluster labels and full-text summary labels; and a label fusion module, configured to fuse the text block labels, the text cluster labels and the full-text summary labels to obtain the label of the document to be processed.

[0014] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method of any one of the first aspect.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps of the method of any one of the first aspect.

[0016] The present application has the following beneficial effects:

[0017] The application provides the above-mentioned document tagging method and system based on entity enhancement and multi-granularity fusion, first, the to-be-processed document is divided into multiple text blocks, and entity extraction is performed on each text block to obtain a first entity list of each text block; then, based on the text block and the first entity list, text block level tagging, text cluster level tagging and full-text abstract level tagging are respectively performed to generate text block labels, text cluster labels and full-text abstract labels; finally, the text block labels, the text cluster labels and the full-text abstract labels are fused to obtain the labels of the to-be-processed document. In the above method, the first entity list corresponding to each text block is obtained through entity extraction, then the first entity list and the text block are used to obtain the text block labels, the text cluster labels and the full-text abstract labels through parallel processing of the three routes, and finally, the text block labels, the text cluster labels and the full-text abstract labels are fused, and all entities are fused into the label set to obtain the final labels of the to-be-processed document. The above method generates the labels of the document from three aspects of the text block, the text cluster and the full-text abstract, and fuses the entities of the document to obtain the final labels, thereby improving the comprehensiveness and accuracy of the document label generation.

[0018] Other features and advantages of the present application will be set forth in the descriptions below, and in part will be apparent from the description, or can be learned by practice of the present application. The purposes and other advantages of the present application will be realized and attained by the structures particularly pointed out in the description, the claims and the drawings.

[0019] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following will be described in detail in combination with the attached drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0021] Figure 1 A flowchart of a document tagging method based on entity enhancement and multi-granularity fusion provided by an embodiment of the present application is shown in the figure;

[0022] Figure 2 A flowchart of another document tagging method based on entity enhancement and multi-granularity fusion provided by an embodiment of the present application is shown in the figure;

[0023] Figure 3 A structure schematic diagram of a document tagging system based on entity enhancement and multi-granularity fusion provided by an embodiment of the present application is shown in the figure;

[0024] Figure 4 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in detail with reference to the drawings. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of the present application.

[0026] At present, most of the existing document labeling methods rely on a single method to generate local or coarse-grained labels, and the label generation credibility is insufficient.

[0027] Based on this, the document labeling method and system based on entity enhancement and multi-granularity fusion provided by the embodiments of the present application can improve the comprehensiveness and accuracy of document label generation.

[0028] To facilitate the understanding of the present embodiment, first, a document labeling method based on entity enhancement and multi-granularity fusion disclosed by the embodiments of the present application will be described in detail. The method can be executed by an electronic device, such as a smart phone, a computer, a tablet computer, etc. Referring to Figure 1 The flowchart of the document labeling method based on entity enhancement and multi-granularity fusion shown in the figure shows that the method mainly includes the following steps S101 to S103:

[0029] Step S101: The document to be processed is divided into multiple text blocks, and entity extraction is performed on each text block to obtain a first entity list of each text block.

[0030] In an implementation, a pre-trained industry named entity recognition (NER) model or a large language model (LLM) is used to perform entity extraction operation on each text block (chunk). In the entity extraction process, the extracted entities in a single chunk are de-duplicated to finally form a first entity list of all chunks.

[0031] Step S102: Based on the text blocks and the first entity list, text block level labeling, text cluster level labeling and full text summary level labeling are respectively performed to generate text block labels, text cluster labels and full text summary labels.

[0032] In an implementation, the text blocks and the corresponding first entity list are used to independently generate labels by three routes, i.e. Route 1: generating text block labels by text block level labeling; Route 2: generating text cluster labels by text cluster level labeling; Route 3: generating full text summary labels by full text summary level labeling.

[0033] Step S103: fusing the text block label, the text cluster label and the full-text summary label to obtain the label of the document to be processed.

[0034] In an embodiment, the text block label, the text cluster label and the full-text summary label generated by the above three routes are fused and optimized to obtain the final label.

[0035] The above document labeling method based on entity enhancement and multi-granularity fusion provided by the embodiments of the present application first performs entity extraction on each text block to obtain a corresponding first entity list, then uses the first entity list and the text block to obtain the text block label, the text cluster label and the full-text summary label by adopting three routes in parallel, and finally fuses the text block label, the text cluster label and the full-text summary label, and fuses all entities into the label set to obtain the final label of the document to be processed. The above method generates the label of the document from three aspects of the text block, the text cluster and the full-text summary, and fuses the entities of the document to obtain the final label, thereby improving the comprehensiveness and accuracy of the generation of the document label.

[0036] In an embodiment, for the foregoing step S102, i.e., when the text block label, the text cluster label and the full-text summary label are generated by performing text block level labeling, text cluster level labeling and full-text summary level labeling based on the text block and the first entity list, the following three routes can be adopted, including but not limited to the following three routes:

[0037] Route 1: based on the text block and the first entity list, generating the first system label and the first LLM label of each text block, and fusing the first system label and the first LLM label to obtain the text block label of each text block and the corresponding first label score.

[0038] In a specific implementation, first, the first system label of each text block is obtained based on the similarity between the vector of each text block and the label vector in the pre-constructed label library.

[0039] Specifically, the labels related to the current chunk text are retrieved from the pre-established label library, and the label vectors in the label library are pre-loaded. First, the vector of the current chunk text is generated; then the cosine similarity between the vector and each label vector in the label library is calculated to obtain the similarity; finally, according to the order from high to low of the similarity, the top 10 labels are screened out as the system label (i.e., the first system label) of each text block, and these labels are standardized and supplemented with metadata such as source information “sys” and route information “1”.

[0040] Secondly, prompt words are generated based on the text block and the corresponding first entity list, and the prompt words are input into the large language model to obtain the first LLM label of the text block and the confidence of each first LLM label.

[0041] Specifically, the LLM is used to generate labels for the chunk text. First, a specific prompt word is input into the large language model (LLM), which is required to generate a specified number of the most relevant labels for the given chunk text in combination with the first entity list of the current chunk. Each label has a length of 1-10 characters, and the domain-specific terminology is preferred. The output format is a JSON array containing "text" and "confidence" (0-1). The LLM returns the first LLM label of the text block and the corresponding confidence, and the generated labels are standardized, supplemented with metadata such as "llm" and "1" route information.

[0042] Next, the normalized scores of the first system label and the first LLM label are calculated, and the first system label and the first LLM label are de-duplicated based on the normalized scores to obtain the text block label of each text block. The normalized score of the first system label is the similarity between the first system label and the label vector in the label library multiplied by a preset adaptation coefficient. The normalized score of the first LLM label is the confidence of the first LLM label multiplied by the accuracy coefficient of the large language model.

[0043] Specifically, the first system label and the first LLM label generated above are fused. First, the first system label and the first LLM label are normalized scored, i.e., the normalized scores of the first system label and the first LLM label are calculated. For the first system label, the similarity between the first system label and the label vector in the label library is multiplied by a preset adaptation coefficient (previously set according to the domain, for example, the general domain can be 0.9) to obtain the normalized score. For the first LLM label, the confidence of the first LLM label is multiplied by the accuracy coefficient of the large language model (e.g., 0.85) to obtain the normalized score. Then, the first system label and the first LLM label are de-duplicated and conflict processed. The first system label and the first LLM label are grouped according to the label text. For the first system label and the first LLM label of the same text, the label with higher normalized score is retained. Finally, after de-duplication and conflict processing, the text block label of each text block is obtained.

[0044] Then, based on the position of the text block in the document to be processed, the position weight corresponding to each text block is determined.

[0045] Finally, the normalized score of the text block label of each text block is multiplied by the corresponding position weight to obtain the first label score of each text block label.

[0046] Specifically, according to the position of the text block in the document to be processed, the position weight is calculated in a manner similar to a normal distribution, for example, the weights of the first and last chunks are higher, and the weights of the middle chunks gradually decrease; then the normalized score of each text block label is multiplied by the corresponding position weight to obtain the final score of each text block label (i.e., the first label score), and the text block labels are sorted according to the first label score.

[0047] Route 2: clustering the text blocks to obtain multiple text clusters, generating a second entity list corresponding to the text cluster based on the first entity list, and generating a second system label and a second LLM label for each text cluster based on the text cluster and the second entity list, and fusing the second system label and the second LLM label to obtain a text cluster label for each text cluster and a corresponding second label score.

[0048] In specific implementation, first, the text blocks are adaptively clustered, which can be performed in the following manner, including but not limited to: using the HDBSCAN algorithm to process the vector of each text block to obtain the cluster label of each text block; grouping the text blocks based on the cluster label to obtain multiple text clusters; determining the cluster center of the text cluster based on the average value of the vector of each text block in the text cluster, and selecting a first preset number of text blocks based on the similarity between each text block and the cluster center to obtain the cluster-level text of the text cluster; merging and deduplicating the first entity list of the text blocks corresponding to the cluster-level text to obtain the second entity list corresponding to the text cluster.

[0049] Specifically, the HDBSCAN algorithm is used to adaptively cluster the chunks, the minimum cluster size and the minimum number of core points are set, and the cosine distance is used as the measurement method. The vector of the chunk is processed by the HDBSCAN algorithm to obtain the cluster label of each chunk. Then, the chunks are grouped according to the cluster label to form different text clusters, each text cluster contains the text, entity, index and vector of all chunks in the cluster, etc., wherein the cluster with a label of -1 is regarded as a noise cluster.

[0050] For non-noise clusters, cluster-level text is generated. First, the average value of the vectors of all chunks in the text cluster is calculated as the cluster center, then the cosine similarity between each chunk and the cluster center is calculated, and the top 3 chunks with the highest cosine similarity are selected, and the text of the selected chunks is combined as the cluster-level text; finally, the first entity list of the cluster-level text is merged and deduplicated to obtain the cluster-level entity list (i.e., the second entity list corresponding to the text cluster).

[0051] Then, the second system label and the second LLM label of each text cluster are obtained by searching the system label and generating the label by using the LLM, based on the cluster-level text and the cluster-level entity list. It should be noted that the method for generating the second system label and the second LLM label in the embodiment is similar to the method for generating the first system label and the first LLM label in the aforementioned route 1, and only the number of generated labels is different, which will not be described here.

[0052] Finally, the second system label and the second LLM label are fused to obtain the text cluster label of each text cluster and the corresponding second label score. Specifically, the following methods can be used, but are not limited to:

[0053] (1) The normalized scores of the second system label and the second LLM label are calculated, and the second system label and the second LLM label are de-duplicated based on the normalized scores to obtain the text cluster label of each text cluster. Specifically, the method for calculating the normalized score and de-duplicating the label in the embodiment is the same as that in route 1, which can be referred to the aforementioned embodiment.

[0054] (2) The cluster weight of each text cluster is determined based on the cluster size of the text cluster and the average similarity between the text blocks in the text cluster and the cluster center.

[0055] (3) The second label score of each text cluster label is obtained by multiplying the normalized score of the text cluster label of each text cluster by the corresponding cluster weight.

[0056] Specifically, a cluster weight is assigned to each text cluster, and the cluster weight is the product of the cluster size and the average similarity of the text cluster. The cluster size is not more than 10. The average similarity is the average value of the similarity between the text blocks in the text cluster and the cluster center. Finally, the final score (i.e., the second label score of the text cluster label) is obtained by multiplying the normalized score of the fused text cluster label by the cluster weight, and the information of the route “2” is supplemented. The text cluster labels of all text clusters are summarized.

[0057] Route 3: The full-text summary is generated based on the text blocks, the full-text entity list is generated by merging and de-duplicating the first entity list, the third system label and the third LLM label corresponding to the full-text summary are generated based on the full-text summary and the full-text entity list, and the third system label and the third LLM label are fused to obtain the full-text summary label and the corresponding third label score.

[0058] In specific implementation, first, a full-text summary is generated based on the text blocks, which can be implemented in the following ways, including but not limited to: dividing the text blocks into multiple dynamic semantic groups based on the similarity between the text blocks; splicing the text blocks in the dynamic semantic groups, and inputting the spliced text into a large language model to generate a group-level summary and an intra-group core topic of each dynamic semantic group; merging the group-level summaries of each dynamic semantic group, and inputting the merged text and the intra-group core topic of each dynamic semantic group into the large language model to generate a full-text summary.

[0059] In the embodiment of the application, when generating a hierarchical summary (i.e., a full-text summary), a three-level progressive mode of "original chunk dynamic clustering-group level summary-full text summary" can be used to construct natural topic grouping based on semantic association. Specifically, the following processes are included:

[0060] (1) Dynamic semantic grouping of original chunks is performed. In each original chunk, the original chunk is automatically divided into a semantically coherent group by vector similarity calculation. Specifically, the average cosine similarity of the current chunk and multiple chunks in the existing group is calculated. When the average cosine similarity is not less than a threshold and the number of chunks in the group does not exceed the maximum limit, the chunk is assigned to the current group. Otherwise, a new group is created. In this way, the text in each group has strong semantic association.

[0061] (2) Generate a group-level summary. The original chunks in each dynamic semantic group are spliced (to ensure complete context information), and then a 50-80 word summary (i.e., a group-level summary) of the spliced text is generated using LLM to accurately extract the core topic in the group and clearly define the core content around each group, laying the foundation for global aggregation.

[0062] (3) Perform full-text summary aggregation. The group-level summaries of all dynamic semantic groups are merged into a complete text, and a 200-300 word full-text summary is generated with the help of LLM. The generated full-text summary needs to cover all the core topics in the group, clearly reflect the overall logic of the document (such as the progressive relationship of background-method-conclusion), and highlight 3-5 core topics throughout the document, forming a complete capture of the document's full-text semantics.

[0063] Then, the tags of the full-text summary are generated, including:

[0064] (1) Based on the similarity between the vector of the full-text summary and the vector of the pre-constructed tag library, the third system tag corresponding to the full-text summary is obtained.

[0065] In a specific implementation, the system tag retrieval based on vector similarity is performed on the full-text abstract. First, the text of the full-text abstract is embedded into a vector, and the cosine similarity between the vector and each label vector in the label library is calculated. Then, the top-4 labels with the highest similarity are selected as the initial label candidates (i.e., the third system label) most relevant to the document topic.

[0066] (2) Based on the full-text abstract, the in-group core topics of each dynamic semantic group, and the full-text entity list, input into a large language model to obtain the third LLM label corresponding to the full-text abstract and the confidence of each third LLM label.

[0067] In a specific implementation, the named entity recognition (NER) results of all chunks (i.e., the first entity list) are merged and de-duplicated to form a full-text entity list covering the full text. Subsequently, a large language model is called with "full-text abstract + in-group core topics of all semantic groups + full-text entity list" as input, requiring the large model to generate 5 core labels (i.e., third LLM labels), preferentially using general terms in the professional field, and outputting the confidence of each label to capture the deep semantic of the document.

[0068] Finally, the third system label and the third LLM label are fused, which can be achieved by including but not limited to the following methods: calculating the normalized scores of the third system label and the third LLM label, and de-duplicating the third system label and the third LLM label based on the normalized scores to obtain the full-text abstract label; multiplying the normalized score of each full-text abstract label by a predetermined coefficient to obtain the third label score of each full-text abstract label.

[0069] In a specific implementation, the third system label and the third LLM label are fused. First, the similarity scores and confidences of the two types of labels are normalized (refer to the normalization scheme of Route 1) to obtain the normalized scores of the third system label and the third LLM label, and the labels are removed and conflicted according to the normalized scores (refer to the normalization scheme of Route 1) to obtain the fused full-text abstract label. Finally, for the fused full-text abstract label, a global perspective weight is assigned, i.e., the normalized score is multiplied by a predetermined coefficient (such as 1.2) to obtain the final score (i.e., the third label score), to strengthen the influence of the global label in the overall label system, and mark the source information as "3" to form the final global label set.

[0070] In an embodiment, for the aforementioned step S103, i.e., when the text block label, the text cluster label, and the full-text abstract label are fused to obtain the label of the document to be processed, the following methods can be used, including but not limited to:

[0071] Firstly, the label texts, score information and source information of the text block label, text cluster label and full-text summary label are obtained, and the text block label, text cluster label and full-text summary label are standardized.

[0072] In specific implementation, the format of all labels is unified. For all labels generated by the three routes, the label texts, final scores, source information, route information and the like are extracted, and the field names are unified, for example, the label texts are processed into a form with spaces removed at the beginning and end, and the initial frequency is set to 1, so that all labels have a consistent structure, facilitating subsequent fusion processing.

[0073] Secondly, the text block label, text cluster label and full-text summary label are grouped based on the label texts to obtain label groups.

[0074] In specific implementation, all labels are grouped according to the label texts, and labels of the same text are grouped into a group, that is, the label group includes one or more labels with the same label text, and the labels are one or more of the text block label, text cluster label and full-text summary label. That is, the labels in the same label group have the same label text, and the labels belong to one or more of the text block label, text cluster label and full-text summary label.

[0075] Next, for each label group, the weighted average score of the labels in the label group is calculated based on the preset weight coefficient, and the diversity bonus of the labels in the label group is determined based on the source information of the labels.

[0076] In specific implementation, for each label group, the comprehensive score of the label is calculated. The preset weight coefficient in the embodiment of the application is a route weight, that is, the weight is set according to the route information of the label, specifically, route 2 (0.4), route 1 (0.3), and route 3 (0.3). The weighted average score of each label in the label group is obtained by multiplying the normalized score of each label in the label group by the weight coefficient of the corresponding route and summing. At the same time, a bonus is given according to the diversity of the sources of the labels in the group, wherein the source category is added by 0.1 for each additional one. For example, there are three labels in the label group, which come from route 1, route 2 and route 3 respectively, and the diversity bonus is 0.3.

[0077] Then, based on the weighted average score and the diversity bonus of the labels in the label group, the comprehensive score of the labels in the label group is determined, and the labels in the label group are merged to obtain a merged label.

[0078] In specific implementation, the comprehensive score is obtained by adding all the weighted average scores and the diversity bonus, and each label group forms a merged label, including: label text, comprehensive score, source quantity and involved route information.

[0079] Afterwards, based on the comprehensive score of the merged labels, a second preset number of merged labels are selected as the preliminary screening labels.

[0080] Finally, based on the similarity between the preliminary screening labels, synonymous labels are removed, and a third number of labels are obtained as the labels of the to-be-processed document.

[0081] In specific implementation, the final label set is optimized. First, the merged labels are sorted from high to low according to the comprehensive score, and the top 20 (second preset number) merged labels are selected as the preliminary screening labels. Then, the near-synonymous labels are removed, the cosine similarity of the vector of each preliminary screening label and the vector of the selected preliminary screening label is calculated, if the cosine similarity exceeds a threshold, it is considered as a near-synonymous label and is removed, until 10 labels are selected or all preliminary screening labels are screened. Finally, if 10 labels are selected, the screening is ended, if not, the labels with higher comprehensive score are selected from the remaining labels to supplement, to ensure that the final output is Top10 labels.

[0082] The above method provided by the embodiment of the application utilizes a pre-trained industry named entity recognition model or a large language model to perform entity extraction on each text segment. In this process, the entities in a single chunk need to be de-duplicated to form an entity list of all chunks. The entity list can assist the LLM in generating labels in subsequent links; meanwhile, in the final fusion stage, all entities need to be de-duplicated and integrated before being merged into the label set, so as to make up for the possible deficiencies when relying solely on semantics to generate labels.

[0083] For ease of understanding, the embodiment of the application also provides another flowchart of a document labeling method based on entity enhancement and multi-granularity fusion, which is shown in Figure 2 and mainly includes the following processes:

[0084] (1) Input the document chunks.

[0085] Specifically, the document chunks are input, and entity extraction is performed through NER to obtain an entity list for assisting the LLM in generating labels.

[0086] (2) Route 1: Chunk-level fine-grained labeling. Route 2: Cluster label generation. Route 3: Full-text summary label generation. The specific implementation mode can be referred to the corresponding route of the preceding embodiment.

[0087] (3) Standardization processing.

[0088] Specifically, the labels generated by the three routes are standardized, including but not limited to: format unification, confidence normalization, and weight standardization.

[0089] (4) Multi-dimensional fusion calculation.

[0090] Specifically, the labels generated by the three routes are fused in relevance (i.e., weighted calculation), diversity balance (i.e., diversity bonus), coverage optimization, and hierarchical fusion to obtain fused labels, and the entities in the generated entity list are merged with the fused labels.

[0091] (5) Conflict resolution and optimization.

[0092] Specifically, the fused labels are subjected to conflict resolution and optimization, including but not limited to semantic deduplication, hierarchical conflict processing, and quality filtering.

[0093] (6) TopK label.

[0094] Specifically, after screening, the final label set and confidence score are obtained, and after manual review and system optimization, they are incorporated into the label library.

[0095] The above method provided by the embodiment of the application is aimed at an existing hierarchical label library, and generates full-text labels containing global themes and details based on upstream Chunk text through a technical architecture of multi-granularity coverage + entity empowerment + dynamic fusion. The method adopts an "entity enhancement + three-line parallelism + hierarchical fusion" mode, enhances semantic understanding through entity extraction, performs multi-dimensional fusion after labels are independently generated by three routes, and finally outputs TopK full-text labels and establishes hierarchical association, thereby improving the relevance, comprehensiveness, and accuracy of document labels.

[0096] For the document labeling method based on entity enhancement and multi-granularity fusion provided by the foregoing embodiment, the embodiment of the application further provides a document labeling system based on entity enhancement and multi-granularity fusion, which is shown in Figure 3 The structure diagram of the document labeling system based on entity enhancement and multi-granularity fusion is shown in FIG. 1.

[0097] The entity extraction module 301 is configured to split the to-be-processed document into a plurality of text blocks, and perform entity extraction on each text block to obtain a first entity list of each text block.

[0098] The label generation module 302 is configured to perform text block level labeling, text cluster level labeling, and full-text abstract level labeling based on the text blocks and the first entity list to generate text block labels, text cluster labels, and full-text abstract labels.

[0099] The label fusion module 303 is configured to fuse the text block labels, the text cluster labels, and the full-text abstract labels to obtain labels of the to-be-processed document.

[0100] The document tagging system based on entity enhancement and multi-granularity fusion provided by the embodiment of the present application first performs entity extraction on each text block to obtain a corresponding first entity list, then uses the first entity list and the text block to obtain text block labels, text cluster labels and full-text abstract labels by parallel processing of three routes, and finally fuses the text block labels, the text cluster labels and the full-text abstract labels, and fuses all entities into a label set to obtain the final label of the document to be processed. The system generates labels of the document from three aspects of the text block, the text cluster and the full-text abstract, and fuses entities of the document to obtain the final label, thereby improving the comprehensiveness and accuracy of the generation of the document label.

[0101] It should be noted that the system provided by the embodiment of the present application has the same implementation principle and technical effects as the foregoing method embodiments, and for brevity of description, the system embodiment part is not mentioned in the foregoing method embodiments. The specific numerical values provided in the embodiment of the present application are only exemplary and are not limited herein.

[0102] The embodiment of the present application also provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program executes the method according to any one of the above embodiments when executed by the processor.

[0103] Figure 4 The structure diagram of the electronic device provided by the embodiment of the present application is shown in the figure, and the electronic device 100 includes a processor 40, a memory 41, a bus 42 and a communication interface 43, the processor 40, the communication interface 43 and the memory 41 are connected through the bus 42; the processor 40 is used to execute the executable modules stored in the memory 41, such as computer programs.

[0104] The memory 41 may contain a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 43 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0105] The bus 42 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0106] The memory 41 is configured to store a program, and the processor 40 is configured to execute the program after receiving an execution instruction. The method performed by the device for defining a flow process according to any of the foregoing embodiments of the present application can be applied to the processor 40 or implemented by the processor 40.

[0107] The processor 40 can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the foregoing method can be completed by an integrated logic circuit of hardware in the processor 40 or an instruction in the form of software. The processor 40 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), and the like; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and the like storage media mature in the art. The storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and combines the hardware to complete the steps of the foregoing method.

[0108] The computer program product of the readable storage medium provided by the embodiments of the present application includes a computer readable storage medium storing a program code, and the program code includes instructions for executing the method described in the foregoing method embodiments. For specific implementation, reference can be made to the foregoing method embodiments, which will not be described here.

[0109] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0110] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some technical features. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A document tagging method based on entity enhancement and multi-granularity fusion, characterized in that, include: The document to be processed is divided into multiple text blocks, and entity extraction is performed on each text block to obtain a first entity list for each text block; Based on the text block and the first entity list, text block-level tagging, text cluster-level tagging and full-text summary-level tagging are performed respectively to generate text block tags, text cluster tags and full-text summary tags; The text block tags, text cluster tags, and full-text summary tags are merged to obtain the tags of the document to be processed; Based on the text blocks and the first entity list, text block-level tagging, text cluster-level tagging, and full-text summary-level tagging are performed respectively to generate text block tags, text cluster tags, and full-text summary tags. This includes: generating a first system tag and a first LLM tag for each text block based on the text blocks and the first entity list; fusing the first system tag and the first LLM tag to obtain a text block tag and a corresponding first tag score for each text block; wherein, the system tag is a tag generated based on the similarity between the vector of each text block and tag vectors in a pre-built tag library; the LLM tag is a tag generated using a large-scale language model; and clustering the text blocks to obtain multiple text clusters. The system generates a second entity list corresponding to the text cluster based on the first entity list, and generates a second system tag and a second LLM tag for each text cluster based on the text cluster and the second entity list. The second system tag and the second LLM tag are then merged to obtain a text cluster tag and a corresponding second tag score for each text cluster. A full-text summary is generated based on the text block, and the first entity list is merged and deduplicated to generate a full-text entity list. A third system tag and a third LLM tag corresponding to the full-text summary are generated based on the full-text summary and the full-text entity list. The third system tag and the third LLM tag are then merged to obtain a full-text summary tag and a corresponding third tag score.

2. The method according to claim 1, characterized in that, Based on the text blocks and the first entity list, a first system tag and a first LLM tag are generated for each text block, and the first system tag and the first LLM tag are merged to obtain the text block tag and the corresponding first tag score for each text block, including: Based on the similarity between the vector of each text block and the tag vectors in the pre-built tag library, a first system tag for each text block is obtained; Based on the text block and the corresponding first entity list, a prompt word for a large language model is generated, and the prompt word is input into the large language model to obtain the first LLM label of the text block and the confidence level of each first LLM label; Calculate the normalized scores of the first system tag and the first LLM tag, and deduplicate the first system tag and the first LLM tag based on the normalized scores to obtain the text block tag for each text block; wherein, the normalized score of the first system tag is the similarity between the first system tag and the tag vector in the tag library multiplied by a preset adaptation coefficient; the normalized score of the first LLM tag is the confidence score of the first LLM tag multiplied by the accuracy coefficient of the large language model. Based on the position of the text block in the document to be processed, determine the position weight corresponding to each text block; The normalized score of the text block label of each text block is multiplied by the corresponding position weight to obtain the first label score of each text block label.

3. The method according to claim 1, characterized in that, Clustering the text blocks to obtain multiple text clusters, and generating a second entity list corresponding to the text clusters based on the first entity list, includes: The HDBSCAN algorithm is used to process the vector of each text block to obtain the cluster label of each text block; The text blocks are grouped based on the cluster labels to obtain multiple text clusters; Based on the average value of the vector of each text block within the text cluster, the cluster center of the text cluster is determined, and based on the similarity between each text block and the cluster center, a first preset number of text blocks are selected and merged to obtain the cluster-level text of the text cluster; The first entity list of the text blocks corresponding to the cluster-level text is merged and deduplicated to obtain the second entity list corresponding to the text cluster.

4. The method according to claim 3, characterized in that, The second system label and the second LLM label are fused to obtain the text cluster label and the corresponding second label score for each text cluster, including: Calculate the normalized scores of the second system tag and the second LLM tag, and deduplicate the second system tag and the second LLM tag based on the normalized scores to obtain the text cluster tag for each text cluster; The cluster weight of each text cluster is determined based on the cluster size of the text cluster and the average similarity between the text blocks in the text cluster and the cluster center. The normalized score of the text cluster label of each text cluster is multiplied by the corresponding cluster weight to obtain the second label score of each text cluster label.

5. The method according to claim 1, characterized in that, Generate a full-text summary based on the text blocks, including: Based on the similarity between the text blocks, the text blocks are divided into multiple dynamic semantic groups; The text blocks within the dynamic semantic group are concatenated, and the concatenated text is input into a large language model to generate a group-level summary and core topic within each dynamic semantic group. The group-level summaries of each dynamic semantic group are merged, and the merged text and the core topics within each dynamic semantic group are input into the large language model to generate a full-text summary.

6. The method according to claim 5, characterized in that, Based on the full-text summary and the full-text entity list, a third system tag and a third LLM tag corresponding to the full-text summary are generated. The third system tag and the third LLM tag are then fused to obtain the full-text summary tag and the corresponding third tag score, including: Based on the similarity between the vector of the full-text summary and the tag vector in the pre-built tag library, the third system tag corresponding to the full-text summary is obtained; Based on the full-text summary, the core topics within each dynamic semantic group, and the full-text entity list, the large-scale language model is input to obtain the third LLM label corresponding to the full-text summary and the confidence level of each third LLM label; Calculate the normalized scores of the third system tag and the third LLM tag, and remove duplicates of the third system tag and the third LLM tag based on the normalized scores to obtain the full-text summary tag; The normalized score of each full-text summary tag is multiplied by a preset coefficient to obtain the third tag score of each full-text summary tag.

7. The method according to claim 1, characterized in that, The tags for the document to be processed are obtained by fusing the text block tags, the text cluster tags, and the full-text summary tags, including: Obtain the tag text, score information, and source information of the text block tags, text cluster tags, and full-text summary tags, and then standardize the text block tags, text cluster tags, and full-text summary tags; Based on the tag text, the text block tags, the text cluster tags, and the full-text summary tags are grouped to obtain tag groups; wherein, the tag group includes one or more tags with the same tag text, and the tags are one or more of the text block tags, text cluster tags, and full-text summary tags; For each of the tag groups, a weighted average score of the tags in the tag group is calculated based on a preset weighting coefficient, and a diversity score of the tags in the tag group is determined based on the source information of the tags. Based on the weighted average score and diversity score of the tags in the tag group, the comprehensive score of the tags in the tag group is determined, and the tags in the tag group are merged to obtain merged tags; Based on the comprehensive score of the merged tags, a second preset number of merged tags are selected as preliminary screening tags; Based on the similarity between the initially screened tags, synonymous tags are removed, and a third number of tags are obtained as the tags for the document to be processed.

8. A document tagging system based on entity augmentation and multi-granularity fusion, characterized in that, include: The entity extraction module is used to divide the document to be processed into multiple text blocks and extract entities from each text block to obtain a first entity list for each text block. The tag generation module is used to perform text block-level tagging, text cluster-level tagging, and full-text summary-level tagging based on the text block and the first entity list, respectively, to generate text block tags, text cluster tags, and full-text summary tags; The tag fusion module is used to fuse the text block tags, the text cluster tags, and the full-text summary tags to obtain the tags of the document to be processed; The tag generation module is specifically used for: generating a first system tag and a first LLM tag for each text block based on the text block and the first entity list; fusing the first system tag and the first LLM tag to obtain a text block tag and a corresponding first tag score for each text block; wherein, the system tag is a tag for the text block generated based on the similarity between the vector of each text block and the tag vector in a pre-built tag library; the LLM tag is a tag for the text block generated using a large language model; clustering the text blocks to obtain multiple text clusters, and generating a second entity list corresponding to the text cluster based on the first entity list. The system generates a second system tag and a second LLM tag for each text cluster based on the text cluster and the second entity list, and merges the second system tag and the second LLM tag to obtain a text cluster tag and a corresponding second tag score for each text cluster; it generates a full-text summary based on the text block, merges and deduplicates the first entity list to generate a full-text entity list, and generates a third system tag and a third LLM tag corresponding to the full-text summary based on the full-text summary and the full-text entity list, and merges the third system tag and the third LLM tag to obtain a full-text summary tag and a corresponding third tag score.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the steps of the method according to any one of claims 1 to 7.

Images