Cross-document relationship hierarchical classification method based on large language model

Abstract

The invention provides a cross-document relationship hierarchical classification method based on a large language model. The method comprises the following steps: acquiring a predefined relationship set and a division standard; performing split hierarchical clustering on all predefined labels in the predefined relation set according to the division standard to obtain a hierarchical label tree; and splicing the cross-document context, the head and tail entities and the candidate node set of the current layer of the hierarchical tag tree into an input text input large language model by adopting a prompt template, so that the large language model predicts proper nodes from the candidate node set of the current layer, and the like, until a corresponding predefined label is predicted in the last layer; therefore, the predefined relationship is organized into a hierarchical label tree by utilizing a large language model, and the number of candidate labels needing to be considered by the model in each prediction is reduced by combining a hierarchical classification algorithm, so that the prediction difficulty is reduced.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and artificial intelligence, and in particular to a hierarchical classification method for cross-document relationships based on a large language model. Background Technology

[0002] In related technologies, existing methods for cross-document relationship extraction can be broadly divided into two categories: The first category, such as MRCOD and REIC, mainly focuses on filtering long documents from different perspectives to retain useful information, thereby enabling small language models to be effectively applied to this task; the second category, such as ECRIM and NEPD, mainly focuses on using small language models to encode various entities in documents, thereby modeling long-distance head-to-tail entity relationships.

[0003] While these studies have made progress, their methods remain confined to the "small language model + classification head" paradigm, with performance limited by the relatively weak semantic modeling capabilities of small language models. Furthermore, this paradigm requires predicting the correct answer by simultaneously treating all predefined relations as candidate labels. When the number of predefined relations is large—for example, the commonly used cross-document relation extraction dataset CodRED has 277 predefined relations—predicting the target relation from so many predefined relations at once remains a significant challenge for existing models. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. To this end, one objective of this invention is to propose a hierarchical classification method for cross-document relationships based on a large language model. This method utilizes a large language model to organize predefined relationships into a hierarchical label tree, and combines it with a hierarchical classification algorithm to reduce the number of candidate labels that the model needs to consider in each prediction, thereby reducing the difficulty of prediction.

[0005] To achieve the above objectives, this invention proposes a hierarchical classification method for cross-document relationships based on a large language model. This method includes the following steps: obtaining a predefined set of relationships and a partitioning criterion; performing split-level hierarchical clustering on all predefined labels in the predefined set of relationships according to the partitioning criterion to obtain a hierarchical label tree; using a prompt template to concatenate the cross-document context, head and tail entities, and the candidate node set of the current layer of the hierarchical label tree into input text for the large language model, so that the large language model can predict suitable nodes from the candidate node set of the current layer, and so on, until the corresponding predefined label is predicted in the last layer; thus, the large language model organizes the predefined relationships into a hierarchical label tree, and combined with a hierarchical classification algorithm, reduces the number of candidate labels that the model needs to consider in each prediction, thereby reducing the prediction difficulty.

[0006] In addition, the cross-document relation hierarchical classification method based on a large language model proposed in the above embodiments of the present invention may also have the following additional technical features:

[0007] Optionally, a split-level hierarchical clustering is performed on all predefined labels in the predefined relation set according to the partitioning criteria to obtain a hierarchical label tree, including: taking all predefined labels as an initial cluster, dividing the initial cluster into multiple sub-clusters according to the partitioning criteria, and defining a label name for each sub-cluster, wherein each sub-cluster corresponds to a node in the hierarchical label tree; recursively partitioning each sub-cluster to obtain a corresponding subtree; and concatenating all subtrees to obtain the hierarchical label tree.

[0008] Optionally, a prompt template is used to concatenate the candidate node set across the document context, head and tail entities, and the current layer of the hierarchical tag tree into an input text input large language model, so that the large language model predicts a suitable node from the candidate node set of the current layer. This includes: using a prompt template to concatenate the candidate node set across the document context, head and tail entities, and the current layer of the hierarchical tag tree into an input text input large language model to obtain multiple initial predicted nodes; expanding the multiple initial predicted nodes into lower-level nodes containing fine-grained semantic information to obtain multiple new candidate node sets, and prompting the large language model to predict again based on the multiple new candidate node sets to obtain a new predicted node corresponding to each new candidate node set; comparing and verifying the new predicted node corresponding to each new candidate node set with the initial predicted node with the highest prediction probability among the multiple initial predicted nodes to obtain a verification result; if the verification result is successful, the initial predicted node with the highest prediction probability is the suitable node; if the verification result is unsuccessful, the initial predicted node with the highest prediction probability is deleted from the initial candidate node set, and the prediction verification is performed again, and so on, until a suitable node is predicted.

[0009] Optionally, the plurality of initial prediction nodes are expanded into lower-level nodes containing fine-grained semantic information to obtain a plurality of new candidate node sets, including: expanding each initial prediction node into a lower-level node containing fine-grained semantic information; individually replacing each corresponding initial prediction node in the initial candidate set with a lower-level node; and simultaneously replacing each corresponding initial prediction node in the initial candidate set with a lower-level node to obtain a plurality of new candidate node sets.

[0010] Optionally, the plurality of new prediction nodes are compared and verified with the initial prediction node with the highest prediction probability among the plurality of initial prediction nodes to obtain a verification result, including: comparing the plurality of new prediction nodes with the initial prediction node with the highest prediction probability among the plurality of initial prediction nodes; if the number of the plurality of new prediction nodes exceeds a preset threshold and is the same as the initial prediction node with the highest prediction probability or belongs to a lower-level node of the initial prediction node with the highest prediction probability, then the initial prediction node with the highest prediction probability is verified; otherwise, the initial prediction node with the highest prediction probability fails to be verified. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a cross-document relational hierarchical classification method based on a large language model according to an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of a prompt template according to an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram illustrating the construction process of a hierarchical tag tree according to an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the model reasoning process according to an embodiment of the present invention;

[0015] Figure 5 This is a block diagram of a cross-document relationship hierarchical classification device based on a large language model according to an embodiment of the present invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0017] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0018] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1As shown, the cross-document relational hierarchical classification method based on a large language model according to an embodiment of the present invention includes the following steps:

[0020] S101, obtain the predefined set of relations and the partitioning criteria.

[0021] It should be noted that the predefined relation set consists of 277 predefined relations from the existing cross-document relation extraction dataset CodRED. The categorization criteria are for the effective classification of predefined relations. For example, they can be categorized by domain, entity type, etc. For instance, the predefined relations are first categorized by domain, and then further categorized by entity type.

[0022] S102, perform split-type hierarchical clustering on all predefined labels in the predefined relation set according to the partitioning criteria to obtain a hierarchical label tree.

[0023] As an example, a hierarchical clustering method is used to perform splitting hierarchical clustering on all predefined labels in the predefined relation set according to the partitioning criteria to obtain a hierarchical label tree. This includes: taking all predefined labels as an initial cluster, dividing the initial cluster into multiple sub-clusters according to the partitioning criteria, and defining a label name for each sub-cluster, wherein each sub-cluster corresponds to a node in the hierarchical label tree; recursively partitioning each sub-cluster to obtain the corresponding subtree; and concatenating all subtrees to obtain the hierarchical label tree.

[0024] In other words, OpenAI o1's powerful ability to solve complex problems is combined with the idea of ​​split-level hierarchical clustering to construct a hierarchical label tree structure layer by layer. Before starting this process, partitioning criteria are defined. o1 is prompted to generate several partitioning criteria based on the definition of cross-document relationship extraction to effectively distinguish relationship labels. These criteria are used for subsequent clustering of relationship labels. Next, the tree construction process begins. All predefined labels are treated as an initial cluster. o1 is then prompted to divide this cluster into several sub-clusters according to the partitioning criteria, and a suitable label name is assigned to each cluster. Each cluster corresponds to a node in the label tree. Then, the above partitioning is recursively performed on each sub-cluster, thereby deriving a corresponding subtree for each cluster. Finally, all subtrees are concatenated to obtain the complete hierarchical label tree.

[0025] Specifically, such as Figure 3 As shown, the hierarchical label tree has five levels. The top level is a unique root node, representing all relational semantics. The second level has two nodes, indicating whether the relation label belongs to a predefined relation. The third and fourth levels are intermediate semantic nodes generated by the large language model, summarizing the semantics of the lower-level nodes. The bottom level nodes are the predefined relation labels, representing the most specific relation. Each node is connected to its corresponding upper-level node, forming the hierarchical label tree.

[0026] S103, using a prompt template, concatenates the candidate node set of the current layer across the document context, head and tail entities, and hierarchical label tree into the input text input to the large language model, so that the large language model can predict the appropriate node from the candidate node set of the current layer, and so on, until the corresponding predefined label is predicted in the last layer.

[0027] In other words, such as Figure 2 As shown, a prompt template is used to concatenate the candidate node set across the document context, head and tail entities and the current layer into the input text, requiring the large language model to predict the correct node from the given candidate node set of the current layer.

[0028] As one embodiment, a prompt template is used to concatenate the candidate node set of the current layer across document context, head and tail entities, and hierarchical tag tree into an input text input large language model, so that the large language model can predict suitable nodes from the candidate node set of the current layer. This includes: using a prompt template to concatenate the candidate node set of the current layer across document context, head and tail entities, and hierarchical tag tree into an input text input large language model to obtain multiple initial predicted nodes; expanding the multiple initial predicted nodes into lower-level nodes containing fine-grained semantic information to obtain multiple new candidate node sets, and prompting the large language model to predict again based on the multiple new candidate node sets to obtain new predicted nodes corresponding to each new candidate node set; comparing and verifying the new predicted nodes corresponding to each new candidate node set with the initial predicted node with the highest prediction probability among the multiple initial predicted nodes to obtain a verification result; if the verification result is successful, the initial predicted node with the highest prediction probability is the suitable node; if the verification result is unsuccessful, the initial predicted node with the highest prediction probability is removed from the initial candidate node set, and the prediction verification is performed again, and so on, until a suitable node is predicted.

[0029] As one embodiment, multiple initial prediction nodes are expanded into lower-level nodes containing fine-grained semantic information to obtain multiple new candidate node sets, including: expanding each initial prediction node into a lower-level node containing fine-grained semantic information; individually replacing each corresponding initial prediction node in the initial candidate set with a lower-level node; and simultaneously replacing each corresponding initial prediction node in the initial candidate set with a lower-level node to obtain multiple new candidate node sets.

[0030] As an example, multiple new prediction nodes are compared and verified with the initial prediction node with the highest prediction probability among multiple initial prediction nodes to obtain a verification result. This includes: comparing multiple new prediction nodes with the initial prediction node with the highest prediction probability among multiple initial prediction nodes; if the number of multiple new prediction nodes exceeds a preset threshold and is the same as the initial prediction node with the highest prediction probability or belongs to a lower-level node of the initial prediction node with the highest prediction probability, then the initial prediction node with the highest prediction probability is verified successfully; otherwise, the initial prediction node with the highest prediction probability fails to be verified.

[0031] In other words, such as Figure 4 As shown, during model inference, hierarchical classification is performed based on a hierarchical label tree. Simultaneously, a prediction-validation strategy is used at each layer to improve the accuracy of single relation predictions, thereby mitigating the error propagation problem caused by hierarchical classification. Specifically, at the start of inference, the large language model is placed at the root node of the tree. Each prediction requires the model to predict a suitable lower-level node from the child nodes of the current node. This process is iteratively repeated until the specific relation label is predicted at the last layer, completing the hierarchical classification. At each layer, a prediction-validation strategy is used: first, in the prediction step, the model is prompted to select a suitable node from the child nodes of the current node, called the initial prediction node; then, in the validation step, the initial prediction node is expanded to include lower-level nodes containing fine-grained semantic information, reorganized into several new candidate node sets, and the model is prompted to predict again to obtain new prediction nodes. Next, the compatibility between the new prediction node and the initial prediction node is verified. If the new prediction node is the same as the initial prediction result or is a child node of the initial prediction result, the verification is successful; otherwise, the initial prediction node will be removed and the prediction will start again until a suitable lower-level node is selected and the verification is successful.

[0032] Specifically, Figure 4 This approach utilizes a hierarchical label tree to guide a large language model to infer the target relation r layer by layer for each input instance x (context, head entity, and tail entity). It begins with the root node and treats its child nodes as a set of candidate nodes. A prompt template is used to format instance x and the candidate node set into text prompts, instructing the model to select the most suitable node r. 1st The selected node is then treated as the current node, and this process is repeated until a leaf node is reached. Finally, the predefined relations associated with the selected leaf node are considered predefined relations. Since the number of sub-relations of each node relation is far less than the entire set of predefined relations, hierarchical classification effectively reduces the number of relation options that a large language model needs to consider at each layer.

[0033] Specifically, in the "predict first, verify later" inference strategy, although the model effectively reduces the number of relational options considered at each layer, errors in the earlier layers may propagate and affect predictions in subsequent layers. To address the error propagation problem, this application proposes a "predict first, verify later" inference strategy that optimizes model predictions at each layer through multi-view verification. The strategy primarily improves the reliability of the model at each layer by iterating through the following two steps:

[0034] In the prediction step, at the l-th level of the hierarchical label tree, the child nodes of the nodes predicted at level (l-1) are first used as the candidate node set. Then, the best node r is obtained. 1st and a suboptimal node r 2nd (It should be noted that, according to the preset requirements, the model with the highest output probability r is selected.) 1st And the second highest probability r 2nd (as a prediction node). However, the model tends to use r 1st and r 2nd Confusion can lead to incorrect predictions.

[0035] Verification steps, to reduce r 1st and r 2nd To avoid confusion between different approaches, a set of verification options is constructed from multiple perspectives to verify r. 1st The correctness of the statement is as follows: Specifically, first, replace r with the corresponding child node from the initial candidate node set R0. 1st and r 2nd This creates two validation option sets, R1 and R2. Furthermore, a third validation option set, R3, is constructed by simultaneously replacing two nodes with child nodes from R0. Finally, the model is prompted to select three validation nodes r from R1, R2, and R3 respectively. ′ r ″ and r ″′ .

[0036] Repeat the above two steps until a majority of the three nodes satisfy the condition that r is true. 1st or the conditions of its child nodes. At this point, r... 1st If a prediction is considered reliable, the process continues to the next level of the hierarchical label tree. Otherwise, r 1st It is considered a false prediction and removed from R0.

[0037] Please note that R1, R2, and R3 are essentially three equivalent views of R0 at a finer granularity. During validation, due to the inclusion of next-level nodes, the validation option set contains more refined semantic information compared to the initial candidate node set R0. Therefore, these validation sets enable the model to effectively distinguish r 1st and r 2nd The subtle differences between them ultimately lead to more reliable predictions.

[0038] As an example, this application selects LLaMA3.1-8B-Instruct as the large language model for relation prediction. During inference, the kernel sampling rate and temperature are set to 1.0 and 0, respectively, and a maximum of 10 tokens are allowed to be generated per inference. During model training, LLaMA-Factory combined with LoRA technology is used for supervised fine-tuning of the model, where the LoRA parameters r=8, α=16, the dropout rate is 0, the training objective is the cross-entropy loss function, and the loss is calculated only on the labels. The AdamW optimizer is used with a learning rate of 5e-5, a total of 6400 training steps, and a total batch size of 32.

[0039] In summary, the hierarchical classification method for cross-document relationships based on a large language model according to embodiments of the present invention constructs a hierarchical label tree based on a predefined set of relationships, laying the foundation for subsequent hierarchical classification. The hierarchical label tree is organized in a hierarchical structure, with the bottom layer containing all predefined relationships, and each node in each layer above representing the superordinate concept of its child node, with each node connected to its corresponding superordinate concept. Using the hierarchical label tree, the large language model can progressively restrict the semantic scope of predicted relationships from top to bottom until the bottom-level predefined relationships are selected. Furthermore, the model does not need to consider all predefined relationships simultaneously in each prediction, but only needs to consider predictions from the child nodes of the current node, reducing the difficulty of model prediction. However, since each prediction depends on the result of the previous layer's prediction, this often introduces the problem of error propagation. To alleviate this problem, this application proposes a prediction verification strategy: after the initial prediction is completed, the predicted nodes are expanded into finer-grained semantic nodes, and the candidate node set is reorganized for prediction, thereby verifying the initial prediction results and improving the accuracy of the model's predictions at each layer. Finally, experimental results on the CodRED dataset demonstrate significant performance improvements. Specifically, the application achieves a micro F1 score of 44.84%, outperforming the state-of-the-art method by 1.88%; and a binary F1 score of 57.70%, outperforming the state-of-the-art method by 9.32%. These data fully demonstrate the technical advantages and practicality of this application in cross-document relation extraction tasks.

[0040] Figure 5 This is a block diagram of a cross-document hierarchical classification device based on a large language model according to an embodiment of the present invention. Figure 5 As shown, the cross-document hierarchical classification device includes: an acquisition module 10, a hierarchical tag tree construction module 20, and a cross-document hierarchical classification module 30;

[0041] The acquisition module 10 is used to acquire a predefined set of relations and a partitioning criterion; the hierarchical label tree construction module 20 is used to perform split hierarchical clustering on all predefined labels in the predefined set of relations according to the partitioning criterion to obtain a hierarchical label tree; the cross-document relation hierarchical classification module 30 is used to use a prompt template to concatenate the candidate node set of the current layer of the cross-document context, head and tail entities and hierarchical label tree into the input text input to the large language model, so that the large language model can predict the appropriate node from the candidate node set of the current layer, and so on, until the corresponding predefined label is predicted in the last layer.

[0042] It should be noted that the explanations and descriptions of the embodiments of the cross-document relationship hierarchical classification method based on large language models described above also apply to the cross-document relationship hierarchical classification device based on large language models in this embodiment, and will not be repeated here.

[0043] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0048] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0050] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0051] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0052] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0053] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0054] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A hierarchical classification method for cross-document relationships based on a large language model, characterized in that, Includes the following steps: Obtain the predefined set of relations and the partitioning criteria; Based on the partitioning criteria, perform split-level hierarchical clustering on all predefined labels in the predefined relation set to obtain a hierarchical label tree; The input text is fed into a large language model by concatenating the candidate node set across the document context, head and tail entities, and the current layer of the hierarchical tag tree using a prompt template. This allows the large language model to predict appropriate nodes from the candidate node set of the current layer, and so on, until the corresponding predefined tag is predicted in the last layer.

2. The hierarchical classification method for cross-document relationships based on a large language model as described in claim 1, characterized in that, Based on the aforementioned partitioning criteria, all predefined labels in the predefined relationship set are subjected to split-level hierarchical clustering to obtain a hierarchical label tree, including: All predefined labels are taken as an initial cluster. The initial cluster is divided into multiple sub-clusters according to the partitioning criteria, and a label name is defined for each sub-cluster. Each sub-cluster corresponds to a node in the hierarchical label tree. Recursively divide each sub-cluster to obtain the corresponding subtree; Concatenate all subtrees to obtain a hierarchical label tree.

3. The hierarchical classification method for cross-document relationships based on a large language model as described in claim 1, characterized in that, A prompt template is used to concatenate the candidate node set across the document context, head and tail entities, and the current layer of the hierarchical tag tree into an input text input large language model, so that the large language model can predict appropriate nodes from the candidate node set of the current layer, including: The prompt template is used to concatenate the candidate node set across the document context, head and tail entities and the current layer of the hierarchical tag tree into the input text input large language model to obtain multiple initial prediction nodes; The initial prediction nodes are expanded into lower-level nodes containing fine-grained semantic information to obtain multiple new candidate node sets. The large language model is then prompted to make predictions again based on the multiple new candidate node sets to obtain new prediction nodes corresponding to each new candidate node set. The new predicted node corresponding to each new candidate node set is compared and verified with the initial predicted node with the highest prediction probability among the multiple initial predicted nodes to obtain the verification result; If the verification result is successful, then the initial prediction node with the highest prediction probability is the appropriate node; If the verification result is unsuccessful, the initial prediction node with the highest prediction probability is removed from the initial candidate node set, and the prediction verification is performed again, and so on, until a suitable node is predicted.

4. The hierarchical classification method for cross-document relationships based on a large language model as described in claim 3, characterized in that, The initial prediction nodes are each expanded into lower-level nodes containing fine-grained semantic information to obtain a set of new candidate nodes, including: Each initial prediction node is expanded into a lower-level node containing fine-grained semantic information; Each initial prediction node in the initial candidate set is individually replaced with a lower-level node, and each initial prediction node in the initial candidate set is simultaneously replaced with a lower-level node to obtain multiple new candidate node sets.

5. The hierarchical classification method for cross-document relationships based on a large language model as described in claim 4, characterized in that, The plurality of new prediction nodes are compared and verified with the initial prediction node with the highest prediction probability among the plurality of initial prediction nodes to obtain verification results, including: The plurality of new prediction nodes are compared with the initial prediction node with the highest prediction probability among the plurality of initial prediction nodes; If the number of new prediction nodes exceeds a preset threshold and is the same as or a lower-level node of the initial prediction node with the highest prediction probability, then the initial prediction node with the highest prediction probability is verified; otherwise, the initial prediction node with the highest prediction probability fails to be verified.

Images