A contract element extraction method and related device

Abstract

The application discloses a contract element extraction method and a related device, relates to the technical field of natural language processing, and comprises the following steps: obtaining a target contract; and performing contract element extraction on the target contract based on a target contract element extraction model, wherein the target contract element extraction model is a contract element extraction model based on a target language model as a semantic understanding base, the target language model is a model obtained by fine-tuning a pre-training language model on a hierarchical data set corresponding to a contract sample on a plurality of different logical levels of contract element extraction tasks, or is an optimized model of the fine-tuned model, and each piece of data in the hierarchical data set comprises the content of a contract clause, an entity identified from the contract clause, a semantic dependency relationship between the contract clause and other contract clauses, and a logical level to which the contract clause belongs. The contract element extraction method provided by the application can accurately and efficiently extract contract elements from a contract.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method and related apparatus for extracting contract elements. Background Technology

[0002] With the increasing frequency and complexity of business activities, contracts, as the legal carriers of commercial transactions, are growing exponentially in number and their content complexity is constantly increasing. The core technical means of contract management is contract element extraction. Currently, the main method for contract element extraction is manual, where legal professionals, contract administrators, and other specialists manually read and understand the contracts to extract the contract elements.

[0003] However, it typically takes professionals several hours to process a moderately complex contract. When faced with a massive volume of contracts, labor costs rise sharply. At the same time, differences in the professional backgrounds and experience of different personnel lead to inconsistent information extraction standards, which can easily result in subjective biases. Furthermore, human processing capacity is limited and cannot meet the processing needs of tens of thousands of new contracts each year. In other words, current contract element extraction methods suffer from problems such as low efficiency, high labor costs, reliance on personal experience for extraction accuracy, and inability to handle the processing needs of massive volumes of contracts. Summary of the Invention

[0004] In view of this, this application provides a method and related apparatus for extracting contract elements, which solves the problems of low efficiency, high labor costs, reliance on personal experience for extraction accuracy, and inability to handle the processing needs of massive contracts in current contract element extraction methods. The technical solution is as follows:

[0005] The first aspect of this application provides a method for extracting contract elements, including:

[0006] Obtain the target contract;

[0007] Based on the target contract element extraction model, contract elements are extracted from the target contract to obtain the contract element extraction results of the target contract.

[0008] The target contract element extraction model is a contract element extraction model based on a target language model as the semantic understanding foundation. The target language model is a model obtained by fine-tuning a pre-trained language model on multiple contract element extraction tasks at different logical levels, based on a hierarchical dataset corresponding to contract samples in the contract sample set, or it is an optimized model obtained by fine-tuning the model.

[0009] Each data item in the hierarchical dataset corresponds to a contract clause in the contract sample. Each data item in the hierarchical dataset includes the clause content of the corresponding contract clause, entity information identified from the corresponding contract clause, semantic dependencies between the corresponding contract clause and other contract clauses, and the logical level to which the corresponding contract clause belongs. The multiple contract element extraction tasks at different logical levels are defined based on the hierarchical dataset.

[0010] In one possible implementation, the plurality of different logical layers include: a core layer, an association layer, and a support layer;

[0011] The core layer's contract element extraction task is to identify and extract the elements that constitute the basic framework of the contract.

[0012] The task of extracting contract elements in the association layer is to identify and extract the elements that constrain the contract performance process.

[0013] The task of extracting contract elements in the support layer is to identify and extract elements that address contract performance risks.

[0014] In one possible implementation, the process of obtaining the hierarchical dataset corresponding to the contract sample includes:

[0015] Identify entity information for each contract clause in the contract sample;

[0016] By combining the entity information identified from each contract clause in the contract sample, the semantic dependencies between different contract clauses in the contract sample are analyzed;

[0017] Using each contract clause in the contract sample as a node and the semantic dependencies between different contract clauses in the contract sample as directed edges, a directed semantic network graph is constructed.

[0018] The nodes in the directed semantic network graph are topologically sorted, and the contract terms corresponding to each node in the directed semantic network graph are logically hierarchically divided according to the topological sorting results.

[0019] Based on the logical hierarchy division results, a hierarchical dataset corresponding to the contract sample is constructed.

[0020] In one possible implementation, the step of performing topological sorting on the nodes in the directed semantic network graph and logically hierarchically dividing the contract terms corresponding to each node in the directed semantic network graph according to the topological sorting result includes:

[0021] Calculate the in-degree of each node in the directed semantic network graph, and add nodes with an in-degree of 0 to the processing queue.

[0022] The following process is executed iteratively until the processing queue is empty: a node is taken from the head of the processing queue and added to the sorting sequence, while the in-degree of the successor node of the currently taken node in the directed semantic network graph is reduced by 1, and the successor node whose in-degree is reduced to 0 is added to the tail of the processing queue.

[0023] Based on the final sorted sequence, the logical level to which the contract terms corresponding to each node in the directed semantic network graph belong is determined.

[0024] In one possible implementation, the process of optimizing the fine-tuned model includes:

[0025] Set the initial parameters of the fine-tuned model as the target parameters, initialize the optimization control parameters, and repeat the following steps until the preset optimization termination condition is met:

[0026] Based on the current optimized control parameters, the current target parameters are perturbed to obtain candidate parameters. The larger the current optimized control parameters are, the greater the perturbation to the current target parameters.

[0027] The performance of the fine-tuned model under the candidate parameters is evaluated using a test dataset to obtain the model performance evaluation value corresponding to the candidate parameters.

[0028] Based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter, a decision is made on whether to accept the candidate parameter.

[0029] If the candidate parameter is accepted, the current target parameter is updated to the candidate parameter, and the current optimization control parameter is reduced according to the predetermined decay strategy; if the candidate parameter is not accepted, the current target parameter remains unchanged.

[0030] Among them, the optimized model obtained by fine-tuning is the model configured with the final target parameters.

[0031] In one possible implementation, the step of determining whether to accept the candidate parameter based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter includes:

[0032] If the model performance evaluation value corresponding to the candidate parameter is better than the model performance evaluation value corresponding to the current target parameter, then the candidate parameter is accepted.

[0033] If the model performance evaluation value corresponding to the candidate parameter is not better than the model performance evaluation value corresponding to the current target parameter, then a decision is made on whether to accept the candidate parameter based on the probability acceptance criterion related to the current optimization control parameter.

[0034] In one possible implementation, determining whether to accept the candidate parameter based on a probability acceptance criterion related to the current optimized control parameters includes:

[0035] Calculate the difference between the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter;

[0036] Based on the difference and the current optimized control parameters, determine the acceptance probability for the candidate parameter;

[0037] Whether to accept the candidate parameter is determined based on the acceptance probability of the candidate parameter.

[0038] In one possible implementation, the step of extracting contract elements from the target contract based on the target contract element extraction model includes:

[0039] Based on the target contract element extraction model, the clause type of each contract clause in the target contract is determined from a set of preset clause types;

[0040] Based on the target contract element extraction model, and combined with the clause type of each contract clause in the target contract, entity information is identified and extracted from each contract clause in the target contract.

[0041] Based on the target contract element extraction model, and combining the clause types of each contract clause in the target contract with the entity information extracted from each contract clause, the semantic dependencies between different contract clauses in the target contract are analyzed.

[0042] The clause types of each contract clause in the target contract, the entity information extracted from each contract clause, and the semantic dependencies between different contract clauses are integrated into structured data to obtain the structured contract element data corresponding to the target contract.

[0043] In one possible implementation, determining the clause type of each contract clause in the target contract from a preset plurality of clause types based on the target contract element extraction model includes:

[0044] For each contract clause in the target contract, obtain the word embedding representation of that contract clause;

[0045] Using the target language model in the target contract element extraction model, the word embedding representation of the contract clause is semantically encoded to obtain the contextual semantic features of the contract clause.

[0046] Using the clause classification header in the target contract element extraction model, the clause type of the contract clause is determined from multiple preset clause types based on the contextual semantic features of the contract clause.

[0047] In one possible implementation, the step of identifying and extracting entity information from each contract clause of the target contract based on the target contract element extraction model, combined with the clause type of each contract clause in the target contract, includes:

[0048] For each contract clause in the target contract, the contract clause and its clause type are combined to form an input sequence, and the word embedding representation of the input sequence is obtained.

[0049] Using the target language model in the target contract element extraction model, the word embedding representation of the input sequence is semantically encoded to obtain contextual semantic features containing clause type information;

[0050] Using the entity recognition head of the target contract element extraction model, based on the contextual semantic features containing clause type information, the tags corresponding to each word in the contract clause are determined from the preset tag set, resulting in the tag sequence corresponding to the contract clause. The preset tag set includes tags representing various entities and tags representing non-entities.

[0051] Extract entity information from the contract terms based on the tag sequence corresponding to those terms.

[0052] In one possible implementation, the step of parsing the semantic dependencies between different contract clauses in the target contract based on the target contract element extraction model, combined with the clause types of each contract clause in the target contract and the entity information extracted from each contract clause, includes:

[0053] For the target contract, construct contract clause pairs to be parsed, resulting in multiple contract clause pairs;

[0054] For each contract clause pair, an input sequence corresponding to each contract clause in the contract clause pair is constructed to obtain the input sequence pair corresponding to the contract clause pair. Each input sequence in the input sequence pair includes the corresponding contract clause, the clause type of the corresponding contract clause, and entity information extracted from the corresponding contract clause.

[0055] Obtain the word embedding representation of the input sequence pair corresponding to the contract terms;

[0056] By utilizing the target language model in the target contract element extraction model, and processing the word embedding representation of the input sequence pair corresponding to the contract clause pair, the contextual semantic interaction features of the two contract clauses contained in the contract clause pair are captured.

[0057] Using the relation classification head in the target contract element extraction model, based on the contextual semantic interaction features of the two contract clauses contained in the contract clause pair, it is determined whether there is a semantic dependency relationship between the two contract clauses contained in the contract clause pair, and if so, the type of semantic dependency relationship.

[0058] A second aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0059] The memory is used to store computer programs;

[0060] The processor is used to execute the computer program so that the electronic device can implement any of the above-described contract element extraction methods.

[0061] A third aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement any of the above-described contract element extraction methods.

[0062] A fourth aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement any of the above-described contract element extraction methods.

[0063] By means of the above technical solution, the contract element extraction method provided in this application can extract contract elements of the target contract based on the target contract element extraction model. That is, this application will change from a manual processing mode that relies on personal experience and subjective judgment to an automated and standardized process driven by a professional model. On the one hand, the contract element extraction method provided in this application significantly reduces the processing time of a single contract from several hours required manually to seconds, supporting batch and instantaneous processing of contracts, thereby enabling enterprises to cope with the processing needs of massive contracts. On the other hand, the target contract element extraction model in the contract element extraction method provided in this application uses a target language model as its semantic understanding core. Since the target language model has been fine-tuned on multiple different logical levels of contract element extraction tasks using a hierarchical dataset corresponding to the contract samples, it can not only accurately understand the professional terminology in the contract, but also deeply grasp the internal logical structure of the contract and the complex semantic dependencies between clauses. Therefore, based on the above-mentioned deep semantic understanding capabilities, the target contract element extraction model can exhibit high accuracy, consistency and robustness in contract element extraction tasks, effectively eliminating the problem of unstable quality of contract element extraction results caused by differences in personnel background, fluctuations in experience level, subjective judgment bias and fatigue factors in traditional manual processing. The contract element extraction method provided in this application not only significantly improves the efficiency of contract element extraction, but also lays a solid technical foundation for enterprises' digital contract management, intelligent risk control, and scientific business decision-making by providing stable and accurate automated analysis capabilities. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0065] Figure 1 A flowchart illustrating the contract element extraction method provided in this application embodiment;

[0066] Figure 2 A flowchart illustrating the process of obtaining a hierarchical dataset corresponding to a contract sample, provided in an embodiment of this application;

[0067] Figure 3 This is a schematic diagram illustrating the process of optimizing the fine-tuned model as provided in an embodiment of this application.

[0068] Figure 4 This is a schematic diagram illustrating the process of extracting contract elements from a target contract based on the target contract element extraction model provided in this application embodiment.

[0069] Figure 5 This is a schematic diagram of the structure of the contract element extraction device provided in the embodiments of this application. Detailed Implementation

[0070] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0071] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0072] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0073] Current methods for extracting contract elements are mainly based on manual extraction. However, this method suffers from many problems, such as low efficiency, high labor costs, reliance on personal experience for extraction accuracy, low accuracy of extraction results, and poor scalability. Therefore, this study was conducted. The initial idea was to adopt a rule-based method for extracting contract elements, that is, for domain experts to manually write and maintain a large number of extraction rules, and then extract contract elements based on these rules.

[0074] Although rule-based contract element extraction schemes improve extraction efficiency, reduce labor costs, and improve extraction accuracy to some extent, they still have many shortcomings, such as high rule maintenance costs, poor generalization ability (they can only handle contracts with specific formats and structures and are sensitive to format changes and language diversity), and inability to handle complex logic.

[0075] Given the numerous shortcomings of the aforementioned rule-based contract element extraction schemes, further research was conducted. Through continuous research, a more effective contract element extraction method was finally proposed, which overcomes the deficiencies of both manual and rule-based contract element extraction methods.

[0076] The method for extracting contract elements provided in this application will be described below through the following embodiments.

[0077] Please see Figure 1 The diagram illustrates a flowchart of a contract element extraction method provided in an embodiment of this application. This contract element extraction method may include:

[0078] Step S101: Obtain the target contract.

[0079] The target contract is the contract from which contract elements are to be extracted.

[0080] Step S102: Based on the target contract element extraction model, extract contract elements from the target contract to obtain the result of the target contract element extraction.

[0081] Preferably, when extracting contract elements from a target contract based on the target contract element extraction model, the target contract can be preprocessed to transform it into a standard contract. The preprocessed contract can then be used as the final target contract, and contract elements can be extracted from the final target contract (standard contract) based on the target contract element extraction model. Alternatively, preprocessing of the target contract may be omitted, and contract elements can be extracted directly from the target contract based on the target contract element extraction model.

[0082] The preprocessing of the target contract may include, but is not limited to, some or all of the following: removing irrelevant content from the target contract (such as headers, footers, numbering, tables of contents, special characters, etc.), correcting typos and abnormal punctuation marks in the target contract, standardizing the text encoding format of the target contract, and dividing the target contract into a clear clause structure. After preprocessing, a standard contract with a uniform format, no redundant information, a clear clause structure, and ease of subsequent natural language processing operations can be obtained.

[0083] This embodiment implements contract element extraction for target contracts based on a target contract element extraction model. The target contract element extraction model is a contract element extraction model with a target language model as the semantic understanding foundation. In one possible implementation, a pre-trained language model can be fine-tuned on multiple contract element extraction tasks at different logical levels based on a hierarchical dataset corresponding to contract samples in the contract sample set. The fine-tuned model is then used as the target language model, thereby constructing a contract element extraction model with the target language model as the semantic understanding foundation. In another possible implementation, a pre-trained language model can first be fine-tuned on multiple contract element extraction tasks at different logical levels based on a hierarchical dataset corresponding to contract samples in the contract sample set. Then, the fine-tuned model can be further optimized based on an optimization algorithm. The optimized model is then used as the target language model, thereby constructing a contract element extraction model with the target language model as the semantic understanding foundation.

[0084] The hierarchical dataset is constructed based on the corresponding contract samples. Each data point in the hierarchical dataset corresponds to a contract clause in the contract sample. Each data point in the hierarchical dataset includes the content of the corresponding contract clause, entity information identified from the corresponding contract clause, semantic dependencies between the corresponding contract clause and other contract clauses (such as causal relationships, conditional relationships, constraint relationships, reference relationships, etc.), and the logical level to which the corresponding contract clause belongs. The above-mentioned multiple contract element extraction tasks at different logical levels are defined based on the hierarchical dataset.

[0085] When fine-tuning a pre-trained language model on multiple logical levels of contract element extraction tasks using a hierarchical dataset corresponding to contract samples in a contract sample set, training samples for contract element extraction tasks at different logical levels can be constructed based on the hierarchical dataset corresponding to the contract samples. Then, the training samples can be used to perform supervised fine-tuning of the pre-trained language model (such as a pre-trained BERT model) to obtain a model with the ability to extract contract elements at different logical levels.

[0086] In one possible implementation, multiple logical levels may include a core layer, an associated layer, and a support layer. Any contract clause in the contract sample belongs to one of these three logical levels: the core layer, the associated layer, or the support layer. The core layer contract clauses constitute the static foundation of the contract, defining its basic, overall, and unique elements. The associated layer contract clauses constitute the dynamic execution of the contract, specifying the rights and obligations, performance procedures, and standards. The support layer contract clauses constitute the risk management of the contract, setting out mechanisms for breach of contract, risk allocation, and dispute resolution.

[0087] The contract element extraction tasks at multiple logical levels can include core-level, related-level, and support-level tasks. Specifically, the core-level task involves identifying and extracting elements that constitute the basic framework of the contract (the basic framework refers to the most fundamental and stable set of information that enables a contract to be established, has legal effect, and distinguishes it from other contracts), such as key entities like the contract parties, the subject matter of the contract, and the total contract amount. The related-level task involves identifying and extracting elements that constrain the contract performance process, such as the performance period and payment method. The support-level task involves identifying and extracting elements that address risks associated with contract performance.

[0088] The contract element extraction method provided in this application embodiment can extract contract elements of a target contract based on a target contract element extraction model. That is, this application embodiment will transform from a manual processing mode that relies on personal experience and subjective judgment to an automated and standardized process driven by a professional model. On the one hand, the contract element extraction method provided in this application significantly reduces the processing time of a single contract from several hours required manually to seconds, supporting batch and instantaneous processing of contracts, thereby enabling enterprises to cope with the processing needs of massive contracts. On the other hand, the target contract element extraction model in the contract element extraction method provided in this application uses a target language model as its semantic understanding core. Since the target language model has been fine-tuned on multiple different logical levels of contract element extraction tasks using a hierarchical dataset corresponding to the contract samples, it can not only accurately understand the professional terms in the contract, but also deeply grasp the internal logical structure of the contract and the complex semantic dependencies between clauses. Therefore, based on the above-mentioned deep semantic understanding capabilities, the target contract element extraction model can exhibit high accuracy, consistency and robustness in contract element extraction tasks, effectively eliminating the problem of unstable quality of contract element extraction results caused by differences in personnel background, fluctuations in experience level, subjective judgment bias and fatigue factors in traditional manual processing.

[0089] In addition, compared with rule-based contract element extraction schemes, the contract element extraction method provided in this application does not require writing and maintaining a large number of extraction rules. Since the target language model in the target contract element extraction model has been fine-tuned on contract element extraction tasks at multiple different logical levels based on the hierarchical dataset corresponding to the contract sample, it can better understand and process logically complex contracts.

[0090] The above embodiments mention that the pre-trained language model can be fine-tuned based on the hierarchical dataset corresponding to the contract samples in the contract sample set. In some embodiments of this application, the process of obtaining the hierarchical dataset corresponding to the contract samples is described.

[0091] like Figure 2 As shown, the process of obtaining the hierarchical dataset corresponding to the contract samples may include:

[0092] Step S201: Identify entity information for each contract clause in the contract sample.

[0093] Preferably, the contract sample can be preprocessed to transform it into a standardized contract with a uniform format, no redundant information, clear clause structure, and ease of subsequent natural language processing. The preprocessed contract sample can then be used as the final contract sample. Of course, preprocessing of the contract sample is not required.

[0094] Optionally, a named entity recognition (NAME) model can be used to analyze contract samples to extract core entity information, such as the contracting parties, contract subject matter, amount, date, term, rights and obligations, liability for breach of contract, and agreed terms. The NAME recognition model is trained on contract samples with annotated entities. The initial NAME recognition model may include a pre-trained language model and an entity recognition head. The final NAME recognition model is obtained by training the initial NAME recognition model on the annotated contract samples.

[0095] Step S202: Combine the entity information identified from each contract clause in the contract sample to analyze the semantic dependencies between different contract clauses in the contract sample.

[0096] Optionally, based on the relation extraction model, and combined with entity information identified from each contract clause in the contract sample, the semantic dependencies between different contract clauses in the contract sample can be analyzed. The relation extraction model is trained using training contract clause pairs labeled with semantic dependencies. The initial relation extraction model may include a pre-trained language model and a relation classification head. The final relation extraction model is obtained by training the initial relation extraction model using training contract clause pairs labeled with semantic dependencies.

[0097] For contract clause pairs in a contract sample whose semantic dependencies need to be determined, the language model in the relation extraction model can be used, combined with the entity information identified from the contract clause pair, to mine the contextual semantic interaction features of the contract clause pair. Then, the relation classification head in the relation extraction model can be used to determine whether there is a semantic dependency between the contract clause pairs and the type of semantic dependency when a semantic dependency exists (such as causality, condition, constraint, reference, etc.) based on the contextual semantic interaction features of the contract clause pair.

[0098] Step S203: Construct a directed semantic network graph using each contract clause in the contract sample as a node and the semantic dependencies between different contract clauses in the contract sample as directed edges.

[0099] Nodes are constructed based on each contract clause in the contract sample, and directed edges are constructed between nodes based on the semantic dependencies between different contract clauses in the contract sample, thus forming a directed semantic network graph that reflects the internal logical structure of the contract.

[0100] Each node in the directed semantic network graph corresponds to a contract clause in the contract sample. There is a directed edge between two nodes in the directed semantic network graph, which indicates that there is a semantic dependency between the two nodes. The directed edge between the nodes identifies the logical dependency between the contract clauses. Each directed edge in the directed semantic network graph has an attribute, and the attribute of the directed edge is the category of the semantic dependency.

[0101] Step S204: Perform topological sorting on the nodes in the directed semantic network graph, and divide the contract terms corresponding to each node in the directed semantic network graph into logical levels based on the topological sorting results.

[0102] In one possible implementation, the process of topologically sorting nodes in a directed semantic network graph may include:

[0103] Step S2041: Calculate the in-degree of each node in the directed semantic network graph and add nodes with an in-degree of 0 to the processing queue.

[0104] In a directed semantic network graph, the in-degree of any node is the number of edges that enter that node.

[0105] For example, a sample contract includes the following contract terms:

[0106] C1 (Definition): "Party A refers to XX Technology Co., Ltd., and Party B refers to YY Trading Company."

[0107] C2 (Definition): "Product refers to the smart device with model number X100 as specified in this contract".

[0108] C3 (Amount): "The total contract price is RMB 1 million."

[0109] C4 (Delivery): "Party B shall deliver all products by June 30, 2024."

[0110] C5 (Payment): "Party A shall pay 95% of the total contract price within 30 days after acceptance."

[0111] C6 (Quality): "Products shall conform to the national standard GB / T 12345-2020."

[0112] C7 (Breach of Contract): "If Party B fails to deliver on time, Party B shall pay Party A a penalty of 0.1% of the total contract price for each day of delay."

[0113] C8 (Disputes): "Any dispute arising from the performance of this Contract shall be settled amicably through negotiation between the parties; if the negotiation fails, the dispute shall be submitted to the Beijing Arbitration Commission for arbitration."

[0114] The semantic dependencies between the contract terms in the directed semantic network graph corresponding to the above contract samples are shown below:

[0115] C4←C1: The obligation to deliver requires knowing who "Party B" is;

[0116] C4←C2: The obligation to deliver requires knowing what the "product" is;

[0117] C4←C3: Delivery time is related to the total contract price;

[0118] C5←C1: The obligation to make payments requires knowing who "Party A" is;

[0119] C5←C2: Payment requires knowledge of the "product" being paid for;

[0120] C5←C3: Payment amount is based on the total contract price;

[0121] C5←C4: Payment is conditional upon delivery;

[0122] C6←C2: Quality standards are specific to certain products;

[0123] C7←C1: The breaching party involves "Party B";

[0124] C7←C3: Penalty for breach of contract is calculated based on the total contract price;

[0125] C7←C4: Liability for breach of contract is contingent upon the obligation to deliver;

[0126] C8←C1: Dispute resolution involves two parties.

[0127] Calculate the in-degree of each node (how many edges point to it):

[0128] C1's in-degree: 0;

[0129] C2's in-degree: 0;

[0130] C3's in-degree: 0;

[0131] C4's in-degree: 3;

[0132] C5's in-degree: 4;

[0133] C6's in-degree: 1;

[0134] C7's in-degree: 3;

[0135] C8's in-degree: 1.

[0136] Since the in-degree of C1, C2, and C3 is 0, they are added to the processing queue, which is now [C1, C2, C3]. It should be noted that the initial processing queue is empty.

[0137] Step S2042: Iteratively execute the following process until the processing queue is empty: Take a node from the head of the processing queue and add it to the sorting sequence. At the same time, decrement the in-degree of the successor node of the currently taken node in the directed semantic network graph by 1, and add the successor node with an in-degree reduced to 0 to the tail of the processing queue.

[0138] For the example above, the first iteration is as follows:

[0139] Take C1 from the processing queue [C1, C2, C3] and add it to the sorting sequence. The sorting sequence is now [C1] (the initial sorting queue is empty). Decrement the in-degree of the successor node of C1 in the directed semantic network graph by 1. The successor nodes of C1 are C4, C5, C7, and C8. Decrement the in-degree of C4, C5, C7, and C8 by 1. The in-degree of C4 decreases from 3 to 2, the in-degree of C5 decreases from 4 to 3, the in-degree of C7 decreases from 3 to 2, and the in-degree of C8 decreases from 1 to 0. Since the in-degree of C8 decreases to 0, C8 is added to the processing queue. The processing queue is now [C2, C3, C8].

[0140] The second iteration is as follows:

[0141] Remove C2 from the processing queue [C2, C3, C8] and add it to the sorting sequence. The sorting sequence is now [C1, C2]. Decrement the in-degree of the successor node of C2 in the directed semantic network graph by 1. The successor nodes of C2 are C4, C5, and C6. Decrement the in-degree of C4, C5, and C6 by 1. The in-degree of C4 decreases from 2 to 1, the in-degree of C5 decreases from 3 to 2, and the in-degree of C6 decreases from 1 to 0. Since the in-degree of C6 decreases to 0, add C6 to the processing queue. The processing queue is now [C3, C8, C6].

[0142] The third iteration is as follows:

[0143] Remove C3 from the processing queue [C3, C8, C6] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3]. Decrement the in-degree of the successor node of C3 in the directed semantic network graph by 1. The successor nodes of C3 are C4, C5, and C7. Decrement the in-degree of C4, C5, and C7 by 1. The in-degree of C4 decreases from 1 to 0, the in-degree of C5 decreases from 2 to 1, and the in-degree of C7 decreases from 2 to 1. Since the in-degree of C4 decreases to 0, add C4 to the processing queue. The processing queue is now [C8, C6, C4].

[0144] The fourth iteration is as follows:

[0145] Remove C8 from the processing queue [C8, C6, C4] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3, C8]. Since C8 has no successor node in the directed semantic network graph, this iteration ends.

[0146] The fifth iteration is as follows:

[0147] Remove C6 from the processing queue [C6, C4] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3, C8, C6]. Since C6 has no successor node in the directed semantic network graph, this iteration ends.

[0148] The sixth iteration is as follows:

[0149] Remove C4 from the processing queue [C4] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3, C8, C6, C4]. Decrement the in-degree of the successor node of C4 in the directed semantic network graph by 1. The successor nodes of C4 are C5 and C7. Decrement the in-degree of C5 and C7 by 1. The in-degree of C5 decreases from 1 to 0, and the in-degree of C7 decreases from 1 to 0. Since the in-degree of C5 and C7 has decreased to 0, add C5 and C7 to the processing queue. The processing queue is now [C5, C7].

[0150] The seventh iteration is as follows:

[0151] Remove C5 from the processing queue [C5, C7] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3, C8, C6, C4, C5]. Since C5 has no successor node in the directed semantic network graph, this iteration ends.

[0152] The eighth iteration is as follows:

[0153] Remove C7 from the processing queue [C7] and add it to the sorting sequence. The sorting sequence is now [C1, C2, C3, C8, C6, C4, C5, C7]. Since C7 has no successor node in the directed semantic network graph, this iteration ends.

[0154] The final sorted sequence is [C1, C2, C3, C8, C6, C4, C5, C7], which is the topological sorting result of each node in the directed semantic network graph.

[0155] After obtaining the topological sorting result, i.e. the final sorting sequence, the logical level to which the contract terms corresponding to each node in the directed semantic network graph belong can be determined based on the final sorting sequence.

[0156] For a sequence, the contract terms corresponding to the nodes at the front of the sequence do not depend on any other terms. They are the absolute foundation of the contract and logically, they are the "root" of all other terms. Therefore, they belong to the core layer. The contract terms corresponding to the nodes in the middle of the sequence depend on the preceding terms and may also be depended on by the following terms. They belong to the related layer. The contract terms corresponding to the nodes at the end of the sequence depend only on other terms and are rarely or never depended on. They are like "appendages" of the entire logical chain and are usually used to handle anomalies and risks. They belong to the support layer.

[0157] Optionally, in order to more accurately determine the logical level of the contract terms corresponding to each node in the directed semantic network graph, the logical level of the contract terms corresponding to each node can be determined based on the final sorted sequence, combined with the in-degree, out-degree and business nature of each node.

[0158] For the sorted sequence in the example above, [C1, C2, C3, C8, C6, C4, C5, C7], since C1, C2, and C3 all have an in-degree of 0, are relatively early in the topology, and have high out-degrees (high out-degrees indicate that they are depended on by multiple contract clauses), they can be assigned to the core layer. C8 has an in-degree of 1 (depending on C1), an out-degree of 0 (not depended on), and is in the middle of the topology. Considering that C8 is a dispute resolution clause, which is procedural and backup, and its business nature is risk response, and it is the endpoint in the directed semantic graph (its out-degree is 0), it can be assigned to the support layer. C6 has an in-degree of 1 (depending on C2). With an out-degree of 0 and a later position in the topology, it may belong to the support layer. However, considering the business implications, quality standards affect delivery and acceptance. Functionally, C6 belongs to the execution process. Therefore, it can be classified as the related layer. C4 has an in-degree of 3 (depending on C1, C2, and C3) and an out-degree of 2 (depended on by C5 and C7). Although it is later in the topology, it plays a significant role in connecting the upper and lower layers. Therefore, it can be classified as the related layer. C5 and C7 have high in-degrees (3-4), an out-degree of 0, and are later in the topology. Their business nature is also the final execution terms. Therefore, they can be classified as the support layer.

[0159] Step S205: Based on the logical hierarchy division results, construct the hierarchical dataset corresponding to the contract samples.

[0160] For each contract clause in the contract sample, the content of the contract clause, the entity information identified from the contract clause, the semantic dependencies between the contract clause and other contract clauses, and the logical hierarchy to which the contract clause belongs are integrated into structured data to obtain the structured data corresponding to the contract clause. The structured data corresponding to each clause constitute the hierarchical dataset corresponding to the contract sample.

[0161] After obtaining the hierarchical dataset corresponding to the contract samples, the pre-trained language model can be fine-tuned on multiple contract element extraction tasks at different logical levels based on the hierarchical dataset. After obtaining the fine-tuned model, it can be further optimized. Some embodiments of this application describe the process of optimizing the fine-tuned model.

[0162] like Figure 3 As shown, the process of optimizing the fine-tuned model includes:

[0163] Step S301: Set the initial parameters of the model obtained from fine-tuning as the target parameters, and initialize the optimization control parameters.

[0164] Repeat steps S302 to S305 until the preset optimization termination condition is met:

[0165] Step S302: Based on the current optimized control parameters, perturb the current target parameters to obtain candidate parameters.

[0166] Perturbing the current target parameters means adding random variations to them. It should be noted that the larger the current optimization control parameters are, the greater the perturbation to the current target parameters should be.

[0167] Step S303: Use the test dataset to evaluate the performance of the fine-tuned model under the candidate parameters, and obtain the model performance evaluation value corresponding to the candidate parameters.

[0168] The test dataset includes several test contract samples with labeled contract elements.

[0169] In one possible implementation, the model can be fine-tuned using candidate parameters, and its contract element extraction performance (specifically, the performance in extracting entity information) can be tested using a test dataset to obtain the model's precision (the proportion of correctly extracted elements to the total number of extracted elements) and recall (the proportion of correctly extracted elements to the total number of real elements in the test contract sample). After obtaining the precision and recall, the mean of precision and recall can be calculated (to comprehensively evaluate the model's extraction performance). The precision, recall, and the calculated mean (the mean of precision and recall) can be weighted and summed to obtain the model performance evaluation value corresponding to the candidate parameters.

[0170] Step S304: Based on the model performance evaluation value corresponding to the candidate parameters and the model performance evaluation value corresponding to the current target parameters, decide whether to accept the candidate parameters.

[0171] Specifically, if the model performance evaluation value corresponding to the candidate parameter is better than the model performance evaluation value corresponding to the current target parameter, then the candidate parameter is accepted. If the model performance evaluation value corresponding to the candidate parameter is not better than the model performance evaluation value corresponding to the current target parameter, then the decision on whether to accept the candidate parameter is based on the probability acceptance criterion related to the current optimization control parameter.

[0172] In one possible implementation, the process of deciding whether to accept a candidate parameter based on a probability acceptance criterion related to the current optimization control parameters includes: calculating the difference between the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter; determining the acceptance probability of the candidate parameter based on the difference and the current optimization control parameters; and deciding whether to accept the candidate parameter based on the acceptance probability of the candidate parameter.

[0173] Step S305: If the candidate parameter is accepted, the current target parameter is updated to the candidate parameter, and the current optimized control parameter is reduced according to the preset decay strategy.

[0174] It should be noted that if candidate parameters are not accepted, the current target parameters will remain unchanged.

[0175] In this embodiment, the optimization control parameters are dynamically updated during the optimization process. In the initial stage of optimization, the optimization control parameters are relatively large, and the perturbation amplitude is large, allowing for significant parameter adjustments to explore a wider range of model parameter spaces. As the number of iterations increases, the optimization control parameters and the perturbation amplitude are gradually reduced until the objective function value (model performance evaluation value) converges to the optimal value (i.e., the optimization termination condition is met). The objective function value (model performance evaluation value) converging to the optimal value means that the change in the objective function value (model performance evaluation value) gradually tends to stabilize and the change is less than a preset convergence threshold, reaching a stable state.

[0176] When the optimization termination condition is met, the optimization process ends. The optimized model obtained by fine-tuning is the model with the final target parameters configured.

[0177] After obtaining the target language model, a contract element extraction model based on the target language model can be constructed. Specifically, a clause classification header, entity recognition header, and relation classification header can be configured in the output layer of the target language model to obtain the target contract element extraction model. Optionally, the target contract element extraction model can be fine-tuned on the clause classification task, entity recognition task, and relation classification task. The fine-tuned model serves as the final target contract element extraction model. During fine-tuning, only the parameters of the clause classification header, entity recognition header, and relation classification header can be updated. Of course, in addition to updating the parameters of the clause classification header, entity recognition header, and relation classification header, a small number of parameters of the target language model can also be updated.

[0178] In some embodiments of this application, the process of extracting contract elements from a target contract based on a target contract element extraction model is described.

[0179] like Figure 4 As shown, based on the target contract element extraction model, the process of extracting contract elements from a target contract may include:

[0180] Step S401: Based on the target contract element extraction model, determine the clause type of each contract clause in the target contract from multiple preset clause types.

[0181] In one possible implementation, the process of determining the clause type of each contract clause in the target contract from a set of predefined clause types based on the target contract element extraction model may include: for each contract clause in the target contract, obtaining the word embedding representation of the contract clause (the word embedding representations of each word in the contract clause constitute the word embedding representation of the contract clause); using the target language model in the target contract element extraction model, semantically encoding the word embedding representation of the contract clause to obtain the contextual semantic features of the contract clause; and using the clause classification head in the target contract element extraction model, determining the clause type of the contract clause from a set of predefined clause types (e.g., payment clauses, delivery clauses, breach of contract clauses, etc.) based on the contextual semantic features of the contract clause.

[0182] Step S402: Based on the target contract element extraction model, and combined with the clause type of each contract clause in the target contract, identify and extract entity information from each contract clause in the target contract.

[0183] In one possible implementation, the process of identifying and extracting entity information from each contract clause in the target contract, based on the target contract element extraction model and combined with the clause type of each contract clause in the target contract, may include: for each contract clause in the target contract, forming an input sequence by combining the contract clause and its clause type, and obtaining the word embedding representation of the input sequence (the word embedding representations of each word in the input sequence constitute the word embedding representation of the input sequence); using the target language model in the target contract element extraction model, semantically encoding the word embedding representation of the input sequence to obtain contextual semantic features containing clause type information; using the entity recognition head of the target contract element extraction model, determining the labels corresponding to each word in the contract clause from a preset label set based on the contextual semantic features containing clause type information, and obtaining the label sequence corresponding to the contract clause; and extracting entity information from the contract clause based on the label sequence corresponding to the contract clause. The preset label set includes labels representing various types of entities and labels representing non-entities (e.g., O: non-entity, B-PARTY: contracting party (initiator), I-PARTY: contracting party (internal), etc.).

[0184] Step S403: Based on the target contract element extraction model, and combining the clause types of each contract clause in the target contract with the entity information extracted from each contract clause, analyze the semantic dependencies between different contract clauses in the target contract.

[0185] In one possible implementation, the process of parsing the semantic dependencies between different contract clauses in the target contract, based on the target contract element extraction model and combining the clause types of each contract clause in the target contract with the entity information extracted from each contract clause, may include: constructing contract clause pairs to be parsed for the target contract, resulting in multiple contract clause pairs; for each contract clause pair, constructing the input sequence corresponding to each contract clause in that contract clause pair to obtain the input sequence pair corresponding to that contract clause pair, wherein the input sequence corresponding to each contract clause in that contract clause pair includes the corresponding contract clause, the clause type of the corresponding contract clause, and the entity information extracted from the corresponding contract clause. Extract entity information; obtain word embedding representations of the input sequence pairs corresponding to the contract clause pair; utilize the target language model in the target contract element extraction model to process the word embedding representations of the input sequence pairs corresponding to the contract clause pair, and capture the contextual semantic interaction features of the two contract clauses contained in the contract clause pair; utilize the relation classification head in the target contract element extraction model to determine whether there is a semantic dependency relationship between the two contract clauses contained in the contract clause pair, and the type of semantic dependency relationship (such as causality, condition, constraint, reference, etc.) when a semantic dependency relationship exists, based on the contextual semantic interaction features of the two contract clauses contained in the contract clause pair.

[0186] Step S404: Integrate the clause types of each contract clause in the target contract, the entity information extracted from each contract clause, and the semantic dependencies between different contract clauses into structured data to obtain the structured contract element data corresponding to the target contract.

[0187] Specifically, each contract clause is labeled with its corresponding clause type and used as a basic attribute in the structured data. Each extracted entity is standardized and assigned a unique entity identifier, which is then associated with the corresponding clause attribute. The semantic dependencies between clauses (such as causal relationships, conditional relationships, etc.) are transformed into relational attributes in the structured data to construct clear, unambiguous, and easily analyzable structured data, thus obtaining structured contract element data.

[0188] This application also provides a contract element extraction device, such as... Figure 5 As shown, the contract element extraction device may include a contract acquisition unit 501 and a contract element extraction unit 502.

[0189] Contract acquisition unit 501 is used to acquire the target contract.

[0190] The contract element extraction unit 502 is used to extract contract elements from the target contract based on the target contract element extraction model, and obtain the contract element extraction results of the target contract.

[0191] The target contract element extraction model is a contract element extraction model based on a target language model as the semantic understanding foundation. The target language model is a model obtained by fine-tuning a pre-trained language model on multiple contract element extraction tasks at different logical levels, based on a hierarchical dataset corresponding to the contract samples in the contract sample set, or an optimized model obtained by fine-tuning the model. Each data point in the hierarchical dataset corresponding to the contract samples corresponds to a contract clause in the contract sample. Each data point in the hierarchical dataset corresponding to the contract samples includes the clause content of the corresponding contract clause, the entity information identified from the corresponding contract clause, the semantic dependency relationship between the corresponding contract clause and other contract clauses, and the logical level to which the corresponding contract clause belongs. Multiple contract element extraction tasks at different logical levels are defined based on the hierarchical dataset.

[0192] In one possible implementation, multiple different logical layers include: a core layer, an association layer, and a support layer. The core layer's task for extracting contract elements is to identify and extract the elements that constitute the basic framework of the contract; the association layer's task is to identify and extract the elements that constrain the contract performance process; and the support layer's task is to identify and extract the elements that address contract performance risks.

[0193] In one possible implementation, the contract element extraction device may further include a hierarchical dataset construction unit for constructing a hierarchical dataset corresponding to the contract sample.

[0194] The hierarchical dataset building unit is specifically used to construct the hierarchical dataset corresponding to the contract samples when building the dataset:

[0195] Identify entity information for each contract clause in the contract sample;

[0196] By combining the entity information identified from each contract clause in the contract sample, the semantic dependencies between different contract clauses in the contract sample are analyzed;

[0197] Using each contract clause in the contract sample as a node and the semantic dependencies between different contract clauses in the contract sample as directed edges, a directed semantic network graph is constructed.

[0198] The nodes in the directed semantic network graph are topologically sorted, and the contract terms corresponding to each node in the directed semantic network graph are logically hierarchically divided according to the topological sorting results.

[0199] Based on the logical hierarchy division results, a hierarchical dataset corresponding to the contract samples is constructed.

[0200] In one possible implementation, the hierarchical dataset construction unit, when performing topological sorting on the nodes in the directed semantic network graph and logically hierarchically dividing the contract terms corresponding to each node in the directed semantic network graph based on the topological sorting results, is specifically used for:

[0201] Calculate the in-degree of each node in the directed semantic network graph, and add nodes with an in-degree of 0 to the processing queue.

[0202] The following process is executed iteratively until the processing queue is empty: a node is taken from the head of the processing queue and added to the sorting sequence. At the same time, the in-degree of the successor node of the currently taken node in the directed semantic network graph is reduced by 1, and the successor node with an in-degree reduced to 0 is added to the tail of the processing queue.

[0203] Based on the final sorted sequence, the logical level to which the contract terms corresponding to each node in the directed semantic network graph belong is determined.

[0204] In one possible implementation, the contract element extraction device includes a model fine-tuning unit and a model optimization unit.

[0205] The model fine-tuning unit is used to fine-tune the pre-trained language model on multiple logical levels of contract element extraction tasks based on the hierarchical dataset corresponding to the contract samples in the contract sample set.

[0206] The model optimization unit is used to optimize the model obtained through fine-tuning.

[0207] In one possible implementation, the model optimization unit, when optimizing the fine-tuned model, specifically performs the following:

[0208] Set the initial parameters of the fine-tuned model as the target parameters, initialize the optimization control parameters, and repeat the following steps until the preset optimization termination condition is met:

[0209] Based on the current optimized control parameters, the current target parameters are perturbed to obtain candidate parameters. The larger the current optimized control parameters are, the greater the perturbation to the current target parameters.

[0210] The performance of the fine-tuned model under the candidate parameters is evaluated using the test dataset, and the model performance evaluation value corresponding to the candidate parameters is obtained.

[0211] Based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter, a decision is made on whether to accept the candidate parameter.

[0212] If the candidate parameter is accepted, the current target parameter is updated to the candidate parameter, and the current optimization control parameter is reduced according to the predetermined decay strategy. If the candidate parameter is not accepted, the current target parameter remains unchanged.

[0213] Among them, the optimized model obtained by fine-tuning is the model with the final target parameters configured.

[0214] In one possible implementation, when the model optimization unit decides whether to accept the candidate parameter based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter, it specifically performs the following:

[0215] If the model performance evaluation value corresponding to the candidate parameter is better than the model performance evaluation value corresponding to the current target parameter, then the candidate parameter is accepted.

[0216] If the model performance evaluation value corresponding to the candidate parameter is not better than the model performance evaluation value corresponding to the current target parameter, then a decision is made on whether to accept the candidate parameter based on the probability acceptance criterion related to the current optimization control parameter.

[0217] In one possible implementation, the model optimization unit determines whether to accept the candidate parameters based on a probability acceptance criterion associated with the current optimization control parameters, including:

[0218] Calculate the difference between the model performance evaluation value corresponding to the candidate parameters and the model performance evaluation value corresponding to the current target parameters;

[0219] Based on the difference and the current optimized control parameters, determine the acceptance probability for the candidate parameters;

[0220] The decision on whether to accept a candidate parameter is based on the acceptance probability of that candidate parameter.

[0221] In one possible implementation, when the contract element extraction unit extracts contract elements from the target contract based on the contract element extraction model, it is specifically used for:

[0222] Based on the target contract element extraction model, the clause type of each contract clause in the target contract is determined from multiple preset clause types;

[0223] Based on the target contract element extraction model, and combined with the clause type of each contract clause in the target contract, entity information is identified and extracted from each contract clause in the target contract.

[0224] Based on the target contract element extraction model, and combining the clause types of each contract clause in the target contract with the entity information extracted from each contract clause, the semantic dependencies between different contract clauses in the target contract are analyzed.

[0225] The structured contract element data corresponding to the target contract is obtained by integrating the clause types of each contract clause in the target contract, the entity information extracted from each contract clause, and the semantic dependencies between different contract clauses into structured data.

[0226] In one possible implementation, when the contract element extraction unit 502 determines the clause type of each contract clause in the target contract from a set of preset clause types based on the target contract element extraction model, it is specifically used for:

[0227] For each contract clause in the target contract, obtain the word embedding representation of that contract clause;

[0228] By utilizing the target language model in the target contract element extraction model, the word embedding representation of the contract clause is semantically encoded to obtain the contextual semantic features of the contract clause.

[0229] By utilizing the clause classification header in the target contract element extraction model, the clause type of the contract clause is determined from multiple preset clause types based on the contextual semantic features of the contract clause.

[0230] In one possible implementation, when the contract element extraction unit 502 identifies and extracts entity information from each contract clause in the target contract based on the target contract element extraction model and in conjunction with the clause type of each contract clause in the target contract, it is specifically used for:

[0231] For each contract clause in the target contract, the contract clause and its clause type are combined to form an input sequence, and the word embedding representation of the input sequence is obtained.

[0232] By utilizing the target language model in the target contract element extraction model, semantic encoding is performed on the word embedding representation of the input sequence to obtain contextual semantic features containing clause type information;

[0233] Using the entity recognition head of the target contract element extraction model, based on the contextual semantic features containing clause type information, the tags corresponding to each word in the contract clause are determined from the preset tag set, resulting in the tag sequence corresponding to the contract clause. The preset tag set includes tags representing various entities and tags representing non-entities.

[0234] Extract entity information from the contract terms based on the tag sequence corresponding to those terms.

[0235] In one possible implementation, the contract element extraction unit 502, when analyzing the semantic dependencies between different contract clauses in the target contract based on the target contract element extraction model, combined with the clause types of each contract clause in the target contract and the entity information extracted from each contract clause, is specifically used for:

[0236] For the target contract, construct the contract clause pairs to be parsed, resulting in multiple contract clause pairs;

[0237] For each contract clause pair, an input sequence corresponding to each contract clause in the contract clause pair is constructed to obtain the input sequence pair corresponding to the contract clause pair. Each input sequence in the input sequence pair includes the corresponding contract clause, the clause type of the corresponding contract clause, and entity information extracted from the corresponding contract clause.

[0238] Obtain the word embedding representation of the input sequence pair corresponding to the contract terms;

[0239] By utilizing the target language model in the target contract element extraction model, and processing the word embedding representation of the input sequence pair corresponding to the contract clause pair, the contextual semantic interaction features of the two contract clauses contained in the contract clause pair are captured.

[0240] By utilizing the relation classification head in the target contract element extraction model, and based on the contextual semantic interaction features of the two contract clauses contained in the contract clause pair, it is determined whether there is a semantic dependency relationship between the two contract clauses contained in the contract clause pair, and if so, the type of semantic dependency relationship.

[0241] The contract element extraction device provided in this application embodiment can extract contract elements from a target contract based on a target contract element extraction model. On the one hand, it supports batch and instantaneous processing of contracts, and can meet the processing needs of massive contracts. On the other hand, the target contract element extraction model uses a target language model as its semantic understanding core. Since the target language model has been fine-tuned on contract element extraction tasks at multiple different logical levels using a hierarchical dataset corresponding to the contract samples, it can not only accurately understand the professional terms in the contract, but also deeply grasp the internal logical structure of the contract and the complex semantic dependencies between clauses. Therefore, based on the above-mentioned deep semantic understanding capabilities, the target contract element extraction model can exhibit high accuracy, consistency and robustness in contract element extraction tasks, effectively eliminating the problem of unstable quality of contract element extraction results caused by differences in personnel background, fluctuations in experience level, subjective judgment bias and fatigue factors in traditional manual processing.

[0242] This application also provides an electronic device, which may include at least one processor and a memory connected to the processor.

[0243] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application; the memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0244] The memory is used to store computer programs, and the processor is used to execute the computer programs so that the electronic device can implement the contract element extraction method provided in the above embodiments.

[0245] This application also provides a computer storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device is able to implement the contract element extraction method provided in the above embodiments.

[0246] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the contract element extraction method provided in the above embodiments.

[0247] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0248] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0249] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0250] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for extracting contract elements, characterized in that, include: Obtain the target contract; Based on the target contract element extraction model, contract elements are extracted from the target contract to obtain the contract element extraction results of the target contract. The target contract element extraction model is a contract element extraction model based on a target language model as the semantic understanding foundation. The target language model is a model that is fine-tuned on a pre-trained language model and optimized on contract element extraction tasks at multiple different logical levels, based on a hierarchical dataset corresponding to contract samples in the contract sample set. Each data point in the hierarchical dataset corresponds to a contract clause in the contract sample. Each data point includes the clause content, entity information identified from the contract clause, semantic dependencies between the contract clause and other contract clauses, and the logical level to which the contract clause belongs. The multiple contract element extraction tasks at different logical levels are defined based on the hierarchical dataset. These multiple logical levels include a core layer, an association layer, and a support layer. The contract element extraction task at the core layer is to identify and extract the elements that constitute the basic framework of the contract. The content of the contract clauses at the core layer defines the basic, global, and unique elements of the contract. The contract element extraction task at the association layer is to identify and extract the elements that constrain the contract performance process. The content of the contract clauses at the association layer stipulates the specific rights and obligations, performance procedures, and standards. The contract element extraction task at the support layer is to identify and extract the elements that address the risks of contract performance. The content of the contract clauses at the support layer sets out the liability for breach of contract, risk allocation, and dispute resolution mechanisms. The process of obtaining the logical level to which each contract clause in the contract sample belongs includes: Analyze the semantic dependencies between different contract clauses in the contract sample; Using each contract clause in the contract sample as a node and the semantic dependencies between different contract clauses in the contract sample as directed edges, a directed semantic network graph is constructed. The nodes in the directed semantic network graph are topologically sorted, and based on the topological sorting results, combined with the in-degree, out-degree and business nature of each node, the contract terms corresponding to each node in the directed semantic network graph are logically divided into levels to obtain the logical level to which each contract term in the contract sample belongs.

2. The method for extracting contract elements according to claim 1, characterized in that, The process of obtaining the hierarchical dataset corresponding to the contract sample includes: Obtain entity information for each contract clause identified in the contract sample, and obtain the semantic dependencies between different contract clauses in the contract sample and the logical level to which each contract clause in the contract sample belongs; For each contract clause in the contract sample, the content of the contract clause, the entity information identified from the contract clause, the semantic dependencies between the contract clause and other contract clauses, and the logical hierarchy to which the contract clause belongs are integrated into structured data to obtain the structured data corresponding to the contract clause. The structured data corresponding to each contract clause in the contract sample constitute the hierarchical dataset corresponding to the contract sample.

3. The method for extracting contract elements according to claim 1, characterized in that, The process involves performing a topological sort on the nodes in the directed semantic network graph, and based on the sorting results, combined with the in-degree, out-degree, and business nature of each node, logically hierarchically dividing the contract terms corresponding to each node in the directed semantic network graph, including: Calculate the in-degree of each node in the directed semantic network graph, and add nodes with an in-degree of 0 to the processing queue. The following process is executed iteratively until the processing queue is empty: a node is taken from the head of the processing queue and added to the sorting sequence, while the in-degree of the successor node of the currently taken node in the directed semantic network graph is reduced by 1, and the successor node whose in-degree is reduced to 0 is added to the tail of the processing queue. Based on the final sorted sequence, and combined with the in-degree, out-degree, and business nature of each node, the logical level to which the contract terms corresponding to each node in the directed semantic network graph belong is determined.

4. The method for extracting contract elements according to claim 1, characterized in that, The process of optimizing the fine-tuned model includes: Set the initial parameters of the fine-tuned model as the target parameters, initialize the optimization control parameters, and repeat the following steps until the preset optimization termination condition is met: Based on the current optimized control parameters, the current target parameters are perturbed to obtain candidate parameters. The larger the current optimized control parameters are, the greater the perturbation to the current target parameters. The performance of the fine-tuned model under the candidate parameters is evaluated using a test dataset to obtain the model performance evaluation value corresponding to the candidate parameters. Based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter, a decision is made on whether to accept the candidate parameter. If the candidate parameter is accepted, the current target parameter is updated to the candidate parameter, and the current optimization control parameter is reduced according to the predetermined decay strategy; if the candidate parameter is not accepted, the current target parameter remains unchanged. Among them, the optimized model obtained by fine-tuning is the model with the final target parameters configured.

5. The method for extracting contract elements according to claim 4, characterized in that, The step of determining whether to accept the candidate parameter based on the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter includes: If the model performance evaluation value corresponding to the candidate parameter is better than the model performance evaluation value corresponding to the current target parameter, then the candidate parameter is accepted. If the model performance evaluation value corresponding to the candidate parameter is not better than the model performance evaluation value corresponding to the current target parameter, then a decision is made on whether to accept the candidate parameter based on the probability acceptance criterion related to the current optimization control parameter.

6. The method for extracting contract elements according to claim 5, characterized in that, The decision on whether to accept the candidate parameter based on the probability acceptance criterion related to the current optimized control parameters includes: Calculate the difference between the model performance evaluation value corresponding to the candidate parameter and the model performance evaluation value corresponding to the current target parameter; Based on the difference and the current optimized control parameters, determine the acceptance probability for the candidate parameter; Whether to accept the candidate parameter is determined based on the acceptance probability of the candidate parameter.

7. The method for extracting contract elements according to claim 1, characterized in that, The extraction of contract elements from the target contract based on the target contract element extraction model includes: Based on the target contract element extraction model, the clause type of each contract clause in the target contract is determined from a set of preset clause types; Based on the target contract element extraction model, and combined with the clause type of each contract clause in the target contract, entity information is identified and extracted from each contract clause in the target contract. Based on the target contract element extraction model, and combining the clause types of each contract clause in the target contract with the entity information extracted from each contract clause, the semantic dependencies between different contract clauses in the target contract are analyzed. The clause types of each contract clause in the target contract, the entity information extracted from each contract clause, and the semantic dependencies between different contract clauses are integrated into structured data to obtain the structured contract element data corresponding to the target contract.

8. The method for extracting contract elements according to claim 7, characterized in that, The step of determining the clause type of each contract clause in the target contract from a preset set of multiple clause types based on the target contract element extraction model includes: For each contract clause in the target contract, obtain the word embedding representation of that contract clause; Using the target language model in the target contract element extraction model, the word embedding representation of the contract clause is semantically encoded to obtain the contextual semantic features of the contract clause. Using the clause classification header in the target contract element extraction model, the clause type of the contract clause is determined from multiple preset clause types based on the contextual semantic features of the contract clause.

9. The method for extracting contract elements according to claim 7, characterized in that, The step of identifying and extracting entity information from each contract clause in the target contract based on the target contract element extraction model, combined with the clause type of each contract clause in the target contract, includes: For each contract clause in the target contract, the contract clause and its clause type are combined to form an input sequence, and the word embedding representation of the input sequence is obtained. Using the target language model in the target contract element extraction model, the word embedding representation of the input sequence is semantically encoded to obtain contextual semantic features containing clause type information; Using the entity recognition head of the target contract element extraction model, based on the contextual semantic features containing clause type information, the tags corresponding to each word in the contract clause are determined from the preset tag set, resulting in the tag sequence corresponding to the contract clause. The preset tag set includes tags representing various entities and tags representing non-entities. Extract entity information from the contract terms based on the tag sequence corresponding to those terms.

10. The method for extracting contract elements according to claim 7, characterized in that, The method based on the target contract element extraction model, combining the clause types of each contract clause in the target contract and the entity information extracted from each contract clause, analyzes the semantic dependencies between different contract clauses in the target contract, including: For the target contract, construct contract clause pairs to be parsed, resulting in multiple contract clause pairs; For each contract clause pair, an input sequence corresponding to each contract clause in the contract clause pair is constructed to obtain the input sequence pair corresponding to the contract clause pair. Each input sequence in the input sequence pair includes the corresponding contract clause, the clause type of the corresponding contract clause, and entity information extracted from the corresponding contract clause. Obtain the word embedding representation of the input sequence pair corresponding to the contract terms; By utilizing the target language model in the target contract element extraction model, and processing the word embedding representation of the input sequence pair corresponding to the contract clause pair, the contextual semantic interaction features of the two contract clauses contained in the contract clause pair are captured. Using the relation classification head in the target contract element extraction model, based on the contextual semantic interaction features of the two contract clauses contained in the contract clause pair, it is determined whether there is a semantic dependency relationship between the two contract clauses contained in the contract clause pair, and if so, the type of semantic dependency relationship.

11. A method for extracting contract elements, characterized in that, include: Obtain the target contract; Based on the target contract element extraction model, contract elements are extracted from the target contract to obtain the contract element extraction results of the target contract. The target contract element extraction model is a contract element extraction model based on a target language model as the semantic understanding foundation. The target language model is a model obtained by fine-tuning a pre-trained language model on multiple contract element extraction tasks at different logical levels, based on a hierarchical dataset corresponding to contract samples in the contract sample set. Each data point in the hierarchical dataset corresponds to a contract clause in the contract sample. Each data point includes the clause content, entity information identified from the contract clause, semantic dependencies between the contract clause and other contract clauses, and the logical level to which the contract clause belongs. The multiple contract element extraction tasks at different logical levels are defined based on the hierarchical dataset. These multiple logical levels include a core layer, an association layer, and a support layer. The contract element extraction task at the core layer is to identify and extract the elements that constitute the basic framework of the contract. The content of the contract clauses at the core layer defines the basic, global, and unique elements of the contract. The contract element extraction task at the association layer is to identify and extract the elements that constrain the contract performance process. The content of the contract clauses at the association layer stipulates the specific rights and obligations, performance procedures, and standards. The contract element extraction task at the support layer is to identify and extract the elements that address the risks of contract performance. The content of the contract clauses at the support layer sets out the liability for breach of contract, risk allocation, and dispute resolution mechanisms. The process of obtaining the logical level to which each contract clause in the contract sample belongs includes: Analyze the semantic dependencies between different contract clauses in the contract sample; Using each contract clause in the contract sample as a node and the semantic dependencies between different contract clauses in the contract sample as directed edges, a directed semantic network graph is constructed. The nodes in the directed semantic network graph are topologically sorted, and based on the topological sorting results, combined with the in-degree, out-degree and business nature of each node, the contract terms corresponding to each node in the directed semantic network graph are logically divided into levels to obtain the logical level to which each contract term in the contract sample belongs.

12. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the contract element extraction method as described in any one of claims 1 to 11.

13. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the contract element extraction method as described in any one of claims 1 to 11.

14. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the contract element extraction method as described in any one of claims 1 to 11.

Images