A method and system for extracting elements of dispute mediation based on reading comprehension

Abstract

The present application relates to the technical field of legal element extraction, in particular to a dispute mediation element extraction method and system based on reading comprehension, the method comprises the following steps: loading pre-training model parameters of a Bert-Base Chinese model, fine-tuning the model based on unlabeled dispute mediation texts, coarse-grained fine-tuning the model based on a Chinese judicial field data set, fine-grained fine-tuning the model based on a labeled dispute mediation data set, and extracting dispute mediation elements through the adjusted model; the present application fine-tunes the model based on unlabeled dispute mediation texts, and fine-grained fine-tunes the model based on a labeled dispute mediation data set after the original fine-tuning stage, and fine-grained fine-tunes the model by selecting a small sample of labeled dispute mediation data set, so that the model used in the present application understands the elements to be extracted in different types of dispute cases better, and is more suitable for dispute mediation element extraction.

Description

Technical Field

[0001] This invention relates to the field of legal element extraction technology, specifically to a method and system for extracting dispute mediation elements based on reading comprehension. Background Technology

[0002] Element extraction is an important task in the legal field. Civil disputes constitute an increasingly high proportion of all types of judicial cases, and their resolution primarily relies on mediation. Because dispute mediation files are often lengthy, extracting key elements is necessary to understand the dispute.

[0003] Feature extraction often utilizes machine reading comprehension techniques, which can naturally extract fine-grained and unconstrained information. A reading comprehension task can be understood as, given a question and a text containing the answer, having the model extract a continuous sequence from the text as the answer to the question. In dispute mediation files, feature extraction uses the case content as the input text to the model, the feature names as the question, and the content of the features to be extracted as the answer. The start and end positions of the answer are then calculated.

[0004] There are many types of disputes, including private lending disputes, labor disputes, and work injury disputes. When extracting elements for different types of disputes, the elements to be extracted are not the same and can be as many as thirty. This requires the use of a large-scale dataset when training the reading comprehension model used for dispute element extraction, so as to ensure the correctness and accuracy of the extracted elements.

[0005] However, current research does not provide datasets related to dispute cases. The datasets available for training are Chinese legal datasets, which are broader in scope but less targeted. Reading comprehension models trained directly using these datasets will result in poor performance when extracting dispute elements.

[0006] At the same time, even if a dispute case dataset is created, there is still a problem of insufficient samples due to the limited amount of publicly available data related to dispute cases. This is because manually annotating various elements of different types of dispute cases consumes a lot of time and energy, which leads to the existing reading comprehension model being unable to fully learn the characteristics of dispute cases, affecting the accuracy and precision of extracting dispute elements. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, this invention aims to provide a method and system for extracting dispute mediation elements based on reading comprehension, in order to solve the problem that the limited amount of dispute case data results in poor targeting and low accuracy in the extraction of dispute case elements.

[0008] To solve the above problems, the present invention adopts the following technical solution:

[0009] This patent's method for extracting elements from dispute mediation based on reading comprehension can be divided into four stages: loading parameters of the Bert-Based Chinese pre-trained model, fine-tuning the model based on unlabeled dispute mediation text, coarse-grained fine-tuning the model based on a Chinese judicial dataset, and fine-grained fine-tuning the model based on a labeled dispute mediation dataset.

[0010] The model is fine-tuned based on unlabeled dispute mediation text. Similar to one of the pre-training tasks, this task involves masking some words in the unlabeled text and using a "masked language model" to predict the masked words. Specifically, given the limited amount of manually labeled dispute mediation data, by masking some words in the unlabeled dispute mediation text and using the model to predict the masked words, the model is fine-tuned. This allows the model to learn the features of the dispute mediation text and generate text representations with richer semantic information, especially more information related to dispute mediation, effectively addressing the problem of insufficient semantic information caused by the limited sample size of the dispute mediation dataset.

[0011] The model is fine-tuned using a labeled dispute mediation dataset. Existing Chinese judicial datasets cover a wide range of topics, and coarse-grained tuning using these datasets is not targeted enough. Therefore, a smaller labeled dispute mediation dataset is selected for fine-grained tuning, allowing the model to better understand the elements to be extracted from different types of disputes and making it more suitable for extracting dispute mediation elements.

[0012] Specifically, the first aspect of this invention provides a method for extracting elements of dispute mediation based on reading comprehension, comprising:

[0013] Load the pre-trained model parameters of the Bert-Based Chinese model;

[0014] The model was fine-tuned based on unlabeled dispute mediation texts;

[0015] The model was fine-tuned using a Chinese judicial dataset.

[0016] The model was fine-tuned based on a labeled dispute mediation dataset;

[0017] The elements of dispute mediation are extracted using the adjusted model.

[0018] As one possible implementation, the pre-trained model parameters for loading the Bert-Based Chinese model include:

[0019] Download and cache the pre-trained model file of the Bert-Base Chinese model, then load the file, read the model parameters after pre-training, and use the model parameters to initialize the Bert-Base Chinese model; the model has 12 layers, a hidden size of H=768, and 12 self-attention points.

[0020] As one possible implementation, the fine-tuning of the model based on unlabeled dispute mediation text includes:

[0021] Preprocess the input sequence of the model;

[0022] The preprocessed input sequence is fed into the model, which calculates the final hidden vector corresponding to the occluded word. This vector is then fed into the vocabulary-based output softmax function to obtain the occluded word predicted based on context. The final hidden vector of the i-th input word is... ; Final hidden vector Use cross-entropy loss Predicting original words; cross-entropy loss for:

[0023]

[0024] in, For predictions using the softmax function The probability of.

[0025] As one possible implementation, the preprocessing of the model's input sequence includes:

[0026] A sequence consisting of a single sentence or a pair of sentences, with a special classification label [CLS] used for the first label of each sequence;

[0027] For sequences of sentence pairs, an additional special marker [SEP] is used to separate them;

[0028] The sequence was segmented using WordPiece embeddings and a vocabulary of 30,000 words;

[0029] 15% of all WordPiece words in each input sequence are randomly masked. During the random masking process, [mask] is used 80% of the time, random words are used 10% of the time, and the original words are used for the remaining 10% of the time.

[0030] For all words in the input sequence, the input representation of the model is constructed by summing the corresponding word, segment, and position embeddings;

[0031] Before inputting a sequence of sentence pairs, a learned embedding is added to each word to indicate which sentence it belongs to.

[0032] As one possible implementation, the coarse-grained fine-tuning of the model based on a Chinese judicial domain dataset includes:

[0033] Each question in the Chinese judicial domain dataset is concatenated with the text containing the answer and represented as a single sequence. The first label of this sequence is [CLS], and the question and text are separated by the label [SEP]. For questions with the answer type of yes or no, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels, and the sequence is segmented using WordPiece embedding and a vocabulary of 30,000 words.

[0034] Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ;

[0035] Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction.

[0036] In the Chinese judicial domain dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker.

[0037] The loss function for coarse-grained fine-tuning is the sum of the negative log-likelihoods of the correct start and end positions:

[0038]

[0039] in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

[0040] As one possible implementation method, the fine-grained tuning of the model based on the labeled dispute mediation dataset includes:

[0041] Based on the dispute mediation dataset, different questions in the same text within the dataset are traversed and concatenated with the text to form different sequences. Each question in the dispute mediation dataset is represented as a single sequence by concatenating it with the text containing the answer. The first label of this sequence is [CLS], and the question and text are separated by the [SEP] label. For questions with a yes or no answer type, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels. WordPiece embedding and a vocabulary of 30,000 words are used to segment the sequence.

[0042] Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ;

[0043] Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction.

[0044] In the dispute mediation dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker.

[0045] The loss function for fine-grained tuning is the sum of the negative log-likelihoods of the correct start and end positions:

[0046]

[0047] in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

[0048] As one possible implementation, the loss function of the adjusted model for:

[0049] .

[0050] Secondly, the present invention provides a dispute mediation element extraction system based on reading comprehension, including a loading module, a fine-tuning module, a coarse-grained fine-tuning module, a fine-grained fine-tuning module, and an extraction module;

[0051] The loading module is used to load the pre-trained model parameters of the Bert-Based Chinese model;

[0052] The fine-tuning module is used to fine-tune the model based on unlabeled dispute mediation text;

[0053] The coarse-grained fine-tuning module is used to perform coarse-grained fine-tuning of the model based on a Chinese judicial data set.

[0054] The fine-grained fine-tuning module is used to fine-tune the model based on a labeled dispute mediation dataset.

[0055] The extraction module is used to extract dispute mediation elements using the adjusted model.

[0056] Thirdly, the present invention provides an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the above-described method for extracting elements of dispute mediation based on reading comprehension.

[0057] Fourthly, the present invention provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the above-described method for extracting elements of dispute mediation based on reading comprehension.

[0058] The beneficial effects of this invention are as follows: This invention employs two fine-tuning stages for specific dispute mediation. Between the original pre-training stage and the fine-tuning stage, a fine-tuning of the model based on unlabeled dispute mediation text is added, followed by a fine-grained fine-tuning of the model based on labeled dispute mediation datasets after the original fine-tuning stage. Fine-grained fine-tuning of the model based on labeled dispute mediation datasets covers a wider range of Chinese judicial data. Therefore, selecting a small sample of labeled dispute mediation datasets for fine-grained fine-tuning allows the model used in this method to better understand the elements to be extracted from different types of dispute cases, making it more suitable for dispute mediation element extraction. Attached Figure Description

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0060] Figure 1 This is a schematic diagram of a method for extracting elements of dispute mediation based on reading comprehension, according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of step S200 in an embodiment of the present invention.

[0062] Figure 3This is a schematic diagram of step S300 in an embodiment of the present invention.

[0063] Figure 4 This is a schematic diagram of a dispute mediation element extraction system based on reading comprehension, according to an embodiment of the present invention. Detailed Implementation

[0064] The present invention will be further described in detail below with reference to specific embodiments.

[0065] It should be noted that these embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Simple improvements to the method under the premise of the present invention are all within the scope of protection claimed by the present invention.

[0066] See Figure 1 This is a method for extracting elements of dispute mediation based on reading comprehension, including:

[0067] S100, Load the pre-trained model parameters of the Bert-Based Chinese model;

[0068] S200. Fine-tune the model based on unlabeled dispute mediation text;

[0069] S300: The model is fine-tuned using a Chinese judicial data set;

[0070] S400: Fine-tune the model based on a labeled dispute mediation dataset;

[0071] S500: Extract dispute mediation elements using the adjusted model.

[0072] This invention employs two fine-tuning stages for specific dispute mediation. Between the original pre-training and fine-tuning stages, a fine-tuning based on unlabeled dispute mediation texts is added, followed by a fine-grained fine-tuning based on labeled dispute mediation datasets after the original fine-tuning stage. Fine-grained fine-tuning based on labeled dispute mediation datasets is more comprehensive than using Chinese judicial datasets, therefore a smaller sample of labeled dispute mediation datasets is selected for fine-grained fine-tuning. This allows the model used in this method to better understand the elements to be extracted from different types of dispute cases, making it more suitable for dispute mediation element extraction.

[0073] As one possible implementation, step S100, loading the pre-trained model parameters of the Bert-Based Chinese model, includes:

[0074] Download and cache the pre-trained model file of the Bert-Base Chinese model, then load the file, read the model parameters after pre-training, and use the model parameters to initialize the Bert-Base Chinese model; the model has 12 layers, a hidden size of H=768, and 12 self-attention points.

[0075] As one possible implementation method, see Figure 2 Step 200: Fine-tune the model based on the unlabeled dispute mediation text, including:

[0076] The input sequence of the model is preprocessed; the preprocessing of the input sequence of the model specifically includes:

[0077] A sequence consisting of a single sentence or a pair of sentences, with a special classification label [CLS] used for the first label of each sequence;

[0078] For sequences of sentence pairs, an additional special marker [SEP] is used to separate them;

[0079] The sequence was segmented using WordPiece embeddings and a vocabulary of 30,000 words;

[0080] 15% of all WordPiece words in each input sequence are randomly masked. During the random masking process, [mask] is used 80% of the time, random words are used 10% of the time, and the original words are used for the remaining 10% of the time.

[0081] For all words in the input sequence, the input representation of the model is constructed by summing the corresponding word, segment, and position embeddings;

[0082] Before inputting a sequence of sentence pairs, a learned embedding is added to each word to indicate which sentence it belongs to.

[0083] The preprocessed input sequence is fed into the model, which calculates the final hidden vector corresponding to the occluded word. This vector is then fed into the vocabulary-based output softmax function to obtain the occluded word predicted based on context. The final hidden vector of the i-th input word is... ; Final hidden vector Use cross-entropy loss Predicting original words; cross-entropy loss for:

[0084]

[0085] in, For predictions using the softmax function The probability of.

[0086] As one possible implementation method, see Figure 3 Step S300: Perform coarse-grained fine-tuning of the model based on the Chinese judicial data set, including:

[0087] Each question in the Chinese judicial domain dataset is concatenated with the text containing the answer and represented as a single sequence. The first label of this sequence is [CLS], and the question and text are separated by the label [SEP]. For questions with the answer type of yes or no, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels, and the sequence is segmented using WordPiece embedding and a vocabulary of 30,000 words.

[0088] Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ;

[0089] Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction.

[0090] In the Chinese judicial domain dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker.

[0091] The loss function for coarse-grained fine-tuning is the sum of the negative log-likelihoods of the correct start and end positions:

[0092]

[0093] in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

[0094] As one possible implementation, step S400, fine-tuning the model based on the labeled dispute mediation dataset, includes:

[0095] The approach in this stage is similar to that of coarse-grained fine-tuning, with the main difference being the content and format of the dataset. Coarse-grained fine-tuning uses a broader and less targeted dataset from the Chinese judicial field, while fine-grained fine-tuning uses anonymized real dispute mediation data. Furthermore, the Chinese judicial dataset only requires extracting one answer to one question from a text, whereas dispute mediation element extraction requires extracting content corresponding to multiple elements from a text. Therefore, before performing similar operations, it is necessary to iterate through different questions within the same text and concatenate them with the text to form different sequences.

[0096] Specifically:

[0097] Based on the dispute mediation dataset, different questions in the same text within the dataset are traversed and concatenated with the text to form different sequences. Each question in the dispute mediation dataset is represented as a single sequence by concatenating it with the text containing the answer. The first label of this sequence is [CLS], and the question and text are separated by the [SEP] label. For questions with a yes or no answer type, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels. WordPiece embedding and a vocabulary of 30,000 words are used to segment the sequence.

[0098] Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ;

[0099] Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction.

[0100] In the dispute mediation dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker.

[0101] The loss function for fine-grained tuning is the sum of the negative log-likelihoods of the correct start and end positions:

[0102]

[0103] in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

[0104] As one possible implementation, the loss function of the adjusted model for:

[0105] .

[0106] See Figure 4 This is a dispute mediation element extraction system based on reading comprehension, including a loading module 100, a fine-tuning module 200, a coarse-grained fine-tuning module 300, a fine-grained fine-tuning module 400, and an extraction module 500.

[0107] The loading module 100 is used to load the pre-trained model parameters of the Bert-Based Chinese model;

[0108] The fine-tuning module 200 is used to fine-tune the model based on the unlabeled dispute mediation text;

[0109] The coarse-grained fine-tuning module 300 is used to perform coarse-grained fine-tuning of the model based on a Chinese judicial domain dataset.

[0110] The fine-grained fine-tuning module 400 is used to fine-tune the model based on the labeled dispute mediation dataset;

[0111] The extraction module 500 is used to extract dispute mediation elements through the adjusted model.

[0112] The present invention provides an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the above-described method for extracting elements of dispute mediation based on reading comprehension.

[0113] The present invention provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the above-described method for extracting elements of dispute mediation based on reading comprehension.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for extracting elements of dispute mediation based on reading comprehension, characterized in that, include: Load the pre-trained model parameters of the Bert-Based Chinese model; The model was fine-tuned based on unlabeled dispute mediation texts; The model was fine-tuned using a Chinese judicial dataset. The model was fine-tuned based on a labeled dispute mediation dataset; The elements of dispute mediation are extracted using the adjusted model; The fine-tuning of the model based on unlabeled dispute mediation text includes: Preprocess the input sequence of the model; The preprocessed input sequence is fed into the model, which calculates the final hidden vector corresponding to the occluded word. This vector is then fed into the vocabulary-based output softmax function to obtain the occluded word predicted based on context. The final hidden vector of the i-th input word is... ; Final hidden vector Use cross-entropy loss Predicting original words; cross-entropy loss for: in, For predictions using the softmax function The probability of; The coarse-grained fine-tuning of the model based on a Chinese judicial data set includes: Each question in the Chinese judicial domain dataset is concatenated with the text containing the answer and represented as a single sequence. The first label of this sequence is [CLS], and the question and text are separated by the label [SEP]. For questions with the answer type of yes or no, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels, and the sequence is segmented using WordPiece embedding and a vocabulary of 30,000 words. Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ; Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction. In the Chinese judicial domain dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker. The loss function for coarse-grained fine-tuning is the sum of the negative log-likelihoods of the correct start and end positions: in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1. The fine-grained tuning of the model based on the labeled dispute mediation dataset includes: Based on the dispute mediation dataset, different questions in the same text within the dataset are traversed and concatenated with the text to form different sequences. Each question in the dispute mediation dataset is represented as a single sequence by concatenating it with the text containing the answer. The first label of this sequence is [CLS], and the question and text are separated by the [SEP] label. For questions with a yes or no answer type, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels. WordPiece embedding and a vocabulary of 30,000 words are used to segment the sequence. Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ; Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction. In the dispute mediation dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker. The loss function for fine-grained tuning is the sum of the negative log-likelihoods of the correct start and end positions: in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

2. The method for extracting dispute mediation elements based on reading comprehension according to claim 1, characterized in that, The pre-trained model parameters for loading the Bert-Based Chinese model include: Download and cache the pre-trained model file of the Bert-Base Chinese model, then load the file, read the model parameters after pre-training, and use the model parameters to initialize the Bert-Base Chinese model; the model has 12 layers, a hidden size of H=768, and 12 self-attention points.

3. The method for extracting dispute mediation elements based on reading comprehension according to claim 1, characterized in that, The preprocessing of the input sequence of the model includes: A sequence consisting of a single sentence or a pair of sentences, with a special classification label [CLS] used for the first label of each sequence; For sequences of sentence pairs, an additional special marker [SEP] is used to separate them; The sequence was segmented using WordPiece embeddings and a vocabulary of 30,000 words; 15% of all WordPiece words in each input sequence are randomly masked. During the random masking process, [mask] is used 80% of the time, random words are used 10% of the time, and the original words are used for the remaining 10% of the time. For all words in the input sequence, the input representation of the model is constructed by summing the corresponding word, segment, and position embeddings; Before inputting a sequence of sentence pairs, a learned embedding is added to each word to indicate which sentence it belongs to.

4. The method for extracting dispute mediation elements based on reading comprehension according to claim 1, characterized in that, The loss function of the adjusted model for: 。 5. A dispute mediation element extraction system based on reading comprehension, characterized in that, It includes a loading module, a fine-tuning module, a coarse-grained fine-tuning module, a fine-grained fine-tuning module, and an extraction module; The loading module is used to load the pre-trained model parameters of the Bert-Based Chinese model; The fine-tuning module is used to fine-tune the model based on unlabeled dispute mediation text; The coarse-grained fine-tuning module is used to perform coarse-grained fine-tuning of the model based on a Chinese judicial data set. The fine-grained fine-tuning module is used to fine-tune the model based on a labeled dispute mediation dataset. The extraction module is used to extract dispute mediation elements through the adjusted model; The fine-tuning of the model based on unlabeled dispute mediation text includes: Preprocess the input sequence of the model; The preprocessed input sequence is fed into the model, which calculates the final hidden vector corresponding to the occluded word. This vector is then fed into the vocabulary-based output softmax function to obtain the occluded word predicted based on context. The final hidden vector of the i-th input word is... ; Final hidden vector Use cross-entropy loss Predicting original words; cross-entropy loss for: in, For predictions using the softmax function The probability of; The coarse-grained fine-tuning of the model based on a Chinese judicial data set includes: Each question in the Chinese judicial domain dataset is concatenated with the text containing the answer and represented as a single sequence. The first label of this sequence is [CLS], and the question and text are separated by the label [SEP]. For questions with the answer type of yes or no, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels, and the sequence is segmented using WordPiece embedding and a vocabulary of 30,000 words. Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ; Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction. In the Chinese judicial domain dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker. The loss function for coarse-grained fine-tuning is the sum of the negative log-likelihoods of the correct start and end positions: in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1. The fine-grained tuning of the model based on the labeled dispute mediation dataset includes: Based on the dispute mediation dataset, different questions in the same text within the dataset are traversed and concatenated with the text to form different sequences. Each question in the dispute mediation dataset is represented as a single sequence by concatenating it with the text containing the answer. The first label of this sequence is [CLS], and the question and text are separated by the [SEP] label. For questions with a yes or no answer type, during the data processing stage, [YES] and [NO] are concatenated with the text content as two labels. WordPiece embedding and a vocabulary of 30,000 words are used to segment the sequence. Introducing a start vector S and an end vector E, calculate the pair The dot product between S and S is applied to a softmax function based on all words in the text to calculate the probability of the first word i in the answer: ; Replacing S with E in the formula, the score for the candidate answer from position i to position j is defined as... When j≥i, the segment with the highest score is used for prediction. In the dispute mediation dataset, if an instance contains all labeled short answers, the start and end target indices are set to point to the shortest text containing all labeled short answers; if an instance does not contain any labeled short answers, but completely contains a labeled long answer, the start and end target indices are set to point to the entire long text; if neither a short nor a long answer is found in the current instance, the target start and end indices are set to point to the [CLS] marker. The loss function for fine-grained tuning is the sum of the negative log-likelihoods of the correct start and end positions: in This represents the probability that the predicted value is the correct starting position, and its value is either 0 or 1. This represents the probability that the predicted value is the correct ending position, and its value is either 0 or 1.

6. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the dispute mediation element extraction method based on reading comprehension as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the dispute mediation element extraction method based on reading comprehension as described in any one of claims 1 to 4.

Images