Description type judicial knowledge generation method and device driven by judicial data

Abstract

The invention relates to a method and a device for generating description-type judicial knowledge driven by judicial data. The method comprises the following steps: extracting legal concepts and / or legal terms from a legal text, and constructing judicial concept knowledge ontology based on the legal concepts and / or the legal terms; based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained, the judicial concept extraction model is used for generating descriptive judicial knowledge, and the descriptive judicial knowledge comprises specific vocabularies or phrases which have specific legal definitions and serve as judgment elements in judicial judgment. According to the embodiment of the invention, the description-type judicial knowledge can be accurately generated.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a judicial data-driven descriptive judicial knowledge generation method and device. Background Art

[0002] In the judicial knowledge system of the People's Court, descriptive judicial knowledge is the relevant content that explains the essential connotation of objective things related to justice, in order to assist the information system in understanding the essence of objective things, such as: litigation process, trial process, etc.; relational knowledge is the relevant content that describes the relationship between subject objects such as case procedures, parties, judges, lawyers, law firms, legal elements of the case, court affiliation, etc., such as: case-case relationship, person-case relationship, etc.

[0003] It can be seen that accurately extracting descriptive judicial knowledge is of great significance, and how to accurately extract descriptive judicial knowledge is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present disclosure proposes a judicial data-driven descriptive judicial knowledge generation method and device, which can accurately generate descriptive judicial knowledge.

[0005] According to one aspect of the present disclosure, a method for generating descriptive judicial knowledge driven by judicial data is provided, the method comprising: Extracting legal concepts and / or legal terms from legal texts, and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms; Based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained. The judicial concept extraction model is used to generate descriptive judicial knowledge. The descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

[0006] In a possible implementation, extracting legal concepts and / or legal terms from legal texts and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms includes: generating a candidate concept set according to the legal concepts and / or legal terms extracted from the legal text; Performing quality screening on the candidate concept set, and generating a judicial concept set based on legal concepts and / or legal terms with quality scores greater than a threshold; Performing synonym mining on the legal concepts and / or legal terms in the judicial concept set to obtain a judicial synonymous concept set; Entity relationship extraction is performed on the judicial synonymous concept set to obtain the judicial concept knowledge ontology, wherein the judicial concept knowledge ontology is obtained through entity relationship completion and / or error correction.

[0007] In a possible implementation, the step of extracting legal concepts and / or legal terms from a legal text includes: Performing Chinese word segmentation and part-of-speech tagging on a target text in the legal text, and determining the weights of keywords in the target text, wherein the target text is any legal text; Extract fragmented keywords and fuse adjacent fragmented keywords to obtain key phrases; The topic probability distribution of the target text and the topic probability distribution of each key phrase are calculated to obtain a target weight, and the target weight is used to determine the legal concepts and / or legal terms extracted based on the target text.

[0008] In a possible implementation, adjacent fragmented keywords are fused to obtain key phrases, including: Merge adjacent fragmented keywords and recalculate the weight of the merged phrases; A similar phrase filtering operation is performed on the fused phrases, and the filtered phrases are determined as the key phrases.

[0009] In a possible implementation, the step of performing a similar phrase filtering operation on the fused phrases includes: Calculating the relevance score between the candidate phrase and the selected phrase and the similarity score between other candidate phrases, wherein the candidate phrase is the phrase in the fused phrase except the selected phrase, and the selected phrase is the phrase selected as the key phrase; determining a target score for the candidate phrase based on the relevance score and the similarity score; The key phrase is selected based on the target score of the candidate phrase.

[0010] In a possible implementation, determining the target score of the candidate phrase based on the relevance score and the similarity score includes: The target score of the candidate phrase is determined according to the weighted value of the similarity score minus the relevance score, wherein the weighted values ​​of the relevance score and the similarity score are determined based on balancing relevance and diversity requirements.

[0011] In a possible implementation, the weight of a keyword in the target text is directly proportional to the word frequency of the keyword in the target text, and inversely proportional to the word frequency of the keyword in all legal texts.

[0012] In a possible implementation, based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained, including: Obtain a pre-trained model and determine a multi-label classifier to construct a judicial concept extraction model to be trained, wherein the multi-label elements are determined based on the judicial concept knowledge ontology; A training data set is obtained, and the judicial concept extraction model to be trained is trained to obtain a trained judicial concept extraction model.

[0013] In a possible implementation, during the training process, the parameters of the multi-label classifier are frozen.

[0014] According to one aspect of the present disclosure, a judicial data-driven descriptive judicial knowledge generation device is provided, the device comprising: a construction module for extracting legal concepts and / or legal terms from legal texts, and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms; A generation module is used to construct and train a judicial concept extraction model based on the judicial concept knowledge ontology, wherein the judicial concept extraction model is used to generate descriptive judicial knowledge, wherein the descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

[0015] According to the embodiments of the present disclosure, descriptive judicial knowledge can be accurately generated.

[0016] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0018] Figure 1 A schematic diagram showing the design and classification of a diversified knowledge system according to an embodiment of the present disclosure is shown.

[0019] Figure 2 A flowchart of a method for generating descriptive judicial knowledge driven by judicial data according to an embodiment of the present disclosure is shown.

[0020] Figure 3 A schematic diagram of multi-level descriptive judicial knowledge extraction based on a pre-trained large model according to an embodiment of the present disclosure is shown.

[0021] Figure 4 A schematic diagram showing the technical mechanism of knowledge ontology construction according to an embodiment of the present disclosure.

[0022] Figure 5 A schematic diagram showing a candidate concept set according to an embodiment of the present disclosure.

[0023] Figure 6 A schematic diagram showing a descriptive judicial knowledge generation mechanism according to an embodiment of the present disclosure.

[0024] Figure 7 An apparatus for a method of generating descriptive judicial knowledge driven by judicial data according to an embodiment of the present disclosure is shown.

[0025] Figure 8 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0027] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0028] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0029] Figure 1 FIG. 1 is a schematic diagram showing the design and classification of a diversified knowledge system according to an embodiment of the present disclosure. Figure 1 As shown, four types of judicial knowledge forms are specifically shown. Based on the investigation and summary of judicial business with trial as the core, this application focuses on the writing format of judicial data and the key elements of concern in actual business work, and summarizes and classifies a unified judicial knowledge system of four different forms of knowledge: descriptive, rule-based, relational, and regular. At the same time, based on different types of judicial texts, combined with the composition characteristics of the judicial model and the data format, different knowledge extraction frameworks are designed. In response to the knowledge redundancy, knowledge conflict, and unclear direction in the judicial knowledge extraction results, the problems of knowledge redundancy and conflict are solved by aligning the entity knowledge.

[0030] Among them, descriptive judicial knowledge includes judicial concept terms, judicial subject descriptions, judicial case descriptions, electronic files, and judicial information space descriptions.

[0031] 1) Judicial concept terminology Based on the case type and specific cause, the corresponding case characteristic elements, their legal definitions, and related case element naming entities, attributes, etc. are sorted out.

[0032] 2) Description of the judicial process The judicial process refers to the process by which judicial organs hear cases in accordance with the procedures prescribed by law, mainly including the litigation process, trial process, and the handling of irregularities in court trials.

[0033] Litigation process refers to the procedural requirements of the people's court in the process of accepting a case, including the requirements for litigation materials and the determination of whether the case falls under the jurisdiction of the court receiving the case. When a people's court receives the prosecution materials submitted by the plaintiff and finds that they meet the conditions for prosecution after review, it shall register the case. If it is found that the litigation materials are incomplete, the content and form of the complaint do not meet the regulations and need to be supplemented or modified, the case-filing judge shall inform the plaintiff once and issue a notice of supplementation and correction of litigation materials to the plaintiff. The people's court shall accept cases in accordance with the provisions of both hierarchical jurisdiction and territorial jurisdiction.

[0034] The trial process determines the applicable procedure for a case based on the case's cause, subject matter and other basic circumstances, and separates complex and simple cases. For cases that meet the applicable conditions for simplified procedures or fast-track procedures, simplified procedures or fast-track procedures will be applied.

[0035] Irregularities in court refer to behaviors that are not conducted in accordance with relevant legal norms during the litigation service process. They mainly include irregularities caused by outdated judicial concepts, irregularities caused by deviations in legal understanding, irregularities caused by poor control ability, and irregularities caused by lack of professionalism.

[0036] 3) Description of judicial subject It refers to people who are active in all stages of legal and judicial activities, who enjoy rights, bear obligations and responsibilities.

[0037] A law firm is a lawyer's practice organization. A law firm shall be established in accordance with the law and obtain a practice license. Within the prescribed scope of professional activities, a law firm shall accept the commission of Chinese and foreign parties and provide various legal services; it shall be responsible for the specific allocation and guidance of the business work of its lawyers; according to needs, with the approval of the Ministry of Justice, a professional law firm may be established, and a law firm with conditions may set up several business groups internally according to the principle of professional division of labor.

[0038] A lawyer is a professional who obtains a lawyer's license in accordance with the law, accepts commission or designation, and provides legal services to clients. Lawyers should safeguard the legitimate rights and interests of clients, safeguard the correct implementation of the law, and safeguard social fairness and justice.

[0039] 4) Description of judicial cases The civil case cause system, according to the "Provisions on the Causes of Civil Cases of the Supreme People's Court", includes: disputes over personal rights, disputes over marriage and family inheritance rights, property rights disputes, contract and quasi-contract disputes, intellectual property and competition disputes, labor disputes and personnel disputes, maritime and shipping disputes, civil disputes related to companies and securities, tort liability disputes, non-litigation case causes, and special litigation procedure case causes.

[0040] According to the Criminal Law of the People's Republic of China, the criminal offence system is an important part of criminal law. Criminal offences are specific names and legal characterizations of various criminal acts. China's criminal offence system is classified according to the nature of the crime and the social harm it causes. Under these classifications, there are specific offences. Each offence has corresponding legal provisions that make detailed provisions on its constituent elements, penalties, etc. The criminal offence system is an important legal basis for the state to punish crimes, protect social order and the legitimate rights and interests of citizens.

[0041] Litigation request refers to the substantive rights request made by the plaintiff against the defendant through the People's Court. Different types of lawsuits have different litigation requests. The litigation request of a payment lawsuit is to deliver a certain property or perform a certain behavior; the litigation request of a confirmation lawsuit is to determine a certain legal relationship; the litigation request of a change lawsuit is to change or eliminate a certain legal relationship. In a lawsuit, the request made by the defendant in the counterclaim and the request made by the third party in the lawsuit to the plaintiff and defendant in the original lawsuit are also litigation requests.

[0042] The characteristics of the case are summarized and extracted from the basic facts of the case, including disputed facts, evidence, causes of the case, etc.

[0043] 5) Electronic file description The electronic case file directory specification refers to the electronic documents and related electronic data in a specific format that are produced by using digital imaging technology, text recognition technology, database technology and other media technologies before or during the case acceptance process, using paper case files that are bound into volumes. The electronic case file directory specification refers to the construction of a standard directory structure for case electronic case files based on the court industry standard FYB / T 52021-2018 "Electronic Case File Reading Directory Specification" and "Technical Specification for Court Electronic Litigation File Directory". And sort out the mapping relationship between material types and directories.

[0044] As mentioned above, descriptive judicial knowledge includes various types of judicial knowledge. Judicial big data resources have the characteristics of complex content and large magnitude. How to accurately generate descriptive judicial knowledge is of great significance. To solve the above problems, this application provides a method for generating descriptive judicial knowledge driven by judicial data. Figure 2 A flowchart of a method for generating descriptive judicial knowledge driven by judicial data according to an embodiment of the present disclosure is shown. Figure 2 As shown, the method includes: In step S11, extract legal concepts and / or legal terms from the legal text, and construct a judicial concept knowledge ontology based on the legal concepts and / or legal terms; In step S12, based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained. The judicial concept extraction model is used to generate descriptive judicial knowledge. The descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

[0045] In this application, the generation of descriptive judicial knowledge includes two parts: the generation of the knowledge ontology framework and the extraction of descriptive judicial knowledge (for example, essential concepts and terms). Among them, for the generation of the knowledge ontology framework, a variety of natural language processing technologies can be used to extract legal concepts and / or legal terms from legal texts, and then combined with expert experience and legal knowledge guidance to form a judicial concept knowledge ontology; after constructing the judicial concept knowledge ontology, this application can use data-driven methods and technologies to optimize the classification task training based on the pre-trained model of the BERT deep neural network structure for the multi-label elements of the judicial concept knowledge ontology (for example, judicial trial elements), and construct a judicial concept extraction model. The judicial concept extraction model is used to generate descriptive judicial knowledge, which includes specific words or phrases with specific legal definitions and used as adjudication elements in judicial trials.

[0046] It should be understood that descriptive judicial knowledge in the judicial field is the relevant content that explains the essential connotation of judicial-related objective things, in order to assist information systems in understanding the essence of objective things. Descriptive judicial knowledge involves multi-level structured information, including concepts, entities (mentions), relationships, events, etc. To automatically generate descriptive judicial knowledge, it is first necessary to build a hierarchical judicial concept hierarchy system (judicial concept knowledge ontology) based on legal theory and judicial affairs. On the basis of constructing a semantically complete judicial concept knowledge ontology, the core legal elements (concepts or entities) are extracted by comprehensive use of automation technologies such as regular expressions, machine learning, and deep learning semantic features, which can accurately generate descriptive judicial knowledge.

[0047] It should be noted that since ordinary natural language processing or supervised learning technology requires the annotation of the structure and semantics of a large number of case documents, it is difficult to produce results in the short term. This application adopts the "pre-training-tuning" technical route, and fine-tunes the judicial data task based on the pre-training model (basic model) that has achieved significant performance in tasks such as entity recognition, text classification, and intelligent question answering. The basic model used in this application constructs an end-to-end neural network structure through encoder-decoder and cross-attention mechanism to achieve large-scale description knowledge extraction of unstructured judicial documents. At the same time, for semantically similar words or entities, this application also uses a representation learning-based method for semantic alignment. First, the knowledge representation learning model is used to represent entities and relationships as real-valued vectors, and then the concepts or entities to be aligned are calculated by distance to calculate their similarity, for example, JE model, MTransE model, ITransE model, KDCoE model, etc., to complete semantic alignment.

[0048] In this application, descriptive judicial knowledge is based on the core concepts in judicial trials, extracts relevant legal concept terms and their definitions or interpretations from laws and regulations, and mines synonyms or near-synonyms from massive case documents to construct a legal basic concept system. On this basis, the semantic feature labels of the case are further extracted to construct guiding case knowledge, providing judicial field knowledge for judicial personnel in courts and procuratorates to handle cases. At the same time, with the core legal concept system as the link, the structured semantic knowledge obtained from judicial documents and case law is integrated with emerging fields such as the semantic features of electronic files to form multimodal judicial knowledge.

[0049] For ease of understanding, the technologies used in the descriptive judicial knowledge generation method provided in this application are described exemplarily, which may specifically include: (1) N-gram technology N-Gram is an algorithm based on statistical language model, which is based on the assumption that the appearance of the Nth word in the text is only related to the previous N-1 words, and is not related to any other words. Therefore, the probability of a sentence is the product of the probability of each word appearing. These probabilities can be obtained by directly counting the number of times N words appear at the same time in the corpus. Commonly used are Bi-Gram and Tri-Gram.

[0050] In specific operations, N-gram technology performs a sliding window operation of size N on the content of the text according to bytes, forming a sequence of byte segments of length N. Each byte segment is called a gram, and the occurrence frequency of all grams is counted and filtered according to a pre-set threshold to form a key gram list, which is the vector feature space of the text. Each gram in the list is a feature vector dimension.

[0051] (2) Chunking Technology Chunking is a regular entity recognition technology that is used to segment and annotate sequences of multiple tokens. In practice, chunking divides sentences into chunks based on word-level segmentation and part-of-speech tagging. Chunking usually selects a subset of tokens. Like word segmentation, the segments generated by the chunker cannot overlap in the source text.

[0052] (3) Part-of-speech tagging technology based on statistics and rules Part-of-speech tagging is a text data processing technology in corpus linguistics that marks the part of speech of words in a corpus according to their meaning and context. It is also called grammatical annotation and word class resolution. Part-of-speech tagging is essentially a classification problem, that is, classifying words in a corpus according to their part of speech. The part of speech of a word is determined by its meaning, morphology, and grammatical function in the language to which it belongs. Parts of speech are not closed sets, but have the phenomenon of polyspeech, so part-of-speech tagging is related to the context.

[0053] There are many studies on part-of-speech tagging, and the common ones include rule-based part-of-speech tagging methods, statistical model-based part-of-speech tagging methods, and part-of-speech tagging methods based on a combination of statistics and rules. Among them, the part-of-speech tagging technology based on a combination of statistics and rules calculates the probability of a word being tagged with all parts of speech to give a credibility to the statistical tagging results. All corpora are first statistically tagged, the statistical tagging results are screened, and then those statistical tagging results with a credibility less than the threshold are manually proofread and the rule method is used to resolve ambiguity.

[0054] (4) Topic Model In the field of natural language processing, a large amount of text data is unstructured, and it is difficult to directly obtain relevant and expected information from the information. The Topic Model can identify the topics in the document and mine the hidden information in the corpus. It is widely used in scenarios such as topic aggregation, information extraction from unstructured text, and feature selection. The Topic Model usually uses a set of weights to represent the contribution of each word to each topic. These weights can help determine which words in the text are more relevant to a specific topic.

[0055] Latent Dirichlet allocation (LDA) is a classic method for fitting topic models. It treats each document as a mixture of multiple topics, and each topic is a mixture of multiple words. LDA allows documents to overlap in content rather than being divided into discrete groups, which in some sense reflects the typical usage of natural language. Before using LDA, you need to determine the number of topics K to learn (usually you need to choose it through experiments or domain knowledge). LDA uses random initialization to set the parameters of the model, including two main parameter matrices: document-topic distribution and topic-word distribution. LDA gradually optimizes these parameter matrices through iterative training. In each iteration, it considers each word in the document and tries to adjust the model parameters to better explain the text data. For each word in each document, LDA calculates the conditional probability distribution for each topic based on the current parameter estimates. This represents the probability that each word belongs to each topic. Then, based on these conditional probability distributions, LDA reassigns a topic to the word. This is done according to a multinomial distribution, taking into account the probability weight of each topic. Subsequently, LDA will update the model parameters, including document-topic distribution and topic-word distribution, based on the reassigned topics. This process will be repeated many times until the model parameters converge. Once the LDA model is trained, the weight matrix of words to topics can be obtained. This matrix represents the degree of relevance of each word to each topic. Higher weights indicate stronger relevance.

[0056] (5) Word similarity calculation Similarity calculation is a method to measure the semantic proximity between two words or texts, which can help people understand and measure the semantic relationship in text data. Word similarity calculation technology first converts words or phrases into numerical word quantities, also known as word embeddings, so that computers can understand and process text data. These vectors capture the semantic information of words in a continuous numerical space, so they can better measure the similarity between words. The simplest word similarity calculation is cosine similarity, which measures the similarity of two word vectors by calculating the cosine value of the angle between them. If the angle between two word vectors in the vector space is small, the cosine similarity will be close to 1, indicating that they are very similar semantically. Similarity calculation can be used not only to compare the similarity between individual words, but also to measure the similarity between entire text paragraphs or sentences.

[0057] There are many ways to convert words or phrases into numerical vectors, among which the most famous and widely used is Word2Vec. Word2Vec uses a neural network to learn distributed representations of words through a large text corpus. In this representation, each word is mapped to a fixed-length vector whose position and direction reflect the semantic relationship between words. For example, semantically similar words are close in the vector space, while semantically unrelated words are far away. There are two main models for Word2Vec: Skip-gram and CBOW (Continuous Bag of Words). The Skip-gram model predicts the target word from its context words, while CBOW does the opposite. Both models can effectively generate word vectors and can be used for various NLP tasks such as text classification, text generation, semantic search, etc. In addition to Word2Vec, other word vector calculation methods include Global Vectors for Word Representation (GloVe), FastText, etc. GloVe attempts to capture global word co-occurrence statistics through matrix decomposition, while FastText uses subword-level representation to better handle complex word morphology and structure.

[0058] (6) Calculation of maximum marginal correlation Maximum Marginal Relevance (MMR) is an algorithm for text summarization and information retrieval. Its goal is to select the most informative content from a text collection while avoiding similar content to improve diversity and information richness. The MMR algorithm can be used to filter similar phrases to ensure that the selected phrases are significantly different in content. The main idea is to maximize the relevance of the next content to the selected content while minimizing its similarity to the selected content.

[0059] (7) BERT model The technical route adopted by this application for judicial concept extraction is "pre-training-tuning". Among them, the pre-training model can use the classic Bidirectional Encoder Representation from Transformers (BERT) model.

[0060] The BERT model uses a multi-layer bidirectional Transformer architecture as an input encoder and an output decoder. It is an important deep learning framework for processing sequence to sequence (Seq2Seq) problems in the field of artificial intelligence. The encoder encodes the input symbol-based sequence Xj={x1,x2,…,xM} to obtain the input semantic encoding matrix C. The decoder then decodes C and combines the attention mechanism to finally output the sequence representation result Yj={y1,y2,…,yM} based on the embedding vector. At the same time, the encoder has a multi-head self-attention mechanism module and a fully connected feedforward network to achieve parallel calculation of multiple attention information in the sentence. Finally, the different attention information is spliced ​​together to obtain the hidden layer representation hi(i=1,2,…,M) of the input sequence (such as the word sequence that makes up the sentence), so that each token in the sentence can be encoded by integrating context information. Finally, the encoded input sequence features are restricted through the Conditional Random Field (CRF) layer to reduce unreasonable classifications and improve the accuracy of the model.

[0061] Figure 3 FIG. 1 is a schematic diagram showing a multi-level descriptive judicial knowledge extraction based on a pre-trained large model according to an embodiment of the present disclosure. Figure 3 As shown in the figure, when using the BERT model for text classification tasks, the input text is first processed by word segmentation and mapped to index values ​​using a dictionary as the input of the model. Next, the input text sequence is sent to the BERT layer, and the word vector encoding is obtained using the BERT model. These encodings incorporate the context information of the entire sentence. The sentence vector extracted from the BERT model is input as a feature into the subsequent classification head. After randomly inactivating some features through Dropout, the sentence vector is sent to multiple classification heads. Each classification head specifically learns the relationship between the features and the multi-classification parameters of the problem domain it is responsible for. The output of each classification head is processed by the Softmax activation function to obtain the classification probability distribution. As for the loss function, since the model is a multi-label model with multiple outputs, multiple loss functions are required to calculate the loss. For each classification head, the cross entropy loss function is used to calculate the loss, and the losses of each classification head are added. In order to reduce overfitting, the L2 regularization term of the model is also added. Through such a model structure and loss function design, multi-label text classification tasks can be predicted more accurately, and the generalization ability and applicability of the model can be improved.

[0062] Among them, legal concepts and / or legal terms are extracted from legal texts, and based on the legal concepts and / or legal terms, a judicial concept knowledge ontology is constructed. The legal concepts and legal terms can be directly extracted and determined as the judicial concept knowledge ontology, or the extracted legal concepts and legal terms can be processed to obtain the knowledge ontology. This application does not impose any restrictions on this.

[0063] In a possible implementation, extracting legal concepts and / or legal terms from legal texts and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms may include: generating a candidate concept set according to the legal concepts and / or legal terms extracted from the legal text; Performing quality screening on the candidate concept set, and generating a judicial concept set based on legal concepts and / or legal terms with quality scores greater than a threshold; Performing synonym mining on the legal concepts and / or legal terms in the judicial concept set to obtain a judicial synonymous concept set; Entity relationship extraction is performed on the judicial synonymous concept set to obtain the judicial concept knowledge ontology, wherein the judicial concept knowledge ontology is obtained through entity relationship completion and / or error correction.

[0064] In this way, a higher-quality judicial concept knowledge ontology can be obtained, thereby improving the accuracy of the generation of descriptive judicial knowledge in this application.

[0065] Figure 4 A schematic diagram showing the mechanism of knowledge ontology construction technology according to an embodiment of the present disclosure is shown. Figure 4 As shown, a candidate concept set can be generated and the quality of the candidate concept set can be screened. Based on the legal concepts and / or legal terms with quality scores greater than a threshold, a judicial concept set can be generated. Synonym mining can be performed on the legal concepts and / or legal terms in the judicial concept set to obtain a judicial synonymous concept set. Entity relationship extraction can be performed on the judicial synonymous concept set to obtain the judicial concept knowledge ontology. The final judicial concept knowledge ontology is obtained through cyclic entity relationship completion and / or error correction.

[0066] In a possible implementation, the step of extracting legal concepts and / or legal terms from a legal text may include: Performing Chinese word segmentation and part-of-speech tagging on a target text in the legal text, and determining the weights of keywords in the target text, wherein the target text is any legal text; Extract fragmented keywords and fuse adjacent fragmented keywords to obtain key phrases; The topic probability distribution of the target text and the topic probability distribution of each key phrase are calculated to obtain a target weight, and the target weight is used to determine the legal concepts and / or legal terms extracted based on the target text.

[0067] In this way, the element-type semantics reflected by the target text can be identified based on the understanding of its semantics.

[0068] In a possible implementation, the weight of a keyword in the target text is directly proportional to the word frequency of the keyword in the target text, and inversely proportional to the word frequency of the keyword in all legal texts.

[0069] In this way, for a keyword, the higher its frequency in the target text and the lower its frequency in other legal texts, it means that it has a more obvious category and can be well distinguished from the categories of other keywords, thereby improving the category distinction ability of the extracted keywords.

[0070] In a possible implementation, adjacent fragmented keywords are fused to obtain key phrases, including: Merge adjacent fragmented keywords and recalculate the weight of the merged phrases; A similar phrase filtering operation is performed on the fused phrases, and the filtered phrases are determined as the key phrases.

[0071] In this way, repeated key phrases can be reduced, thereby lowering the repetition rate of the extracted legal concepts and / or legal terms.

[0072] In a possible implementation, the step of performing a similar phrase filtering operation on the fused phrases includes: Calculating the relevance score between the candidate phrase and the selected phrase and the similarity score between other candidate phrases, wherein the candidate phrase is the phrase in the fused phrase except the selected phrase, and the selected phrase is the phrase selected as the key phrase; determining a target score for the candidate phrase based on the relevance score and the similarity score; The key phrase is selected based on the target score of the candidate phrase.

[0073] In this way, similar phrases can be accurately filtered out, thereby reducing the duplication rate of the extracted legal concepts and / or legal terms.

[0074] In a possible implementation, determining the target score of the candidate phrase based on the relevance score and the similarity score includes: The target score of the candidate phrase is determined according to the weighted value of the similarity score minus the relevance score, wherein the weighted values ​​of the relevance score and the similarity score are determined based on balancing relevance and diversity requirements.

[0075] This way, you can ensure that the key phrases you choose are clearly different in content from other key phrases.

[0076] The judicial concept extraction task studied in this application is essentially different from the relational word extraction task in normal natural language processing. In the keyword extraction task, only a few fragmentary words are extracted using technologies such as tf-idf and textrank. Such fragmentary words cannot truly express the contextual semantics of the text, let alone be called "knowledge". For this reason, this application defines "concepts" as descriptive judicial knowledge as specific words or phrases that have specific legal or judicial definitions and serve as elements of adjudication in judicial trials. At the same time, legal concepts usually have generalized superordinate semantic characteristics. Therefore, unlike the keyword extraction or named entity recognition tasks in natural language processing, extracting legal concepts from judicial documents is not simply extracting specific keywords or entities from the text, but requires identifying the element-type semantics reflected by them on the basis of understanding the semantics of the text.

[0077] Exemplarily, the target text in the legal text can be segmented and tagged with Chinese words and the weight of the keywords in the target text can be determined. For example, the pkuseg tool can be used to segment and tag the target text (judicial document) to be processed, and then the weight of the keywords in the target text can be calculated using tf-idf. Fragmented keywords can be extracted, and adjacent fragmented keywords can be fused to obtain key phrases. For example, adjacent fragmented keywords can be fused and the weight of the fused phrases can be recalculated; similar phrases can be filtered on the fused phrases, and the filtered phrases can be determined as the key phrases. For example, Pycharm can be used to open the Chinese_keyphrase_extractor module of the project to extract fragmented keywords, and then they can be fused according to adjacent fragmented keywords, and the weights can be recalculated to remove similar words. The fused multiple key fragments (phrases) obtained are key phrases.

[0078] The topic probability distribution of the target text and the topic probability distribution of each key phrase are calculated to obtain a target weight, which is used to determine the legal concepts and / or legal terms extracted based on the target text. For example, a pre-trained LDA model can be used to calculate the topic probability distribution of the text and the topic probability distribution of each candidate phrase to obtain the final target weight.

[0079] For example, in the process of extracting keywords using chinese_keyphrase_extractor, the present application configures the parameters of chinese_keyphrase_extractor as follows: param text: utf-8 encoded Chinese text param top_k: how many key phrases to select and return, the default is 5, if it is -1, all phrases will be returned param with_weight: specifies whether the returned key phrases need phrase weights param func_word_num: the number of function words allowed in a phrase, invalid when strict_pos is True param stop_word_num: the number of stop words allowed in a phrase, invalid when strict_pos is True param max_phrase_len: the maximum length allowed for a phrase, defaults to 25 characters param topic_theta: weight adjustment factor of topic weight, default 0.5, range (0~infinity) param stricted_pos: (bool) If True, only noun phrases are allowed param allow_pos_weight: (bool) Consider the part-of-speech weight, that is, the beginning and end of phrases with certain part-of-speech combinations are more likely to become key phrases param allow_length_weight: (bool) Consider part-of-speech weight, that is, phrases with a token length of 2 to 5 tend to become key phrases param allow_topic_weight: (bool) Take topic prominence into account, which helps filter out phrases that are not related to the topic (such as dates, etc.) param without_person_name: (bool) determines whether to remove the person name in the phrase param without_location_name: (bool) determines whether to remove the place name in the phrase param remove_phrases_list: (list) Remove some unwanted phrases so that they do not appear in the final result param remove_words_list: (list) Remove some unwanted words so that the phrase containing the word does not appear in the final result param specified_words: (dict) Industry term: word frequency. If it is not empty, only phrases containing the word are returned. param bias: (int|float) If specified_words is specified, you can choose to define the weight increase value return: key phrases and their weights In some optional embodiments, the step of performing a filtering operation of similar phrases on the fused phrases may include: Calculate the relevance score between the candidate phrase and the selected phrase and the similarity score between other candidate phrases, wherein the candidate phrase is the phrase in the fused phrase except the selected phrase, and the selected phrase is the phrase selected as the key phrase. For example, a dynamic MMR algorithm can be used to filter similar phrases.

[0080] determining a target score for the candidate phrase based on the relevance score and the similarity score; The key phrase is selected based on the target score of the candidate phrase.

[0081] The target score of the candidate phrase is determined according to the weighted value of the similarity score minus the relevance score, wherein the weighted values ​​of the relevance score and the similarity score are determined based on balancing relevance and diversity requirements.

[0082] For example, the relevance score between the candidate content and the selected content and the similarity score between the candidate content can be calculated, and the MMR score can be calculated by subtracting the weighted combination of the relevance score and the similarity score, where the weight adjustment is used to balance the relevance and diversity requirements. The candidate content with the highest MMR score is selected to ensure that the selected content strikes a balance between information relevance and content diversity, improving the effectiveness of tasks such as text summarization, information retrieval, and recommendation systems.

[0083] Figure 5 A schematic diagram of a candidate concept set according to an embodiment of the present disclosure is shown. The judicial concepts in the candidate concept set need to be further checked and confirmed by experts, and finally form judicial concepts (benchmark categories) that can be further used in legal documents.

[0084] In a possible implementation, based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained, including: Obtain a pre-trained model and determine a multi-label classifier to construct a judicial concept extraction model to be trained, wherein the multi-label elements are determined based on the judicial concept knowledge ontology; A training data set is obtained, and the judicial concept extraction model to be trained is trained to obtain a trained judicial concept extraction model, wherein during the training process, the parameters of the multi-label classifier are frozen.

[0085] In this way, a judicial concept extraction model for generating descriptive knowledge can be trained.

[0086] The main steps of the model tuning process for this application are as follows: 1) Pre-trained model selection: In the "pre-training-tuning" pipeline technology route, you first need to select and load the pre-trained BERT model. In this application, we use two widely used pre-trained models: Chinese_L-12_H-768_A-12 model and Chinese_lert_base_L-12_H-768_A-12_tf model. Among them, the model tuned based on the Chinese_L-12_H-768_A-12 pre-trained model is named Tf_12_110 model, and the model tuned based on the Chinese_lert_base_L-12_H-768_A-12_tf pre-trained model is named CL_12_102M_tf model.

[0087] 2) Classifier training: Based on the pre-trained model, a multi-label classifier is constructed. This application adds one or more fully connected layers and uses the sigmoid function in the last layer so that the output of each label is between [0,1].

[0088] 3) Data preparation: Convert the dataset into an input format acceptable to the model. Generally, the text needs to be segmented, truncated, and padded.

[0089] 4) Training model: Use labeled data to train the model, generally using a binary cross entropy loss function, and use the back propagation algorithm to optimize the model. Hyperparameters need to be tuned during the training process, including learning rate, batch size, number of iterations, etc.

[0090] 5) Model evaluation: Use test data to evaluate the trained model, usually using indicators such as F1 value, precision, and recall rate for evaluation.

[0091] 6) Model prediction: Use the trained model to predict new data, that is, perform multi-label classification.

[0092] When using a pre-trained model to tune and train a multi-label classification task, it is necessary to freeze some or all of the parameters of the pre-trained model to avoid destroying the original pre-trained weights during the training process. Common freezing methods include training only the classifier and freezing the parameters of the first few layers of the model. The choice of these methods depends on the specific task and data set. This application uses the frozen classifier training method for model tuning.

[0093] The definition of indicators for model evaluation can include: (1) F1 value In multi-label classification tasks, each sample may correspond to multiple labels, and the F1 value is a commonly used evaluation indicator to measure the performance of the model in multi-label classification tasks.

[0094] The F1 value is the weighted harmonic mean of precision and recall, and its calculation formula is: F1=2*(precision*recall) / (precision+recall) Among them, precision refers to the proportion of positive samples correctly predicted by the model to all predicted positive samples, while recall refers to the proportion of positive samples correctly predicted by the model to all true positive samples. For multi-label classification tasks, the calculation of these indicators needs to take into account multiple labels. When calculating the F1 value, it is necessary to calculate the precision and recall for each label separately, and take their weighted average as the final F1 value. Specifically, the true positives, false positives, and false negatives can be calculated for each label separately, and then the precision and recall are calculated, and finally the F1 value is calculated. The higher the F1 value, the better the performance of the model in multi-label classification tasks, and the more accurately multiple labels can be identified. Therefore, in multi-label classification tasks, the F1 value is usually an important evaluation indicator.

[0095] (2) Accuracy Precision is a commonly used evaluation metric to measure the accuracy of the labels predicted by the model in multi-label classification tasks. Precision refers to the proportion of positive samples correctly predicted by the model to all predicted positive samples. Specifically, for each label, the number of samples predicted by the model as the label and the label actually exists can be regarded as true positives (TP), and the number of samples predicted by the model as the label but the label does not actually exist can be regarded as false positives (FP). The precision of the label is TP / (TP+FP). In multi-label classification tasks, the precision needs to be calculated for each label separately, and the average precision of all labels is taken as the final precision. The higher the precision, the more accurate the label predicted by the model and the better the performance of the model.

[0096] (3) Recall rate Recall is a commonly used evaluation metric used to measure the coverage of labels predicted by the model in multi-label classification tasks. Recall refers to the proportion of positive samples correctly predicted by the model to all true positive samples. Specifically, for each label, the number of samples that the model predicts as the label and the label actually exists can be taken as true positives (TP), and the number of samples that the model fails to predict but actually exists can be taken as false negatives (FN). The recall of the label is TP / (TP+FN). In multi-label classification tasks, the recall needs to be calculated for each label separately, and the average of the recall rates of all labels is taken as the final recall rate. The higher the recall rate, the better the model can cover the actual existing labels and the better the performance of the model.

[0097] (4) Accuracy Accuracy is a commonly used evaluation metric to measure the overall accuracy of a model's predictions in multi-label classification tasks. Accuracy refers to the ratio of the number of labels correctly predicted by the model to the number of all labels. Specifically, for each sample, the number of labels predicted by the model as correct can be taken as true positives (TP), the number of labels predicted by the model as incorrect can be taken as false positives (FP), and the number of labels that the model failed to predict can be taken as false negatives (FN). The accuracy is (TP / (TP+FP+FN)). In multi-label classification tasks, the accuracy needs to be calculated for all samples, and the average of the accuracy of all samples is taken as the final accuracy. The higher the accuracy, the better the overall accuracy of the model's predictions and the better the performance of the model.

[0098] The training and testing data sets of this application may include: for training and testing the automatic extraction technology of descriptive judicial knowledge concepts, for the relevant paragraphs in a given judicial document, systematically judging each sentence in the document, identifying the key case elements therein, and classifying them according to the case element system designed by domain experts. In order to illustrate the performance level of relevant models and technologies, this study used three recognized standardized data sets in the judicial field, including marriage and family, labor disputes, and loan contracts. Each piece of training data consists of a case description fragment of a legal document, in which each sentence is marked with a corresponding category label (the number of category labels corresponding to each sentence is indefinite) The following are some examples of extracting some data: [ { "sentence": "The facts ascertained by this court are as follows:", "labels": [] }, { "sentence": "1. Loan date: August 27, 2013", "labels": [] }, { "sentence": "2. Loan amount: 5,000,000 yuan", "labels": ["LN2"] }, { "sentence": "3. Agreed interest rate: floating annual interest rate, 30% higher than the six-month benchmark annual interest rate of 5.6% announced by the People's Bank of China, adjusted every six months, settled monthly, the interest settlement date is the 20th of each month, the penalty interest rate is 150% of the loan interest rate, and interest that cannot be paid on time during the loan period is calculated and compounded monthly according to the contract interest rate. After the loan is overdue, the contract penalty interest rate will be used to calculate and compound interest.", "labels": [] }, { "sentence": "This court believes that the written contract signed between the original lender and the defendants is the true intention of the parties, legal and valid, and has legal binding force on the parties.", "labels": [] }, { "sentence": "The lender has now assigned the debt to the plaintiff and has completed the announcement procedures. The plaintiff's claims for the principal, interest, penalty interest and compound interest of the loan interest have legal basis and this court supports them.", "labels": ["LN9","LN5"] } ] (1) Marriage and family dataset. Dataset size: 30523.

[0099] The labels corresponding to DV1-DV20 are: having children after marriage, raising children with limited capacity for conduct, having common property of the couple, paying alimony, dividing real estate, separating after marriage, filing for divorce for the second time, paying alimony on a monthly basis, granting divorce, having common debts of the couple, personal property before marriage, statutory divorce, failing to fulfill family obligations, having children out of wedlock, appropriate assistance, failing to fulfill divorce agreement, compensation for damages, separation for two years due to emotional discord, children living with the non-custodial person, and personal property after marriage.

[0100] (2) Labor dispute dataset. Dataset size: 26394.

[0101] The labels corresponding to LB1-LB20 are: termination of labor relations, payment of wages, payment of economic compensation, failure to pay full labor remuneration, existence of labor relations, failure to sign a labor contract, signing of a labor contract, payment of overtime wages, payment of double wage compensation for failure to sign a labor contract, payment of work-related injury compensation, failure to initiate labor arbitration, failure to pay compensation for illegal termination of labor relations, economic layoffs, failure to pay bonuses, illegal collection of property from workers, special types of work, payment of work-related death allowances | funeral allowances | pensions, employer's advance notice of termination, loss of legal person status, and existence of a mediation agreement.

[0102] (3) Loan contract dataset. Dataset size: 17824.

[0103] The labels corresponding to LN1-LN20 are: creditor transfers creditor's rights, loan amount is x million yuan, there is a loan certificate, the lender is a financial institution, loan repayment, loan from a company|unit|other organization, joint guarantee, reminder of repayment and interest, conclusion of a guarantee contract, written repayment commitment, guarantee contract invalid|revoked|cancelled, refusal to perform repayment, exemption of guarantor's guarantee liability, guarantor does not bear guarantee liability, pledger is a company, lender fails to provide loan according to the agreed date|amount, multiple people borrowing, debtor transfers debt, and the agreed interest rate is unclear.

[0104] The Divorce dataset test results are as follows: (1) Tf_12_110 model Operation results: INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.96,'2': 0.91,'3': 0.91,'4': 0.94,'5': 0.92,'6': 0.94,'7': 0.93,'8': 0.96,'9': 0.98,'10': 0.86,'11': 0.85,'12': 0.09,'13': 0.83,'14': 0.47,'15': 0.81,'16': 0.72,'17': 0.67,'18': 0.05,'19': 0.28,'20': 0.66} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.96,'2': 0.94,'3': 0.97,'4': 0.98,'5': 0.96,'6': 0.94,'7': 0.95,'8': 0.97,'9': 0.99,'10': 0.92,'11': 0.92,'12': 1.0,'13': 0.93,'14': 0.93,'15': 0.96,'16': 0.99,'17': 0.88,'18': 1.0,'19': 0.96,'20': 0.91} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.95,'2': 0.89,'3': 0.87,'4': 0.9,'5': 0.88,'6': 0.94,'7': 0.91,'8': 0.96,'9': 0.97,'10': 0.81,'11': 0.79,'12': 0.05,'13': 0.75,'14': 0.31,'15': 0.71,'16': 0.57,'17': 0.54,'18': 0.02,'19': 0.16,'20': 0.52} INFO:root:The total score is as follows: 0.8216224912820579 Table 1 shows the test index values ​​of the Tf_12_110 model, as follows: Table 1

[0105] (2) CL_12_102M_tf model Operation results: INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.97,'2': 0.93,'3': 0.95,'4': 0.95,'5': 0.93,'6': 0.94,'7': 0.94,'8': 0.96,'9': 0.98,'10': 0.93,'11': 0.85,'12': 0.74,'13': 0.86,'14': 0.63,'15': 0.89,'16': 0.88,'17': 0.77,'18': 0.08,'19': 0.39,'20': 0.76} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.96,'2': 0.93,'3': 0.96,'4': 0.97,'5': 0.95,'6': 0.92,'7': 0.93,'8': 0.97,'9': 0.98,'10': 0.93,'11': 0.9,'12': 0.62,'13': 0.89,'14': 0.9,'15': 0.94,'16': 0.95,'17': 0.8,'18': 1.0,'19': 0.93,'20': 0.91} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.97,'2': 0.93,'3': 0.93,'4': 0.92,'5': 0.91,'6': 0.96,'7': 0.94,'8': 0.96,'9': 0.97,'10': 0.93,'11': 0.81,'12': 0.9,'13': 0.83,'14': 0.49,'15': 0.84,'16': 0.82,'17': 0.75,'18': 0.04,'19': 0.25,'20': 0.65} INFO:root: The accuracy value of each category of the model is calculated as follows: {'1': 0.99,'2': 0.98,'3': 0.99,'4': 0.99,'5': 0.99,'6': 1.0,'7': 1.0,'8': 1.0,'9': 1.0,'10': 1.0,'11': 1.0,'12': 0.99,'13': 1.0,'14': 0.99,'15': 1.0,'16': 1.0,'17': 1.0,'18': 1.0,'19': 1.0,'20': 1.0} INFO:root:The total score is as follows: 0.8725342265460727 Table 2 shows the test index values ​​of the CL_12_102M_tf model, as follows: Table 2

[0106] The test results of the Labor dataset for this application are as follows: (1) Using the Tf_12_110 model INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.9,'2': 0.88,'3': 0.94,'4': 0.92,'5': 0.93,'6': 0.94,'7': 0.91,'8': 0.93,'9': 0.94,'10': 0.86,'11': 0.0,'12': 0.75,'13': 0.0,'14': 0.0,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.9,'2': 0.96,'3': 0.97,'4': 0.94,'5': 0.96,'6': 0.95,'7': 0.95,'8': 0.95,'9': 0.96,'10': 0.92,'11': 0.0,'12': 0.82,'13': 0.0,'14': 0.0,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.9,'2': 0.81,'3': 0.91,'4': 0.91,'5': 0.9,'6': 0.93,'7': 0.88,'8': 0.92,'9': 0.91,'10': 0.8,'11': 0.0,'12': 0.69,'13': 0.0,'14': 0.0,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root:The total score is as follows: 0.6998889756162792 Table 3 shows the test index values ​​of the Tf_12_110 model, as follows: Table 3

[0107] (2) Using the CL_12_102M_tf model Operation results: INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.92,'2': 0.9,'3': 0.94,'4': 0.93,'5': 0.94,'6': 0.95,'7': 0.93,'8': 0.94,'9': 0.94,'10': 0.91,'11': 0.0,'12': 0.82,'13': 0.62,'14': 0.11,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 0.63,'20': 0.09} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.89,'2': 0.96,'3': 0.96,'4': 0.94,'5': 0.96,'6': 0.96,'7': 0.95,'8': 0.94,'9': 0.95,'10': 0.93,'11': 0.0,'12': 0.9,'13': 1.0,'14': 1.0,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 1.0,'20': 1.0} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.95,'2': 0.85,'3': 0.93,'4': 0.92,'5': 0.92,'6': 0.95,'7': 0.92,'8': 0.94,'9': 0.93,'10': 0.9,'11': 0.0,'12': 0.75,'13': 0.44,'14': 0.06,'15': 0.0,'16': 0.0,'17': 0.0,'18': 0.0,'19': 0.46,'20': 0.05} INFO:root: The accuracy value of each category of the model is calculated as follows: {'1': 0.99,'2': 0.99,'3': 0.99,'4': 1.0,'5': 1.0,'6': 1.0,'7': 1.0,'8': 1.0,'9': 1.0,'10': 1.0,'11': 1.0,'12': 1.0,'13': 1.0,'14': 1.0,'15': 1.0,'16': 1.0,'17': 1.0,'18': 1.0,'19': 1.0,'20': 1.0} INFO:root:The total score is as follows:0.7500229829438864 Table 4 shows the test index values ​​of the CL_12_102M_tf model, as follows: Table 4

[0108] The test results of the Loan dataset of this application can be as follows: (1) Using the Tf_12_110 model INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.89,'2': 0.8,'3': 0.85,'4': 0.77,'5': 0.87,'6': 0.68,'7': 0.91,'8': 0.96,'9': 0.9,'10': 0.96,'11': 0.1,'12': 0.0,'13': 0.0,'14': 0.0,'15': 0.0,'16': 0.18,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.89,'2': 0.97,'3': 0.95,'4': 0.92,'5': 0.96,'6': 0.92,'7': 0.96,'8': 0.97,'9': 0.96,'10': 0.96,'11': 1.0,'12': 0.0,'13': 0.0,'14': 0.0,'15': 0.0,'16': 1.0,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.89,'2': 0.68,'3': 0.77,'4': 0.66,'5': 0.79,'6': 0.54,'7': 0.86,'8': 0.95,'9': 0.86,'10': 0.95,'11': 0.05,'12': 0.0,'13': 0.0,'14': 0.0,'15': 0.0,'16': 0.1,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root:The total score is as follows: 0.6482984815253561 Table 5 shows the test index values ​​of the Tf_12_110 model, as follows: Table 5

[0109] (2) Using the CL_12_102M_tf model Operation results: INFO:root: The f1 value of each category of the model is calculated as follows: {'1': 0.91,'2': 0.82,'3': 0.87,'4': 0.8,'5': 0.9,'6': 0.74,'7': 0.93,'8': 0.97,'9': 0.91,'10': 0.96,'11': 0.58,'12': 0.0,'13': 0.11,'14': 0.21,'15': 0.34,'16': 0.75,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root: The precision value of each category of the model is calculated as follows: {'1': 0.89,'2': 0.96,'3': 0.94,'4': 0.89,'5': 0.95,'6': 0.91,'7': 0.93,'8': 0.97,'9': 0.94,'10': 0.95,'11': 0.94,'12': 0.0,'13': 1.0,'14': 1.0,'15': 0.89,'16': 0.87,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO: root: The recall value of each category of the model is calculated as follows: {'1': 0.93,'2': 0.72,'3': 0.81,'4': 0.73,'5': 0.85,'6': 0.63,'7': 0.92,'8': 0.96,'9': 0.88,'10': 0.96,'11': 0.42,'12': 0.0,'13': 0.06,'14': 0.12,'15': 0.21,'16': 0.67,'17': 0.0,'18': 0.0,'19': 0.0,'20': 0.0} INFO:root: The accuracy value of each category of the model is calculated as follows: {'1': 0.98,'2': 0.98,'3': 0.98,'4': 0.97,'5': 0.99,'6': 0.97,'7': 0.99,'8': 1.0,'9': 0.99,'10': 1.0,'11': 1.0,'12': 1.0,'13': 1.0,'14': 1.0,'15': 1.0,'16': 1.0,'17': 1.0,'18': 1.0,'19': 1.0,'20': 1.0} INFO:root:The total score is as follows: 0.7078099146511865 Table 6 shows the test index values ​​of the CL_12_102M_tf model, as follows: Table 6

[0110] Table 7 shows the comparison of the indicators of the models under different test sets, as follows:

[0111] Figure 6 A schematic diagram showing a descriptive judicial knowledge generation mechanism according to an embodiment of the present disclosure is shown. Figure 6 As shown, a unified judicial knowledge base is generated through various means, and multi-label feature selection and learning as well as autoregressive time series law learning are performed to obtain a macro law mining model.

[0112] The research on the automatic extraction technology of descriptive judicial knowledge in this application comprehensively utilizes regular expressions, complete matching, machine learning semantic feature classifiers and other automatic learning core legal element extraction models. Based on the judicial knowledge concept system, after model construction, training, testing and comparison, the Divorce value of the Tf_12_110 model is 0.8216, the Labor value is 0.6999, and the Loan value is 0.6483; the Divorce value of the CL_12_102M_tf model is 0.8725, the Labor value is 0.7500, and the Loan value is 0.7078. Through the fine-tuning-encoding-decoding and cross-attention mechanism across scales and tasks, an end-to-end neural network structure is constructed, and multi-source decision-making, data clustering and other methods are used to achieve large-scale descriptive knowledge extraction of unstructured judicial documents.

[0113] Figure 7 The device of the method for generating descriptive judicial knowledge driven by judicial data according to an embodiment of the present disclosure is shown. Figure 7 As shown, the device comprises: A construction module 21 is used to extract legal concepts and / or legal terms from legal texts, and construct a judicial concept knowledge ontology based on the legal concepts and / or legal terms; The generation module 22 is used to construct and train a judicial concept extraction model based on the judicial concept knowledge ontology, and the judicial concept extraction model is used to generate descriptive judicial knowledge. The descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

[0114] It can be understood that the above-mentioned embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined by its function and possible internal logic.

[0115] Figure 8 1 shows a block diagram of an electronic device according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 8 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions that can be executed by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0116] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0117] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0118] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0119] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0121] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0122] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0125] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0126] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating descriptive judicial knowledge driven by judicial data, characterized in that: The method comprises: Extracting legal concepts and / or legal terms from legal texts, and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms; Based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained. The judicial concept extraction model is used to generate descriptive judicial knowledge. The descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

2. The method according to claim 1, characterized in that The extracting of legal concepts and / or legal terms from the legal text and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms include: generating a candidate concept set according to the legal concepts and / or legal terms extracted from the legal text; Performing quality screening on the candidate concept set, and generating a judicial concept set based on legal concepts and / or legal terms with quality scores greater than a threshold; Performing synonym mining on the legal concepts and / or legal terms in the judicial concept set to obtain a judicial synonymous concept set; Entity relationship extraction is performed on the judicial synonymous concept set to obtain the judicial concept knowledge ontology, wherein the judicial concept knowledge ontology is obtained through entity relationship completion and / or error correction.

3. The method according to claim 1 or 2, characterized in that: The steps of extracting legal concepts and / or legal terms from legal texts include: Performing Chinese word segmentation and part-of-speech tagging on a target text in the legal text, and determining the weights of keywords in the target text, wherein the target text is any legal text; Extract fragmented keywords and fuse adjacent fragmented keywords to obtain key phrases; The topic probability distribution of the target text and the topic probability distribution of each key phrase are calculated to obtain a target weight, and the target weight is used to determine the legal concepts and / or legal terms extracted based on the target text.

4. The method according to claim 3, characterized in that Adjacent fragmented keywords are fused to obtain key phrases, including: Merge adjacent fragmented keywords and recalculate the weight of the merged phrases; A similar phrase filtering operation is performed on the fused phrases, and the filtered phrases are determined as the key phrases.

5. The method according to claim 4, characterized in that The steps of performing a similar phrase filtering operation on the fused phrases include: Calculating the relevance score between the candidate phrase and the selected phrase and the similarity score between other candidate phrases, wherein the candidate phrase is the phrase in the fused phrase except the selected phrase, and the selected phrase is the phrase selected as the key phrase; determining a target score for the candidate phrase based on the relevance score and the similarity score; The key phrase is selected based on the target score of the candidate phrase.

6. The method according to claim 5, characterized in that The determining a target score for the candidate phrase based on the relevance score and the similarity score includes: The target score of the candidate phrase is determined according to the weighted value of the similarity score minus the relevance score, wherein the weighted values ​​of the relevance score and the similarity score are determined based on balancing relevance and diversity requirements.

7. The method according to claim 3, characterized in that The weight of a keyword in the target text is directly proportional to the word frequency of the keyword in the target text, and inversely proportional to the word frequency of the keyword in all legal texts.

8. The method according to claim 1, characterized in that Based on the judicial concept knowledge ontology, a judicial concept extraction model is constructed and trained, including: Obtain a pre-trained model and determine a multi-label classifier to construct a judicial concept extraction model to be trained, wherein the multi-label elements are determined based on the judicial concept knowledge ontology; A training data set is obtained, and the judicial concept extraction model to be trained is trained to obtain a trained judicial concept extraction model.

9. The method according to claim 8, characterized in that During the training process, the parameters of the multi-label classifier are frozen.

10. A judicial data-driven descriptive judicial knowledge generation device, characterized in that: The device comprises: A construction module, used for extracting legal concepts and / or legal terms from legal texts, and constructing a judicial concept knowledge ontology based on the legal concepts and / or legal terms; A generation module is used to construct and train a judicial concept extraction model based on the judicial concept knowledge ontology, wherein the judicial concept extraction model is used to generate descriptive judicial knowledge, wherein the descriptive judicial knowledge includes specific words or phrases that have specific legal definitions and serve as elements of judgment in judicial trials.

Images