Semantic extraction model and method for enhanced generation of case information retrieval

Abstract

The invention relates to a semantic extraction model and method for enhanced generation of case information retrieval, and the model comprises four parts: a case file data processing module, a semantic-based document pre-splitting module, a rule-based semantic segmentation post-processing module, and a document splitting result processing and storage module. According to a data form of a case file document, in combination with the advantages of a rule-based document partitioning method and a semantic analysis model SeqModel, understanding and effective segmentation of context semantic contents of the document are realized. According to the method, the model structure is simple, the universality is high, the advantages of a natural language processing semantic analysis model and a rule model are integrated, not only is the dependency relationship of context semantics considered, but also the format characteristics under the specific data format are concerned, the SeqModel model can be trained and the rule can be formulated according to the actually-applied document data field, and the method is suitable for being applied to the field of document data. And efficient splitting of large-scale documents is realized.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, machine learning, data feature extraction and analysis, and relates to a semantic extraction model and method for enhanced generation of case information retrieval. Background Art

[0002] In case information retrieval systems, semantic analysis methods are usually used to split entire paragraphs and understand the meaning of words, sentences and paragraphs. However, under the influence of multiple factors such as redundant context information, complex sentence structure, and diverse emotional tendencies, how to achieve effective semantic analysis is the key and difficulty.

[0003] In the field of Retrieval Augmented Generation (RAG) large model question and answer applications, document semantic analysis is also an important factor affecting the quality of question and answer. The RAG large model mainly includes two core processes: retrieval technology and large model prompts. Its model structure is as follows: Figure 1 As shown. In the information retrieval part, RAG generates a knowledge base to cover professional domain knowledge beyond the cognition of the big model, performs template matching and retrieval of related questions, and recalls information that is highly relevant to the question content, which is used as the background knowledge for the big model to complete the question-answering process. In the RAG big model system, how to efficiently divide text data into relatively independent text blocks with compact semantic relationships is a relatively critical task. Traditional rule-based data segmentation methods have problems such as missing semantics and scattered semantic content, which will directly affect the effects of subsequent functional modules such as knowledge base construction, the accuracy of recall results, and the mining of knowledge data relationships. How to make reasonable and effective divisions based on the existing semantic structure of the document, provide more background information support for subsequent retrieval and recall modules, and improve the accuracy of big model question-answering is a difficult problem in the current field.

[0004] In the knowledge base construction phase, the methods for structured segmentation and document segmentation can generally be divided into two categories: direct segmentation and semantic segmentation. Sentence segmentation is based on punctuation marks, which can generally retain the complete semantics of a sentence. This method has a coarse segmentation granularity and is prone to losing key information such as pronouns in the context. The fixed-length segmentation method sets the window length of the sliding window according to the input length limit of the Embedding model. This segmentation method may cause the sentence to be forced to be separated in part, resulting in a large loss of semantic information.

[0005] Semantic segmentation methods can be segmented based on the logical relationship of document semantics, so that large models can better understand the recalled content. Most of the existing semantic model-based text segmentation methods are based on the optimization and improvement of the BERT model. The Naive BERT method performs semantic analysis based on the BERT model and obtains the semantic correlation by predicting whether two sentences are continuous. Its disadvantages are that the model is large, training and prediction require a lot of computing resources, the input sequence length is fixed, and the processing ability for longer texts is weak; the Cross Segment Attention model predicts by focusing on the local context near the potential text segmentation point, but its focus on local context limits the model's ability to capture information from more distant paragraphs; the SeqModel model encodes multiple sentences and uses the encoder self-attention mechanism to obtain the context and dependencies between sentences, thereby improving the accuracy of segmentation. Its disadvantages are that the model structure is complex and training requires a lot of computing resources. For documents in a specific field such as case files, there may be problems such as structural confusion and symbol independent segmentation. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a semantic extraction model and a semantic extraction method for enhanced generation of case information retrieval. According to the data form of case file documents, the rule-based document segmentation method and the advantages of the semantic analysis model SeqModel are combined to realize the understanding and effective segmentation of the document context semantic content. The focus is on formulating reasonable and effective data processing rules according to the document format of the case file, the retrieval recall mechanism and the principle of large model question and answer, so as to improve the recall rate while reducing the additional overhead required for data training.

[0007] The technical solution of the present invention is as follows:

[0008] A case-oriented information retrieval enhancement generation model, characterized by comprising four parts: a case file data processing module, a semantic-based document pre-segmentation module, a rule-based semantic segmentation post-processing module, and a document segmentation result processing and storage module;

[0009] The case file data processing module contains basic information and materials related to the case, comprehensively records all details of the case, uniformly processes data from different sources into the same format, splits the key-value pairs of data labeled by title, content, time, etc. one by one, and simply integrates the content that jumps, has unclear logic, and abuses the paragraph format;

[0010] The semantic-based pre-splitting module, based on the SeqModel model commonly used in the field of natural language processing, imports batches of case file data into the trained SeqModel model, each sentence in the document is segmented by the WordPiece word segmenter, encoded by the input representation layer, the above word embedding is input into the Transformer encoder, the hidden state corresponding to the tag is output, and the sentence encoding is obtained by the mean pooling method, and finally the encoding of all sentences is input into the binary classifier to determine whether each sentence is a paragraph boundary;

[0011] The rule-based semantic segmentation post-processing module adds some splitting and merging rules based on the semantic pre-segmentation module, including the following four categories:

[0012] Symbol-based rules: merge text blocks based on symbols with enumeration meanings such as semicolons and colons;

[0013] Rules based on semantic content: merge text blocks based on incoherent numerical expressions;

[0014] Rules based on content summarization: Generate a brief description or summary of each case and add it to some text blocks as supplementary information to ensure the quality of retrieved information;

[0015] Rules based on list structure: merge text blocks based on numbered list structures;

[0016] The split result processing and storage module processes and stores the split results to ensure the accuracy and availability of the obtained data.

[0017] The semantic extraction method for enhancing the generation of a large model using the case-oriented information retrieval is characterized by comprising the following steps:

[0018] Step 1: Input case file document-related data and model-related basic parameters, initialize the model, and input the data into the data processing module;

[0019] Step 2: Perform unified processing of format and content in the data processing module. Split the data by key-value pairs labeled with titles, contents, and time, integrate the messy formats, and process the data into a format that is suitable for the input of the semantic processing model.

[0020] Step 3: Input the processed case file document data into the semantic segmentation model, and use the encoder self-attention mechanism to obtain the context and dependencies between sentences to achieve pre-segmentation of the document;

[0021] Step 4: Input the results of the semantic pre-splitting module into the rule-based post-processing model, and post-process the pre-splitting results from multiple dimensions such as symbols, semantic content, summary content, and list results to obtain the final document splitting results;

[0022] Step 5: Process and store the results of semantic splitting to facilitate connection with subsequent Embedding models.

[0023] The present invention has simplicity, versatility and high efficiency. The model structure of the method of the present invention is simple and highly versatile. The SeqModel model can be trained and rules can be formulated according to the document data field of actual application, and can be applied to document data in various fields. The method of the present invention combines the advantages of the natural language processing semantic analysis model and the rule model, taking into account the dependency relationship of contextual semantics and the format characteristics under the specific data format, and can achieve efficient splitting of large-scale documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a structural diagram of the large model generated by retrieval enhancement;

[0025] Figure 2 It is a structural diagram of a model module of the present invention;

[0026] Figure 3 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] like Figure 2 As shown, the present invention proposes a semantic extraction model for enhancing the generation of a large model for case information retrieval, which is used to realize a comprehensive question-and-answer function related to the content of case file documents. Its main structure is mainly divided into four parts: a case file data processing module, a semantic-based document pre-segmentation module, a rule-based semantic segmentation post-processing module, and a document segmentation result processing and storage module.

[0028] 1. Case file data processing module:

[0029] The present invention is mainly aimed at case file documents, whose contents include basic information and materials related to the case, comprehensively record all aspects of the case, and have strong narrative and contextual relevance. Case file documents mostly contain detailed information of the case, litigation process, trial results, etc., and contain a large number of pronouns and domain-specific terms. Retrieval enhancement and generation of large-model question and answer for such case file data have high requirements for the rationality of document segmentation.

[0030] The content of a case file document generally includes the case name, case details, judgment results, judgment basis, etc. Since case file data may have different data sources, data preprocessing is required before the data is uniformly input into the semantic segmentation module. Data from different sources are uniformly processed into the same format; key-value pairs of labeled data such as title, content, and time are split one by one; content that jumps, has unclear logic, and abuses paragraph format is simply integrated.

[0031] 2. Semantic-based pre-splitting module:

[0032] After obtaining the case file data in a unified format, it is necessary to establish a model that can correctly understand the semantic information so that the document can be reasonably split according to the contextual logical relationship. In the semantic-based pre-splitting module, the main body of the model is based on the SeqModel model commonly used in the field of natural language processing, which can achieve good results in semantic analysis tasks. SeqModel is a model for text segmentation, which predicts text segmentation points by encoding multiple sentences and modeling the dependencies between contexts. SeqModel models document segmentation as a sentence-level sequence labeling task. It uses BERT to encode multiple sentences simultaneously, models the dependencies between longer contexts, and then calculates the sentence vector, and finally predicts whether text segmentation is needed after each sentence.

[0033] In the semantic-based pre-splitting module, batches of case file data are imported into the trained SeqModel model. Each sentence in the document is segmented by the WordPiece tokenizer and encoded through the input representation layer. The above word embedding is input into the Transformer encoder, the hidden state corresponding to the tag is output, and the sentence encoding is obtained using the mean pooling method. Finally, the sentence encodings of all sentences are input into the binary classifier to determine whether each sentence is a paragraph boundary.

[0034] After being processed by the semantic-based pre-splitting module, batches of case file documents are pre-splited into text blocks based on semantic connection relationships.

[0035] 3. Rule-based post-processing module:

[0036] The SeqModel model can effectively analyze the dependency relationship between sentences based on sentence encoding and vector prediction. However, for patterned data in specific fields such as case files, the results of semantic segmentation have problems such as insensitivity to list segmentation structure and unclear reference to the subject of text blocks, which will affect the accuracy and completeness of the text segmentation results. Therefore, the rule-based post-processing module adds some splitting and merging rules based on the semantic-based pre-splitting module.

[0037] According to the rules, post-processing rules can be roughly divided into four categories:

[0038] (1) Symbol-based rules:

[0039] Case files contain a large number of descriptions of cases and judgment results, and they frequently use semicolons, colons, and other enumerative punctuation marks. The semantic segmentation model is sensitive to symbolic identifiers. The above-mentioned punctuation marks will increase the probability of text segmentation, resulting in the loss of text semantic content, thereby reducing the probability of recalling the text block. Therefore, post-processing is performed based on symbol setting rules to merge text blocks for symbols with enumerative meanings.

[0040] (2) Rules based on semantic content:

[0041] Case file documents contain numerical descriptions such as case time and the amount involved, and the expression of numerical content is frequently used. The semantic splitting model is sensitive to numerical identifiers. During the splitting process, the text may be split in the middle of the amount, time, etc., resulting in the loss of text content, thus affecting the quality and accuracy of the recalled text block content. Therefore, post-processing is performed based on semantic content setting rules to merge text blocks for incoherent numerical expressions.

[0042] (3) Rules based on content summarization:

[0043] Case files generally contain a large number of documents, and the case details and judgment results contain a wide range of written expressions, which are likely to contain more pronouns. In the text semantics-based splitting method, the model can only judge the dependency relationship based on the coherence of semantics, and cannot restore the specific meaning of the pronouns in the text block. In particular, some short text blocks contain less semantic information, and some texts with short lengths cannot be selected in the retrieval recall stage, resulting in information loss.

[0044] To address the above issues, a brief description or summary of each case can be generated and added to some text blocks as supplementary information to ensure the quality of retrieved information. Considering the integrity of text information and the limitation of vector length, the case name can be used as a form of brief description, or other forms can be selected according to specific circumstances. Therefore, post-processing is performed based on the document summary setting rules to supplement the content of the text block based on the case summary.

[0045] (4) Rules based on list structure:

[0046] Completely structured case files may split the document content into a numbered list structure. Judging from the semantic connection between the contexts, this type of list structure is relevant; from the availability of the recalled content, the text blocks split by number will affect the integrity of the long paragraphs, thereby affecting the integrity of the recalled content. Therefore, post-processing is performed based on the symbol setting rules to merge the text blocks for the numbered list structure.

[0047] 4. Split result processing and storage module:

[0048] This module needs to process and store the split results to ensure the accuracy and availability of the data. Because the method of the present invention is a document segmentation module that serves the retrieval enhancement generation model, the results of document segmentation need to be used as the input of the subsequent Embedding model for sentence embedding, so it is necessary to format the document segmentation results and split the document into text blocks by character to facilitate the connection with the subsequent Embedding model.

[0049] like Figure 3 As shown, the operation method of the semantic extraction model for enhancing the generation of a large model for case information retrieval includes the following steps:

[0050] Step 1: Input case file document-related data and model-related basic parameters, initialize the model, and input the data into the data processing module;

[0051] Step 2: Perform unified processing of format and content in the data processing module. Split the data by key-value pairs such as title, content, time, etc., integrate the messy formats, and process the data into a format that is suitable for the input of the semantic processing model.

[0052] Step 3: Input the processed case file document data into the semantic segmentation model, and use the encoder self-attention mechanism to obtain the context and dependencies between sentences to achieve pre-segmentation of the document;

[0053] Step 4: Input the results of the semantic pre-splitting module into the rule-based post-processing model, and post-process the pre-splitting results from multiple dimensions such as symbols, semantic content, summary content, and list results to obtain the final document splitting results;

[0054] Step 5: Process and store the results of semantic splitting to facilitate connection with subsequent Embedding models.

Claims

1. A semantic extraction model for enhanced generation of case information retrieval, characterized in that: It includes four parts: case file data processing module, semantic-based document pre-segmentation module, rule-based semantic segmentation post-processing module, and document segmentation result processing and storage module; The case file data processing module contains basic information and materials related to the case, comprehensively records all details of the case, uniformly processes data from different sources into the same format, splits the key-value pairs of data labeled by title, content, time, etc. one by one, and simply integrates the content that jumps, has unclear logic, and abuses the paragraph format; The semantic-based pre-splitting module, based on the SeqModel model commonly used in the field of natural language processing, imports batches of case file data into the trained SeqModel model, each sentence in the document is segmented by the WordPiece word segmenter, encoded by the input representation layer, the above word embedding is input into the Transformer encoder, the hidden state corresponding to the tag is output, and the sentence encoding is obtained by the mean pooling method, and finally the encoding of all sentences is input into the binary classifier to determine whether each sentence is a paragraph boundary; The rule-based semantic segmentation post-processing module adds some splitting and merging rules based on the semantic pre-segmentation module, including the following four categories: Symbol-based rules: merge text blocks based on symbols with enumeration meanings such as semicolons and colons; Rules based on semantic content: merge text blocks based on incoherent numerical expressions; Rules based on content summarization: Generate a brief description or summary of each case and add it to some text blocks as supplementary information to ensure the quality of retrieved information; Rules based on list structure: merge text blocks based on numbered list structures; The split result processing and storage module processes and stores the split results to ensure the accuracy and availability of the obtained data.

2. A semantic extraction method for the semantic extraction model for enhanced generation of case information retrieval as claimed in claim 1, characterized in that: The following steps are involved: Step 1: Input case file document-related data and model-related basic parameters, initialize the model, and input the data into the data processing module; Step 2: Perform unified processing of format and content in the data processing module. Split the data by key-value pairs labeled with titles, contents, and time, integrate the messy formats, and process the data into a format that is suitable for the input of the semantic processing model. Step 3: Input the processed case file document data into the semantic segmentation model, and use the encoder self-attention mechanism to obtain the context and dependencies between sentences to achieve pre-segmentation of the document; Step 4: Input the results of the semantic pre-splitting module into the rule-based post-processing model, and post-process the pre-splitting results from multiple dimensions such as symbols, semantic content, summary content, and list results to obtain the final document splitting results; Step 5: Process and store the results of semantic splitting to facilitate connection with subsequent Embedding models.

Images