A knowledge reorganization retrieval method based on unstructured documents

Abstract

The present invention discloses a knowledge reorganization retrieval method based on unstructured documents, which relates to the field of knowledge reorganization retrieval. The main scheme is: extracting text data for analysis, and obtaining the posterior probability and category probability of each relationship analysis text by performing in-depth analysis on the text data, so as to realize classified storage of all unstructured document data; converting retrieval prompt words into retrieval document data, and obtaining the posterior probability and category probability of the corresponding retrieval document data through data processing and analysis, and screening out unstructured document data that meets the requirements according to the posterior probability and category probability, thereby avoiding the problem that traditional unstructured document processing methods are difficult to efficiently convert a large amount of unstructured data into a structured form for analysis, entity recognition and relationship extraction are performed independently, and there is a lack of effective integration of the correlation between the two, which leads to incomplete and inaccurate information extraction, and significantly improves the accuracy and efficiency of text analysis and classification.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge reorganization retrieval, and in particular to a knowledge reorganization retrieval method based on unstructured documents. Background Art

[0002] In today's digital age, the processing and retrieval of unstructured document data has become a major challenge facing various industries. With the explosive growth of information, traditional text processing and retrieval methods can no longer meet the requirements of efficiency and accuracy.

[0003] Furthermore, existing search technologies often rely on keyword matching or simple semantic analysis when processing unstructured documents, making it difficult to capture the deep semantic relationships and knowledge structure within the documents. This results in insufficient accuracy and relevance in search results, failing to provide a satisfactory search experience for users. In the legal field in particular, the complexity and specialized nature of documents further complicate search, making it difficult for traditional search methods to quickly and accurately locate documents highly relevant to user needs.

[0004] Therefore, there is an urgent need for a technical solution that can effectively process unstructured documents, realize knowledge reorganization and improve retrieval efficiency and accuracy to meet the complex needs in practical applications. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a knowledge reorganization retrieval method based on unstructured documents, which at least solves the problems in the existing technology of high retrieval difficulty, inaccurate output results and low relevance.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a knowledge reorganization and retrieval method based on unstructured documents, characterized by comprising:

[0007] Step 1: Obtain unstructured document data, preprocess the unstructured document data, and extract text data for analysis to form structured text;

[0008] Step 2: Use entity recognition tools to identify each structured text and output entity information text annotated with entity types and vocabulary relationships;

[0009] Step 3: Analyze each entity information text using a relationship extraction tool, extract the entity types and relationship terms between different entity types in the entity information text, and form a relationship analysis text;

[0010] Step 4: Process the relationship analysis text through the BERT pre-trained model to obtain the embedding vector corresponding to each relationship analysis text;

[0011] Step 5: The embedded vector is input into the graph neural network for analysis, and the embedded representation vector is output;

[0012] Step 6: Analyze the embedded representation vector using the Naive Bayes model, output the posterior probability and category probability of each relational analysis text, and classify and store all unstructured document data;

[0013] Step 7: After entering the search prompt word in the legal field, the search prompt word forms the search document data. Through the data processing and analysis of steps 1 to 6, the posterior probability and category probability of the corresponding search document data are obtained. Based on the posterior probability and category probability, the unstructured document data that meets the requirements is screened out as the search result of this search.

[0014] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, text is extracted from each file in the unstructured document data by using OCR technology, and the text in the document and image data is converted into text data. The text data is then cleaned by a text editor or an automated cleaning script group to generate standardized text; a word segmentation tool is used to split the text in the standardized text into several words to generate structured text.

[0015] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, each entity information text is input into the relationship extraction model, the entity type and the relationship words between different entity types in each text are extracted, and the extracted entity types and corresponding relationship words constitute a relationship analysis text.

[0016] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, the relational analysis text is input into the BERT pre-trained model, the entity types and lexical relationships in the relational analysis text are mapped into the vocabulary of the model, corresponding to unique identifiers or indexes, to form an integer sequence T={t1,t2,…,tn}, the integer sequence T is input into the embedding layer of the BERT pre-trained model to obtain the initial embedding vector E={e1,e2,…,en}, the initial embedding vector E is input into the Transformer encoder of the BERT pre-trained model, and an attention score vector is output; the attention score vector is input into the feedforward neural network, and the attention conversion vector is obtained by performing a nonlinear transformation on the attention score vector; the attention score vector and the attention conversion vector are residually connected and layer normalized to obtain an embedding vector.

[0017] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, the method for obtaining the attention score vector is:

[0018] The initial embedding vector E is input into each Transformer encoder. For the input initial embedding vector E, three different linear transformations are performed to generate the query vector Q, key vector K and value vector V respectively. The three vectors have the same dimension, which is recorded as the key vector dimension dk. The formulas for calculating the query vector Q, key vector K and value vector V respectively using the initial embedding vector E are as follows:

[0019] ;

[0020] ;

[0021] ;

[0022] in, Q h It is h The query vector generated by the encoder; K h It is h The key vector generated by the encoder; V h It is h A vector of values ​​generated by the encoder; h is the serial number of the encoder; W Q h It is h The weight matrix of the query vector for each encoder; W K h It is h The weight matrix of the key vectors of the encoders; W V h It is h The weight matrix of the encoder value vector;

[0023] The attention score of each query vector with all key vectors is calculated by each encoder according to the following formula:

[0024] ;

[0025] in, ZYL h It is h The attention scores output by the encoder;

[0026] The output results of all encoders are concatenated and linearly transformed to obtain the attention score vector based on the formula:

[0027] ;

[0028] in, is the transformation weight matrix.

[0029] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, the attention conversion vector is calculated as follows:

[0030] The attention score vector is input into the feedforward neural network, and the attention conversion vector is obtained by performing a nonlinear transformation on the attention score vector. The formula is as follows:

[0031] ;

[0032] in, W1 is the conversion weight two, W2 is the conversion weight three, b1 is the bias parameter one, and b2 is the bias parameter two.

[0033] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, the formula for obtaining the embedding vector is as follows:

[0034] ;

[0035] In the preferred embodiment of the above-mentioned knowledge reorganization and retrieval method based on unstructured documents, the embedding representation vector is calculated based on the following formula:

[0036] ;

[0037] Where QR is the embedding representation vector; H i is the weight coefficient of the embedding vector, i It represents the sequence number of the feature in the embedding vector, ranging from 1 to n; n is a positive integer, and σ is the activation function.

[0038] In the preferred embodiment of the above-mentioned knowledge reorganization retrieval method based on unstructured documents, the calculation method of the posterior probability and the category probability is:

[0039] Calculate the likelihood probability of the embedded representation vector of all entity information texts under a given prior probability. Specifically, the likelihood probability can be decomposed into the probability product of each feature in the embedded representation vector. The formula is as follows:

[0040] ;

[0041] in, is the i-th feature in the embedding representation vector QR, and n is the number of features;

[0042] The posterior probability is calculated according to Bayes' theorem, and the formula is as follows:

[0043] ;

[0044] After traversing all posterior probabilities, the maximum posterior probability is selected as the category probability value based on the following formula:

[0045] ;

[0046] Select the category corresponding to the category probability value as the classification result, and then calculate all entity information texts according to the corresponding classification result C. QR Carry out classified storage to form a classified database.

[0047] In the preferred embodiment of the above-mentioned knowledge reorganization retrieval method based on unstructured documents, a corresponding classification database is selected according to the category probability of the retrieved document data, the posterior probability is traversed and compared with all posterior probabilities in the classification database, and a number of posterior probabilities whose similarity index meets the requirements are screened out;

[0048] The method for judging whether the similarity index meets the requirements is:

[0049] The similarity between the posterior probability corresponding to the retrieved document data and each posterior probability in the classification database is calculated. The unstructured document data corresponding to the posterior probability whose similarity exceeds the similarity threshold meets the requirements.

[0050] The present invention provides a knowledge reorganization retrieval method based on unstructured documents, which has the following beneficial effects:

[0051] (1) Obtain unstructured document data, pre-process the unstructured document data, and extract text data for analysis to form structured text. It can convert unstructured documents in various formats, such as PDF, images, etc., into a unified structured text format, thereby improving the availability of data and the efficiency of subsequent analysis, laying a solid foundation for subsequent entity recognition and relationship extraction, and avoiding the problem that traditional unstructured document processing methods are difficult to efficiently convert large amounts of unstructured data into a structured form for analysis, resulting in low data utilization and low analysis efficiency.

[0052] (2) By using entity recognition tools to identify entity information in structured texts and using relationship extraction tools to analyze the relationships between entities, combining entity recognition and relationship extraction can more comprehensively extract key information from texts. Entity recognition tools can accurately label entity types and lexical relationships, while relationship extraction tools further analyze the relationships between entities to form a complete relationship analysis text. This combination not only improves the accuracy of information extraction, but also better captures the semantic relationships in documents, providing rich information for subsequent knowledge graph construction and semantic understanding. It overcomes the problem in existing technologies where entity recognition and relationship extraction are performed independently and there is a lack of effective integration of the correlation between the two, resulting in incomplete and inaccurate information extraction.

[0053] (3) The BERT pre-trained model is used to obtain the embedding vector of the relational analysis text, which is then analyzed using a graph neural network. Finally, the naive Bayes model is used for classification. This allows for deep semantic encoding of the relational analysis text, obtaining high-quality embedding vectors, and capturing the semantic features of the text. The graph neural network further analyzes the embedding vectors and outputs embedding representation vectors, enhancing the understanding of complex relationships between entities. The naive Bayes model performs classification based on the embedding representation vectors, outputs posterior probabilities and category probabilities, and achieves efficient classification and storage of unstructured documents. This comprehensive application of deep learning models fully utilizes the advantages of each model, significantly improving the accuracy and efficiency of text analysis and classification, and avoiding the limitations of traditional text analysis methods in processing complex semantic relationships and large-scale data, which makes it difficult to meet the needs of deep semantic understanding and efficient classification.

[0054] (4) After inputting the search prompt word in the legal field, the posterior probability and category probability of the corresponding search document data are obtained through data processing and analysis, and the unstructured document data that meets the requirements are screened out based on these probabilities. By comprehensively applying the various advanced technologies in the above steps, it is possible to conduct in-depth semantic understanding and analysis of the search prompt word. After obtaining the posterior probability and category probability of the search document data, the unstructured document data that meets the user's needs is accurately screened out based on the set threshold or ranking strategy. This method not only improves the accuracy and relevance of the search results, but also can quickly respond to the user's search request, greatly improving the user experience and search efficiency; it avoids the problem of relying solely on keyword matching or simple semantic analysis when processing unstructured documents, which makes it difficult to accurately screen out documents that are highly relevant to user needs, resulting in insufficient accuracy and relevance of the search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 The figure is a schematic diagram of the steps of a knowledge reorganization and retrieval method based on unstructured documents of the present invention. DETAILED DESCRIPTION

[0056] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Example 1:

[0058] See also Figure 1 The present invention provides a knowledge reorganization and retrieval method based on unstructured documents, comprising:

[0059] Step 1: Obtain unstructured document data, preprocess the unstructured document data, and extract text data for analysis to form structured text.

[0060] Step 101: Obtain unstructured documents related to the legal field, including but not limited to laws and regulations, case judgments, contract texts, legal research reports, etc. For example, legal and regulatory data can be obtained from official government platforms such as the National People's Congress website, the China Government website, and the Ministry of Justice's legal and regulatory database, or commercial databases such as Peking University Law Treasure; case judgments can be obtained from the China Judgments Online, the China Trial Open Network, or open data platforms and APIs, academic and industry resources, such as law school libraries; contract texts and templates can be obtained from public resource platforms such as the SEC EDGAR database, the China Securities Regulatory Commission, or the Internet Open Contract Database; legal research reports and documents can be obtained from academic databases such as China National Knowledge Infrastructure or prescription acquisition platforms such as Google Scholar.

[0061] Step 102: Use OCR technology to extract text from each file in the unstructured document data, convert the text in the document and image data into editable text data, such as a text file (TXT) or an editable Word / PDF file, and then use a text editor or an automated cleaning script group to clean the text data, mainly including deleting garbled symbols, correcting line break errors, merging redundant line breaks in paragraphs, replacing OCR recognition errors, and deleting meaningless repeated paragraphs, etc., to generate standardized text.

[0062] In this step, text extraction from unstructured document data is performed using OCR technology, converting text from images or scans into editable text data, such as TXT files or editable Word / PDF files. This not only improves data processability but also facilitates subsequent text cleaning and analysis. For example, OCR technology can convert scanned legal documents, case judgments, and other documents into editable text, facilitating further processing and analysis. This addresses the problem of large amounts of text data in the form of images or scans within unstructured documents, which makes direct text extraction and analysis difficult using traditional methods. In the legal field, many historical documents and scans are stored as images, making direct text processing difficult. Using a text editor or automated cleaning script to clean text data can significantly improve the quality of text data, reduce the impact of noise on subsequent analysis, and improve processing efficiency and accuracy. This avoids issues such as garbled characters, incorrect line breaks, OCR errors, and repeated paragraphs that often plague text data extracted from unstructured documents, hindering subsequent text analysis and processing efficiency.

[0063] Step 103: Split the text in the standard text into several words through existing word segmentation tools such as the jieba library and the PKUSEG library. Specifically, after each word in the standard text is split, a corresponding processed structured text will be generated.

[0064] In this step, word segmentation tools are used to accurately break down the text in the standard text into several words, generating processed structured text. This not only provides a foundation for subsequent entity recognition and relationship extraction, but also better captures the specialized terminology and semantic information in legal texts, improving the accuracy and efficiency of analysis. This can address the problem that legal texts often contain a high concentration of specialized terminology and complex sentence structures, which traditional word segmentation tools may not be able to accurately handle, resulting in poor word segmentation results and impacting subsequent entity recognition and relationship extraction.

[0065] Step 2: Use entity recognition tools to identify each structured text and output entity information text annotated with entity types and vocabulary relationships;

[0066] Step 201: training an entity recognition NER model tool; specifically, selecting a BERT deep learning model, and before model training, adding vocabulary in the legal field to the model's vocabulary, and expanding its vocabulary to include more legal terms. The expanded vocabulary can include vocabulary in legal dictionaries and legal vocabulary and vocabulary relationships in the legal knowledge base, and pre-training is performed; then, legal text data with annotated entity types and vocabulary relationships, for example, in the sentence "Plaintiff Zhang San sued defendant Li Si for infringement", "Zhang San" is annotated as "plaintiff", "Li Si" is annotated as "defendant", and Zhang San and Li Si are in a "litigation" relationship, is input into the BERT deep learning model for NER task training, thereby obtaining a trained entity recognition NER model tool.

[0067] In this step, the BERT deep learning model was selected and its vocabulary was expanded before training, with legal-related vocabulary (such as terms from legal dictionaries and legal terms and their relationships from legal knowledge bases). This enabled the model to better understand and recognize specialized terminology and entity relationships in legal texts. Training with legal text data annotated with entity types and lexical relationships further improved the model's accuracy and generalization. For example, in the sentence "Plaintiff Zhang San sued defendant Li Si for infringement," the model accurately annotated "Zhang San" as "plaintiff" and "Li Si" as "defendant," and identified the "litigation" relationship between them. This training approach adapts entity recognition tools to the specific needs of the legal field, providing accurate entity information for subsequent relationship extraction and knowledge graph construction. This addresses the problem that existing entity recognition tools suffer from low recognition accuracy when dealing with specialized legal terminology and complex text, and are unable to accurately identify specialized entities in legal documents (such as parties to a case, court names, and legal provisions). Furthermore, the entity relationships in legal texts are complex and diverse, making them difficult for traditional entity recognition tools to capture.

[0068] It should be noted that entity types can be parties to a case, court names, legal provisions, and judgment results, for example; parties to a case include plaintiff and defendant; court name refers to the name of the court that made the judgment; legal provisions can be the legal provisions used as the basis for the adjudication; and judgment results can be the final judgment results, such as the amount of compensation being RMB XXX yuan. The relationship between entity types can be a "litigation" relationship between the plaintiff and the defendant, and a "reference" relationship between the judgment result and the legal provision, etc.

[0069] Step 202: Input the entity information texts of all unstructured document data into the entity recognition NER model tool, and the entity recognition NER model tool outputs the entity information text corresponding to each entity information text, which is annotated with the entity type and vocabulary relationship.

[0070] In this step, by applying the trained entity recognition (NER) model tool to all unstructured document data, each entity information text can be automatically and efficiently analyzed, and the entity information text annotated with entity types and lexical relationships can be output. This automated processing method greatly improves the efficiency and accuracy of information extraction, can handle large-scale legal document data, and provides rich entity information for subsequent knowledge reorganization and retrieval. For example, for a large number of case judgments, the model can quickly identify entities such as the parties to the case, court names, legal provisions, and judgment results, and annotate the relationships between them, such as "reference" relationships. This not only saves the time and cost of manual annotation, but also ensures the accuracy and completeness of entity information, providing reliable data support for subsequent analysis and retrieval; it avoids the problem that traditional methods have difficulty in efficiently extracting and annotating entity information when processing large-scale unstructured document data, resulting in incomplete and inaccurate information extraction, and cannot meet the needs of rapid analysis of large amounts of legal documents in practical applications.

[0071] Step 3: Analyze each entity information text through the relationship extraction tool, extract the entity types and relationship words between different entity types in the entity information text, and form a relationship analysis text.

[0072] Step 301: Train the relationship extraction model. First, extract relevant entity type vocabulary, relationship vocabulary, and concept vocabulary from the legal knowledge base and incorporate them into the model vocabulary to enhance the model's understanding and extraction capabilities of legal knowledge. This may include keywords, legal principles, legal procedures, and other vocabulary in legal texts. For example, add legal action words such as "prosecution," "judgment," and "compensation," legal entity words such as "plaintiff," "defendant," and "court," and relationship-representing vocabulary such as "basis," "result," and "involve." Use legal text data with relationship annotations, such as judgments, contracts, and legal consultation records, to cover various legal fields and case types. Perform preprocessing operations such as cleaning, word segmentation, and part-of-speech tagging on the annotated data to make it meet the model input requirements. The prepared dataset is divided into training, validation, and test sets in a ratio of 7:2:1. Input the data into a CNN convolutional neural network for training and testing to obtain a relationship extraction model.

[0073] In this step, relevant entity types, lexical relationships, and conceptual terms were extracted from the legal knowledge base and incorporated into the model vocabulary, enhancing the model's understanding and extraction capabilities of legal knowledge. Training was performed using legal text data with relationship annotations, covering a wide range of legal fields and case types, to ensure the model's generalization. Data preprocessing operations, such as cleaning, word segmentation, and part-of-speech tagging, ensured that the data met the model input requirements. The dataset was divided into training, validation, and test sets in a ratio of 7:2:1 to ensure the adequacy and reliability of model training. A convolutional neural network (CNN) was used for training and testing, resulting in a relational extraction model that accurately extracts entity relationships from legal text. For example, the model accurately identified the "according to" relationship in the sentence "according to Article 107 of the Contract Law" and the "sued" relationship in the sentence "plaintiff Zhang San sued defendant Li Si." This significantly improved the accuracy and efficiency of relational extraction and overcomes the inaccurate and incomplete extraction issues that existing relational extraction tools often face when dealing with complex legal relationships. The relationships in legal texts are diverse and complex, such as the "citation" relationship between legal provisions and judgment results, and the "litigation" relationship between plaintiffs and defendants. Traditional methods are difficult to effectively capture these relationships.

[0074] Step 302: Input each entity information text into the relationship extraction model, extract the entity type and the relationship words between different entity types in each text, and form a relationship analysis text with the extracted entity types and corresponding relationship words.

[0075] In this step, by applying the trained relationship extraction model to each entity information text, the entity types and the relationship words between them in the text can be automatically and efficiently extracted to form a complete relationship analysis text. This automated processing method greatly improves the efficiency of relationship extraction and can handle large-scale legal document data, providing rich relationship information for subsequent knowledge graph construction and semantic understanding. For example, for a judgment involving multiple parties in a case, the model can accurately extract the entity type of each party (such as plaintiff, defendant) and the relationship between them (such as litigation relationship), and integrate this information into a relationship analysis text, providing strong support for subsequent legal analysis and retrieval; it overcomes the difficulty of traditional methods in efficiently extracting entity types and the relationship words between them when processing large-scale legal documents, resulting in incomplete and inaccurate relationship analysis, which cannot meet the needs of in-depth analysis of legal documents in practical applications.

[0076] Step 4: Process the relationship analysis text through the BERT pre-training model to obtain the embedding vector corresponding to each relationship analysis text.

[0077] Step 401: Build a BERT pre-trained model including an embedding layer and a multi-layer Transformer encoder.

[0078] In this step, by building a BERT pre-trained model consisting of an embedding layer and a multi-layer Transformer encoder, we can perform deep semantic encoding on the relational analysis text, capturing the semantic features and contextual information within the text. This model structure effectively handles specialized legal terminology and complex sentence structures, improving our ability to understand and analyze legal texts.

[0079] Step 402: Special tags are added to the relational analysis text, such as the beginning tag ([CLS]), the ending tag ([SEP]), and the padding tag ([PAD]), so that the model can correctly identify the beginning and end of sentences and align texts of different lengths during batch processing. This can be achieved through string operations or list operations in programming languages, such as Python.

[0080] In this step, by adding special markers such as the start marker [CLS], end marker [SEP], and padding marker [PAD], the model can correctly identify the beginning and end of sentences and align texts of varying lengths during batch processing. This marking method ensures a uniform input format for the model, improving its processing efficiency and accuracy. It also avoids difficulties in correctly identifying the beginning and end of sentences when processing texts of varying lengths, as well as aligning texts of varying lengths during batch processing.

[0081] Step 403: Generate a custom vocabulary by extracting relevant words and concepts from the legal knowledge base through statistics. For example, use the Counter class in the collections library to count the word frequency, and then generate a custom vocabulary based on the word frequency sorting. The custom vocabulary is used as the vocabulary of the BERT pre-trained model. The relational analysis text with special tags added is input into the BERT pre-trained model. Then, the entity types and lexical relationships in the relational analysis text are mapped to the vocabulary of the model through the BertTokenizer in the transformers library. The corresponding unique identifiers or indexes form an integer sequence T={t1,t2,…,tn}, where tn represents the nth word and n is the maximum number of words.

[0082] In this step, by generating a custom vocabulary and using it as the vocabulary for the BERT pre-trained model, the model can better understand and process the specialized terminology and concepts in legal texts. For example, the Counter class in the collections library is used to count word frequencies, and then a custom vocabulary is generated based on word frequency sorting. This ensures that the model can accurately map the entity types and lexical relationships in the relational analysis text into the model's vocabulary. This avoids the problem that the vocabulary of existing pre-trained models may not include specialized legal terminology and conceptual vocabulary, resulting in the model's inability to accurately understand the semantics of legal texts. Furthermore, by converting the relational analysis text into integer sequences, the model can directly process this sequence data. This conversion method not only preserves the semantic information of the text but also enables the model to perform efficient batch processing.

[0083] Step 404: Input the integer sequence T into the embedding layer of the BERT pre-trained model. The embedding layer converts each word into a word embedding vector based on the input integer sequence T, such as using Word2Vec, and then adds a position embedding vector to each word embedding vector. The position embedding can be generated using a deterministic method such as a sine or cosine function, and then searched and added based on the position index of the word in the sentence; the word embedding vector and the position embedding vector are added element by element to obtain the initial embedding vector E={e1,e2,…,en}, where en represents the nth vector, which serves as the initial input of the Transformer model encoder.

[0084] In this step, each word is converted into a word embedding vector through the embedding layer, and the position embedding vector is added to generate an initial embedding vector. This embedding method not only captures the semantic information of the word, but also considers the position information of the word in the sentence, improving the model's ability to understand the text.

[0085] Step 405: Input the initial embedding vector E into each Transformer encoder layer. For the input initial embedding vector E, three different linear transformations are performed to generate the query vector Q, key vector K, and value vector V respectively. The three vectors have the same dimension, which is recorded as the key vector dimension dk. The formulas for calculating the query vector Q, key vector K, and value vector V respectively using the initial embedding vector E are as follows:

[0086] ;

[0087] ;

[0088] ;

[0089] in, Q h It is hThe query vector generated by the encoder; K h It is h The key vector generated by the encoder; V h It is h A vector of values ​​generated by the encoder; h is the serial number of the encoder; W Q h It is h The weight matrix of the query vector for each encoder; W K h It is h The weight matrix of the key vectors of the encoders; W V h It is h The weight matrix of the encoder value vector.

[0090] It should be noted that the initial value of the weight matrix is ​​random, and usually uses methods such as Xavier initialization or He initialization to set the range of the initial value according to the input and output dimensions of the weight matrix.

[0091] In this step, by generating query vectors, key vectors, and value vectors, the model can calculate attention scores between words and capture their relevance. This mechanism enables the model to focus on important information in the text, improving its understanding and analysis capabilities.

[0092] Step 406: Calculate the attention score of each query vector and all key vectors through each encoder according to the following formula:

[0093] ;

[0094] in, ZYL h It is h The attention score vector output by the encoder.

[0095] It's important to note that the attention score represents the correlation or match between the query vector and each key vector. This score reflects the influence of other words on the encoding of the current word. To prevent the value from being too large during exponential calculations, the dot product result is typically scaled by dividing it by the square root of the key vector's dimension.

[0096] In this step, the attention score reflects the correlation between the query vector and each key vector, helping the model determine the degree to which other words influenced the encoding of the current word. By scaling the dot product results, we avoid excessive exponential growth, improve the numerical stability of the model, and address the problem of how to quantify the correlation between words so that the model can better understand the semantic structure of the text.

[0097] Step 407: Concatenate the output results of all encoders and obtain the attention score vector through linear transformation. The formula is:

[0098] ;

[0099] in, It is to convert the weight matrix, usually using methods such as Xavier initialization or He initialization, and set the range of initial values ​​according to the input and output dimensions of the weight matrix to ensure that the network can stably propagate gradients at the beginning of training to avoid gradient disappearance or explosion problems.

[0100] In this step, by concatenating the outputs of all encoders and performing a linear transformation, the model generates a comprehensive attention score vector. This integration fully utilizes the information from multiple encoders, improving the model's expressiveness and accuracy.

[0101] Step 408: Input the attention score vector into the feedforward neural network and perform a nonlinear transformation on the attention score vector to obtain an attention conversion vector. The formula is as follows:

[0102] ;

[0103] in, W1 is the conversion weight two, W2 is the conversion weight three, b1 is the bias parameter one, and b2 is the bias parameter two.

[0104] It should be noted that Xavier initialization or He initialization is usually used to initialize conversion weights 2 and 3; bias parameters 1 and 2 are usually initialized to zero vectors, because the update of the bias term during training mainly depends on the data itself, and the initial value of zero will not have a negative impact on the training of the model.

[0105] By performing nonlinear transformation on the attention score vector through a feedforward neural network, the model can generate an attention transformation vector, further enhancing the ability to understand and analyze text semantics.

[0106] The attention score vector and the attention transformation vector are residually connected and layer normalized to obtain the embedding vector. The formula is as follows:

[0107] ;

[0108] In this step, residual connections and layer normalization help stabilize the model training process, accelerate convergence, and improve the model's generalization ability. By performing residual connections and layer normalization on the attention score vector and the attention transformation vector, the model can better capture the semantic information in the text, generate high-quality embedding vectors, accelerate convergence, and improve the model's generalization ability.

[0109] Step 5: The embedded vector is input into the graph neural network for analysis, and the embedded representation vector is output.

[0110] Specifically, the embedding vector EZ{EZ1, EZ2, …, EZn} is input into the graph neural network, and the embedding representation vector is output. The formula is as follows:

[0111] ;

[0112] Where QR is the embedding representation vector; H i is the weight coefficient of the embedding vector, i It represents the sequence number of the feature in the embedding vector, ranging from 1 to n; n is a positive integer, and σ is the activation function.

[0113] In this approach, by representing embedding vectors as a graph, relationships between entities can be intuitively displayed, such as the litigation relationship between parties involved in a case, or the reference relationship between legal provisions and judgments. This graph-structured representation not only enriches the form of knowledge representation but also provides a foundation for subsequent relationship analysis and knowledge reasoning. It overcomes the difficulty of traditional methods in effectively capturing the complex relationships between entities when processing unstructured documents in the legal field, resulting in incomplete and inaccurate knowledge representation. By using activation functions (such as ReLU) and introducing nonlinearity in the update process of node embedding representations, it can better capture the complex relationships between nodes. This nonlinear transformation improves the model's expressiveness and generalization capabilities, enabling it to more accurately represent the complex knowledge structure in the legal field.

[0114] Step 6: Analyze the embedded representation vector through the Naive Bayes model, output the posterior probability and classification results of each relationship analysis text, and classify and store all unstructured document data.

[0115] Step 601: Define a category variable C to represent the category of the document. In the legal document classification task, the category can be "contract dispute", "traffic accident liability dispute", "intellectual property dispute", etc., represented by C1, C2 and C3 respectively. There can be several categories, represented as Cn.

[0116] In this step, by defining the category variable C, the category of the document is clearly indicated, such as "contract dispute", "traffic accident liability dispute", "intellectual property dispute", etc., which provides a clear category identification for subsequent classification analysis and avoids the problem of inaccurate classification results caused by the lack of clear category definition and representation method in the legal document classification task.

[0117] Step 602: Calculate the prior probability for each category. This involves training the Naive Bayes model using document data with defined categories and calculating the frequency of occurrence of each category using the stable data. For example, if there are 100 documents in the document data, and 30 of them belong to the "Contract Dispute" category, then P(C1=Contract Dispute) = 0.3.

[0118] In this step, the Naive Bayes model is trained using document data with defined categories to calculate the prior probability for each category. For example, if there are 100 documents in the document data, and 30 of them belong to the "Contract Dispute" category, then P(C1=Contract Dispute) = 0.3. This data-based prior probability calculation method can more accurately reflect the distribution of categories in the data, providing reliable prior information for subsequent classification. It avoids the problem that existing classification methods often rely on simple frequency statistics when calculating prior probabilities and lack in-depth analysis of data distribution.

[0119] Step 603: Calculate the likelihood of the embedded representation vectors of all entity information texts under a given prior probability. Specifically, the likelihood can be decomposed into the product of the probabilities of each feature in the embedded representation vector, based on the following formula:

[0120] ;

[0121] in, is the i-th feature in the embedding representation vector QR, and n is the number of features.

[0122] By decomposing the likelihood probability into the product of the probabilities of each feature in the embedded representation vector, the contribution of each feature to the classification can be more accurately calculated. This decomposition method not only improves computational efficiency but also better captures the independence assumption between features, improving the classification accuracy of the model. It avoids the inaccuracy caused by traditional methods when calculating the likelihood probability, which is difficult to effectively decompose and calculate the probability of each feature in the embedded representation vector.

[0123] Step 604: Calculate the posterior probability according to Bayes' theorem, based on the following formula:

[0124] ;

[0125] Using Bayes' theorem, the prior probability and likelihood probability are combined to calculate the posterior probability for each category. This method comprehensively considers the influence of prior knowledge and observed data, improving the accuracy and reliability of classification results. Existing classification methods often ignore the combined influence of prior probability and likelihood probability when calculating posterior probability, resulting in inaccurate classification results.

[0126] Step 605: After traversing all posterior probabilities, the maximum posterior probability is selected as the class probability value, and the class corresponding to the class probability value is selected as the classification result. The formula is as follows:

[0127] ;

[0128] By traversing all posterior probabilities, the maximum value is selected as the category probability value, and the corresponding category is selected as the classification result. This decision rule based on the maximum posterior probability can ensure the optimality of the classification result and improve the classification performance of the model. It overcomes the problem of lacking an effective decision rule to determine the final classification result in multi-category classification tasks.

[0129] Step 606: All entity information texts are sorted according to the corresponding classification results C QR Carry out classified storage to form a classified database.

[0130] By categorizing and storing all entity information text according to the classification results, a structured, classified database is formed. This categorized storage method not only improves document management efficiency but also facilitates subsequent retrieval and analysis. For example, users can quickly locate required documents by category, greatly improving retrieval efficiency. This addresses the problem of existing methods lacking an efficient categorized storage mechanism when processing large-scale legal documents, resulting in chaotic document management and low retrieval efficiency.

[0131] Step 7: After entering the search prompt word in the legal field, the search prompt word forms the search document data. Through the data processing and analysis of steps 1 to 6, the posterior probability and classification results of the corresponding search document data are obtained. Based on the posterior probability and classification results, the unstructured document data that meets the requirements is screened out as the search result of this search.

[0132] Step 701: When inputting, search prompt words are entered in a legal language format, for example, "Plaintiff Zhang San sued defendant Li Si, unpaid wages, the amount is 20,000, the court is xx Court, search for similar cases". After entering the search prompt words in the legal field, the search prompt words form editable search document data.

[0133] In this step, we convert legal search prompts into editable search document data, ensuring that the prompts can be processed by subsequent models. For example, if a user enters "Plaintiff Zhang San sues defendant Li Si, unpaid wages, amounting to 20,000 yuan, from xx Court, searching for similar cases," the system converts this into structured search document data, providing a foundation for subsequent analysis and processing. This converts the user's natural language query into a format that the model can process, resulting in more accurate search results.

[0134] Step 702: Through data processing and analysis from steps 1 to 7, the posterior probability and category probability corresponding to the retrieved document data are obtained.

[0135] Existing retrieval methods fail to fully utilize the embedding vectors and classification information generated in the previous steps, resulting in insufficient relevance in the retrieval results. This solution ensures a comprehensive analysis of the retrieved document data through a complete data processing pipeline, from preprocessing, entity recognition, and relationship extraction to embedding vector generation and classification. This enables the system to accurately understand the semantics and classification of the search prompt terms, providing a precise basis for subsequent screening of similar documents.

[0136] Step 703: Select a corresponding classification database based on the category probability in step 702, compare the posterior probability corresponding to the retrieved document data with all posterior probabilities in this classification database, and select several posterior probabilities whose similarity index meets the requirements. The method for determining whether the similarity index meets the requirements is as follows:

[0137] The posterior probability corresponding to the retrieved document data is calculated with each posterior probability in the classification database using the well-known cosine similarity. The value range should be set in [0,1]. The similarity threshold standard can be set to 0.8, that is, the unstructured document data corresponding to the posterior probability with a similarity exceeding 0.8 meets the standard. The similarity threshold can be adjusted according to the number of documents that meet the standard after retrieval. If too many retrieved documents are output, the similarity threshold standard can be increased to 0.9.

[0138] In this step, by selecting the corresponding classification database and using cosine similarity to calculate the similarity between the search document and the documents in the database, the system can filter out documents that are highly relevant to the search prompt term. For example, if the similarity threshold is set to 0.8, only documents with a similarity exceeding 0.8 will be selected, ensuring the relevance and accuracy of the search results. Traditional methods lack effective similarity calculation and screening mechanisms when screening similar documents, resulting in inaccurate search results.

[0139] Step 704: The unstructured document data corresponding to the posterior probability that meets the requirements is used as the output result of this retrieval.

[0140] In this step, the system outputs unstructured document data corresponding to the required posterior probabilities, providing users with directly usable search results. These results not only accurately match the user's search requirements but are also presented in a user-friendly manner, improving user experience and search efficiency. Existing search systems, on the other hand, struggle to translate model calculations into understandable results, resulting in a poor user experience.

[0141] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0142] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0143] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A knowledge reorganization retrieval method based on unstructured documents, characterized in that: include: Step 1: Obtain unstructured document data, preprocess the unstructured document data, and extract text data for analysis to form structured text; Step 2: Use entity recognition tools to identify each structured text and output entity information text annotated with entity types and vocabulary relationships; Step 3: Analyze each entity information text using a relationship extraction tool, extract the entity types and relationship terms between different entity types in the entity information text, and form a relationship analysis text; Step 4: Process the relational analysis text through the BERT pre-trained model to obtain the embedding vector corresponding to each relational analysis text; specifically, input the relational analysis text into the BERT pre-trained model, map the entity types and lexical relationships in the relational analysis text into the vocabulary of the model, corresponding to unique identifiers or indices, to form an integer sequence T={t1,t2,…,tn}, input the integer sequence T into the embedding layer of the BERT pre-trained model to obtain the initial embedding vector E={e1,e2,…,en}, input the initial embedding vector E into the Transformer encoder of the BERT pre-trained model, and output the attention score vector; input the attention score vector into the feedforward neural network, and obtain the attention conversion vector by performing a nonlinear transformation on the attention score vector; perform residual connection and layer normalization on the attention score vector and the attention conversion vector to obtain the embedding vector; the calculation method of the attention conversion vector is: The attention score vector is input into the feedforward neural network, and the attention conversion vector is obtained by performing a nonlinear transformation on the attention score vector. The formula is as follows: ； in, W 1 is the conversion weight two, W 2 is the conversion weight three, b1 is the bias parameter one, and b2 is the bias parameter two; Step 5: The embedded vector is input into the graph neural network for analysis, and the embedded representation vector is output; Step 6: Analyze the embedded representation vector using the Naive Bayes model, output the posterior probability and category probability of each relational analysis text, and classify and store all unstructured document data; Step 7: After entering the search prompt word in the legal field, the search prompt word forms the search document data. Through the data processing and analysis of steps 1 to 6, the posterior probability and category probability of the corresponding search document data are obtained. Based on the posterior probability and category probability, the unstructured document data that meets the requirements is screened out as the search result of this search.

2. The knowledge reorganization and retrieval method based on unstructured documents according to claim 1 is characterized in that: In step one, text is extracted from each file in the unstructured document data using OCR technology, and the text in the document and image data is converted into text data. The text data is then cleaned using a text editor or an automated cleaning script group to generate standardized text. A word segmentation tool is used to split the text in the standardized text into several words to generate structured text.

3. The knowledge reorganization and retrieval method based on unstructured documents according to claim 2 is characterized in that: In step three, each entity information text is input into the relation extraction model, and the entity types and relation words between different entity types in each text are extracted. The extracted entity types and corresponding relation words are combined to form a relation analysis text.

4. The method for knowledge reorganization and retrieval based on unstructured documents according to claim 3, characterized in that: The method to obtain the attention score vector is: The initial embedding vector E is input into each Transformer encoder. For the input initial embedding vector E, three different linear transformations are performed to generate the query vector Q, key vector K and value vector V respectively. The three vectors have the same dimension, which is recorded as the key vector dimension dk. The formulas for calculating the query vector Q, key vector K and value vector V respectively using the initial embedding vector E are as follows: ； ； ； in, Q h It is h The query vector generated by the encoder; K h It is h The key vector generated by the encoder; V h It is h A vector of values ​​generated by the encoder; h is the serial number of the encoder; W Q h It is h The weight matrix of the query vector for each encoder; W K h It is h The weight matrix of the key vectors of the encoders; W V h It is h The weight matrix of the encoder value vector; The attention score of each query vector with all key vectors is calculated by each encoder according to the following formula: ； in, ZYL h It is h The attention scores output by the encoder; The output results of all encoders are concatenated and linearly transformed to obtain the attention score vector based on the formula: ； in, is the transformation weight matrix.

5. The method for knowledge reorganization and retrieval based on unstructured documents according to claim 4, characterized in that: The formula for obtaining the embedding vector is as follows: 。 6. The knowledge reorganization and retrieval method based on unstructured documents according to claim 5 is characterized in that: The formula for calculating the embedded representation vector is as follows: ； Where QR is the embedding representation vector; H i is the weight coefficient of the embedding vector, i It represents the sequence number of the feature in the embedding vector, ranging from 1 to n; n is a positive integer, and σ is the activation function.

7. The method for knowledge reorganization and retrieval based on unstructured documents according to claim 6, characterized in that: The posterior probability and class probability are calculated as: Calculate the likelihood probability of the embedded representation vector of all entity information texts under a given prior probability. Specifically, the likelihood probability can be decomposed into the probability product of each feature in the embedded representation vector. The formula is as follows: ； in, is the i-th feature in the embedding representation vector QR, and n is the number of features; The posterior probability is calculated according to Bayes' theorem, and the formula is as follows: ； After traversing all posterior probabilities, the maximum posterior probability is selected as the category probability value based on the following formula: ； Select the category corresponding to the category probability value as the classification result, and then calculate all entity information texts according to the corresponding classification result C. QR Carry out classified storage to form a classified database.

8. The knowledge reorganization and retrieval method based on unstructured documents according to claim 7 is characterized in that: In step 7, the corresponding classification database is selected according to the category probability of the retrieved document data, the posterior probability is compared with all the posterior probabilities in the classification database, and several posterior probabilities whose similarity index meets the requirements are screened out; The method for judging whether the similarity index meets the requirements is: The similarity between the posterior probability corresponding to the retrieved document data and each posterior probability in the classification database is calculated. The unstructured document data corresponding to the posterior probability whose similarity exceeds the similarity threshold meets the requirements.

Images