A deep learning-based literature retrieval method and system

Abstract

The application discloses a kind of based on deep learning's literature search method and system.The system includes: recall module, for the query text of user input, based on pre-stored word vector and text vector, generate multiple candidate texts;Reordering module, for the query text of user input and the multiple candidate texts are input to trained long text encoder, obtain query vector and candidate text vector, and calculate the similarity between query vector and candidate text vector, obtain sorted search result.The application effectively solves the problem that unsupervised model ranking effect is not ideal in text retrieval, and supervised text retrieval model cannot directly retrieve in a large number of documents, significantly improves retrieval speed and accuracy, especially suitable for the field involving long text retrieval long text.

Description

Technical Field

[0001] This invention relates to the field of literature retrieval technology, and more specifically, to a literature retrieval method and system based on deep learning. Background Technology

[0002] Literature retrieval is an information retrieval technique that retrieves documents based on learning and work needs. With the development of modern network technology, literature retrieval is typically accomplished using computer technology. Literature retrieval languages ​​can be categorized into classification languages, subject languages, etc. Using standardized literature retrieval languages ​​allows for quick and accurate retrieval of required documents; however, standardized literature retrieval languages ​​have a high learning curve, and ordinary users often struggle to use them effectively due to a lack of understanding of classification rules or inaccurate expression of search keywords. How to directly retrieve the most needed documents by analyzing the natural language input by users has been a focus of attention in academia and industry, and scholars have proposed many solutions to attempt to address this issue.

[0003] One research paper proposes a word vector weighted average matching retrieval method (SIF) based on smoothed inverse frequency. First, the sentences in the search text and documents are converted into word vector representations. Then, a weighted average of all word vectors in the sentence is calculated to obtain an average vector. Next, the first principal component of the matrix formed by all sentence vectors is subtracted from the average sentence vector. This method is unsupervised, requiring no labeled data and having low training costs. However, because it is unsupervised, it learns the spatial distribution of all data, making it difficult to capture the semantic relationship between two specific texts. While the output results have some reference value, the final output performance is not ideal.

[0004] Another research project constructed an interactive attention-based bidirectional encoder (BERT) retrieval model. Question and document characters were concatenated and input into the model. The model's output classification representation vector was used as the interaction vector, and a relevance score was output through a fully connected layer. This method uses labeled data and, because it inputs concatenated text into the model, possesses some interactivity. However, due to the 512-character length limit for concatenated text, text is truncated during concatenation, resulting in the loss of significant semantic information, making it unsuitable for interactions between long texts.

[0005] In existing technologies, text retrieval mainly includes two types of models: unsupervised text retrieval models and supervised text retrieval models. Unsupervised text retrieval models first convert words in the text into word vectors, then use a series of algorithms to generate text vectors based on the word vectors, and finally calculate the similarity between text pairs based on the text vectors. The disadvantages of this method are: 1) The word segmentation effect greatly affects the word vector effect, and general-purpose word segmenters cannot meet the granularity requirements of the legal field, failing to accurately segment some legal phrases, resulting in poorly trained word vectors and ultimately affecting retrieval performance; 2) Unsupervised models learn the overall distribution of data and cannot effectively capture the inherent connections between texts, resulting in weak interaction between texts.

[0006] Supervised text retrieval models are typically based on pre-trained language models. For example, the attention-based bidirectional encoder (BERT) model concatenates user input text with text from a database, calculates their interaction features, and then calculates a relevance score between the question and the document based on these features. In this approach, the model itself has a length limit on the input; text exceeding this limit is truncated, thus losing semantic information and making it unsuitable for long-text matching scenarios. Furthermore, because model interaction cannot be performed offline, it is difficult to apply such models to large-scale datasets. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a document retrieval method and system based on deep learning.

[0008] According to a first aspect of the present invention, a deep learning-based document retrieval system is provided. The system includes:

[0009] Recall module: Used to generate multiple candidate texts based on pre-stored word vectors and text vectors for the query text input by the user;

[0010] The reordering module is used to input the user-input query text and the multiple candidate texts into a trained long text encoder to obtain query vectors and candidate text vectors, calculate the similarity between the query vector and the candidate text vectors, and obtain sorted retrieval results.

[0011] According to a second aspect of the present invention, a deep learning-based document retrieval method is provided. The method includes the following steps:

[0012] For the query text input by the user, multiple candidate texts are generated based on pre-stored word vectors and text vectors;

[0013] The query text input by the user and the multiple candidate texts are input into a trained long text encoder to obtain query vectors and candidate text vectors. The similarity between the query vector and the candidate text vectors is calculated to obtain the sorted retrieval results.

[0014] Compared with existing technologies, the advantages of this invention are that it effectively solves the problems of unsupervised model ranking performance and supervised text retrieval models' inability to directly retrieve data from a large number of documents, significantly improving retrieval speed and accuracy. Furthermore, this invention can be applied to fields involving the retrieval of long texts from long texts, achieving both high retrieval accuracy and high retrieval speed with a relatively small amount of labeled data, thus possessing broad application prospects.

[0015] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0017] Figure 1 This is an architecture diagram of a deep learning-based document retrieval system according to an embodiment of the present invention;

[0018] Figure 2 This is a flowchart of a deep learning-based document retrieval method according to an embodiment of the present invention. Detailed Implementation

[0019] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0020] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0021] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0022] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0024] This invention provides a deep learning-based document retrieval system. Taking legal case studies as an example, when a user provides a judgment, the retrieval system retrieves several judgments most relevant to that judgment from the database and returns them to the user, thus quickly obtaining the required relevant judgment information. In the following description, for clarity, legal case studies will be used as an example; however, it should be understood that this invention is also applicable to other fields involving the retrieval of long texts, such as academic paper plagiarism detection and patent searches.

[0025] Legal case retrieval faces three main challenges: 1) The volume of case documents is enormous. According to the latest data, the total number of documents has exceeded 130 million, and it continues to increase by tens of thousands every day. This massive amount of data makes it impossible to directly use deep learning models to search or rank the entire case database. 2) In legal case retrieval scenarios, user input is a case study, typically thousands of words long. Using traditional Elasticsearch databases based on inverted indexes for retrieval is very slow and cannot account for issues such as synonyms. Furthermore, the length of the retrieved text exceeds the input length limit of existing retrieval models, making text interaction difficult. 3) Legal case retrieval involves numerous case types, in-depth knowledge, and is difficult and costly to manually annotate, resulting in limited labeled data. Therefore, how to obtain a fast and accurate deep learning-based retrieval system with limited labeled data is a pressing issue.

[0026] This invention addresses the characteristics of case retrieval in the legal field by providing a deep learning-based document retrieval system that employs a two-stage approach of recall followed by ranking. The system as a whole includes a recall module and a ranking module.

[0027] The main purpose of the recall module is to quickly recall multiple candidate answers and output them to the reordering module for sorting. Since the user's input text is long, it is not possible to use the traditional ElasticSearch database for retrieval, so a vector retrieval method is adopted.

[0028] For example, the recall module first uses an unsupervised vector retrieval model to generate offline text vectors. This model generates text vectors by weighting word vectors using smoothed inverse frequency. To better generate word vectors for the legal domain, the recall module uses a legal-specific word segmenter trained on labeled legal data, greatly improving the segmentation effect. Subsequently, the uSIF algorithm is used to generate corresponding semantic representations of the text for use in calculating similarity during recall. The recall module significantly improves the recall rate and speed of cases.

[0029] The rearrangement module employs a supervised deep retrieval model that allows text input lengths of up to several thousand characters. During training, it incorporates contrastive learning, enabling the model to achieve more accurate text representation capabilities on small datasets, thereby further improving the overall retrieval performance of the system.

[0030] For example, to address the issue of semantic information loss due to exceeding the maximum input length of supervised models in long text interactions, the reordering stage uses a pre-trained language model (Longformer) based on a sparse self-attention mechanism for long texts as the basic document encoder. This expands the upper limit of the input sequence length to several thousand characters while reducing the computational complexity of the attention mechanism. The model is trained using methods such as pre-training in the legal domain and contrastive learning to capture the semantic features of long texts. This reordering of multiple candidate texts generated in the recall module improves retrieval performance.

[0031] The following will combine Figure 1 This section provides specific implementation examples of the recall module and the rearrangement module.

[0032] In one embodiment, the recall module performs the following steps:

[0033] Step S1: Generate word vectors.

[0034] Word segmentation is performed using a segmenter trained on legal domain-annotated data. This involves segmenting large-scale data and training word vectors. For example, 20 million cases are randomly selected, segmented using a legal domain-specific segmenter, and the segmented text is input into FastText to train word vectors. Finally, the trained word vectors are stored in Redis for persistence.

[0035] Step S2: Generate text vectors.

[0036] For example, for all case data, after segmenting the data using a legal domain-specific word segmenter, the vector of each word is obtained from Redis, forming a word vector matrix. This matrix is ​​then input into the uSIF algorithm to generate the original text vector. Singular value decomposition is performed on the original text vector to obtain the first 5 principal components. Subtracting the first 5 principal components from the original text vector yields the final text vector. In other words, subtracting the common parts from all text vectors gives their characteristics. The final text vector is then stored in the Milvus vector database for persistence.

[0037] Specifically, for a word-segmented case, the vector v of each word is retrieved from Redis. w The word vector matrix of the text is then concatenated and converted into text vectors using the uSIF algorithm. Specifically, it includes:

[0038] First, calculate α, where α represents the probability that the occurrence probability of a word in the vocabulary is greater than the threshold T. T represents the probability of a word appearing in a document, and n represents the average text length.

[0039]

[0040]

[0041] Among them, represents the vocabulary; represents the vocabulary size; represents the indicator function, which takes 1 when the logic inside the parentheses is true and 0 otherwise; p(w) represents the probability of the word w appearing in the corpus.

[0042] Next, attach weights to the word vector v of each word in the text w to calculate the original text vector The weighted part is called the smoothed inverse frequency.

[0043]

[0044]

[0045] Among them, s represents the text; |s| represents the text vocabulary; Z represents the expectation of the sum of the distances from all word vectors to the text vector. Formula (4) is derived from the Taylor expansion of the probability function of generating words from a given text vector at the point 0. p(w) represents the probability of the word w appearing in the corpus.

[0046] By subtracting the common expressions in the corpus (the common expressions are some high-frequency characters or words in the corpus, such as "de"), the semantic expression ability of the sentence is improved. For all texts in the corpus, obtain their text vectors and arrange them into a text vector matrix Perform singular value decomposition on the obtained matrix to obtain the first 5 principal components of the matrix. Thus, the final text vector representation c s is obtained, and the formula is as follows:

[0047]

[0048]

[0049] Among them, σ i represents the eigenvalue, and λ i represents the weight of each principal component. represents the original text vector, and c′ i represents the singular vector of matrix A. proj represents the projection operation. m is the number of principal components, and the value is 5 when applied.

[0050] The above process describes the process of converting a text into a text vector.

[0051] Finally, to facilitate vector retrieval, the generated vectors are stored in the Milvus vector database for persistence.

[0052] Step S3: For the new text input by the user, retrieve the most relevant vectors as the recall text.

[0053] The recall process is similar to the vector generation process. First, the user-input case (or input text) is segmented using a legal-specific word segmenter. Then, the vector corresponding to each segment is found in Redis and input into the uSIF algorithm to obtain the original text vector. The principal components obtained by the vector generation module are then subtracted to obtain the final text vector. Finally, the K most semantically similar vectors and their corresponding document IDs are retrieved using the Milvus database.

[0054] Second, the rearrangement module:

[0055] In one embodiment, the reordering module uses a deep model (Longformer) based on sparse self-attention mechanism for long text as the basic encoder. Positive and negative sample triples are constructed from a legal retrieval labeled dataset as training data for the learning module. A representation-based framework is employed, where each case text is individually encoded into a vector by the text encoder. Finally, the reordering score is obtained by calculating the similarity between the K cases output by the recall module and the case. The reordering module comprises a training module and a ranking module.

[0056] 1) Training module:

[0057] To further improve the quality of document vectors generated by the Longformer encoder, alleviate the anisotropy of its native sentence representation vector space, and address the problem of limited training data, one embodiment uses contrastive learning to fine-tune the Longformer encoder. Training data is constructed from a case-annotated dataset. For a given query text, relevant cases are considered positive samples, while irrelevant cases are considered negative samples. Thus, each query, along with its positive and negative samples, forms a triplet. For a batch of training data, x i Only positive samples Negative samples, except In addition, it contains positive and negative samples of other texts. Each text is encoded by a Longformer encoder, then processed by an average pooling layer to obtain a vector representation. A similarity matrix is ​​constructed by cosine similarity calculation, with each row of the matrix representing x. i All data in the same batch and The similarity score, with cross-entropy as the training objective, is expressed as:

[0058]

[0059] Among them, l i Let represent the loss generated by the i-th text; e is the natural constant; sim() is the cosine similarity function used to calculate the similarity between two vectors; h i For text x i The vector representation of , For text The vector representation of , For text The vector representation of τ. τ represents the temperature hyperparameter.

[0060] Through the training of the above comparative learning, the alignment and uniformity of the model-generated vectors can be improved. That is, similar instances have similar vectors and the vector representations are more evenly distributed in space, so that the similarity calculation between vectors can well represent the similarity between texts.

[0061] 2) Sorting module

[0062] After fine-tuning with contrastive learning on a small dataset, the Longformer long text encoder can capture the semantics of long texts and generate high-quality vector representations. The query text input by the user and the candidate texts recalled in the recall stage are respectively input into the Longformer long text encoder. The output of the first token, namely the [CLS] token, is used as the query vector and the candidate text vector. The cosine similarity is used to calculate the similarity between the query vector and the candidate text vector, and the results are sorted in descending order to obtain the final ranking result.

[0063] In summary, this invention proposes a deep learning-based document retrieval system. By processing text into vectors, it achieves a unified representation of texts of varying lengths. Vector retrieval is then used to recall multiple texts as candidate answers. Finally, a pre-trained language model (Longformer) based on a sparse self-attention mechanism is used as the basic document encoder, combined with contrastive learning to generate more accurate text vector representations. A similarity function is then used to reorder the candidate texts. The system combines the advantages of vector retrieval and deep learning models, significantly improving the performance of long-text retrieval while maintaining retrieval speed.

[0064] Accordingly, the present invention also provides a deep learning-based document retrieval method for implementing one or more aspects of the above-mentioned system. For example, see [link to relevant documentation]. Figure 2As shown, the method includes: step S210, generating multiple candidate texts based on pre-stored word vectors and text vectors for the query text input by the user; step S220, inputting the query text input by the user and the multiple candidate texts into a trained long text encoder to obtain query vectors and candidate text vectors, and calculating the similarity between the query vectors and candidate text vectors to obtain ranked retrieval results.

[0065] To further verify the effectiveness of this invention, based on long legal case documents published on the China Judgments Online website, and using 900 real-world user questions (300 civil, 300 criminal, and 300 administrative questions each), the present invention was used for retrieval. Three sets of experiments were conducted: the first set used a general-purpose word segmenter and a vector recall module, but not a reordering module; the second set used a legal-specific word segmenter and a vector recall module, but not a reordering module; and the third set used a legal-specific word segmenter, a vector recall module, and a reordering module. The evaluation metrics for the experimental results were Mean Precision (MAP), Mean Reciprocity Rating (MRR), and Normalized Diminished Cumulative Gain (NDCG). After manual review by legal professionals, the third set of experiments showed the best results, followed by the second set, and the first set showed the worst results. The experimental results indicate that both the legal-specific word segmenter and the reordering module can improve the system's retrieval performance, namely, improving retrieval speed and accuracy.

[0066] In summary, this invention proposes a deep learning-based document retrieval system that effectively solves the problem of case retrieval using deep models. Combining the characteristics of case retrieval, it employs a two-stage process of recall and reordering for case retrieval, improving both retrieval speed and accuracy. The recall module uses an unsupervised vector retrieval model, which does not limit text length and has high recall rate and speed. The reordering module uses a supervised deep retrieval model, allowing users to input case documents of several thousand words. With limited labeled data, contrastive learning is used to enhance the text representation capabilities of the deep model, further improving the overall retrieval performance of the system. Specifically, the advantages of this invention are mainly reflected in the following aspects:

[0067] 1) A legal-specific word segmenter was used in word vector generation, which can segment legal data more accurately.

[0068] 2) uSIF was used as the vector generation algorithm in the text vector generation stage, which can generate more accurate case semantic vectors and does not limit the input length.

[0069] 3) uSIF was used as the vector generation algorithm and Milvus as the retrieval database in the vector recall stage. This can generate more accurate case semantic vectors, does not limit the input length, and can quickly retrieve semantically relevant text.

[0070] 4) In the rearrangement stage, a contrastive learning framework was used to train the long text encoder, which enabled it to have good generalization effect on a small dataset and made its distribution in the vector space more uniform, thus eliminating anisotropy to a certain extent and improving the semantic expression ability of the model.

[0071] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0072] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0073] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0074] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0075] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0076] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0077] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0079] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A deep learning-based document retrieval system, comprising: Recall module: Used to generate multiple candidate texts based on pre-stored word vectors and text vectors for the query text input by the user; The reordering module is used to input the user-input query text and the multiple candidate texts into a trained long text encoder to obtain query vectors and candidate text vectors, and to calculate the similarity between the query vectors and candidate text vectors to obtain sorted retrieval results. The recall module generates the plurality of candidate texts according to the following steps: For the query text input by the user, a legal-specific word segmenter is used to segment the text, and the word vector corresponding to each segmented word is found in the pre-stored first database. The word vector is then input into the uSIF algorithm to obtain the original vector of the query text. Subtract the pre-stored principal components from the original query text vector to obtain the final query text vector; The K vectors that are semantically most similar to the final query text vector and their corresponding text identifiers are retrieved using a pre-stored second database, and these are used as the multiple candidate texts to be recalled, where K is a set integer; The first and second databases are obtained according to the following steps: Existing case data is randomly selected, and a legal-specific word segmenter is used for word segmentation. The segmented text is then input into a text classifier to train word vectors, and the trained word vectors are stored in the first database for persistent storage. For all selected case data, a legal domain-specific word segmenter is used to segment the data, and the vector of each word is obtained from the first database. The resulting word vector matrix is ​​input into the uSIF algorithm to generate the original text vector of the sample case. Singular value decomposition is then performed on the original text vector of the sample case to obtain a set number of principal components. The original text vector of the sample case is then subtracted from the principal components to obtain the final text vector of the sample case. The final text vector of the sample case is then stored in the second database for persistence.

2. The system according to claim 1, characterized in that, The training process of the long text encoder includes: Training data is constructed from the case sample annotation dataset. For a query text, relevant cases are treated as positive samples and irrelevant cases as negative samples. Each query text is then paired with its positive and negative samples to form a triplet. For a batch of training data, x i Only positive samples Negative samples, except In addition, it includes positive and negative samples of other texts. Each query text is encoded by a long text encoder, then passed through an average pooling layer to obtain a vector representation. A similarity matrix is ​​constructed by cosine similarity calculation, with each row of the matrix representing x. i All of the training data in the same batch and Similarity score; Training is performed with the set loss function as the optimization objective.

3. The system according to claim 2, characterized in that, The loss function is set as follows: Among them, l i The loss generated by the i-th text is represented by e, which is the natural constant; sim() is the cosine similarity function; h i It is x i The vector representation of , yes The vector representation of , yes The vector representation of τ, where τ represents the temperature hyperparameter.

4. The system according to claim 1, characterized in that, The first database is a Redis database, the second database is a Milvus vector database, and the text classifier is a FastText text classifier.

5. The system according to claim 1, characterized in that, The similarity between the query vector and the candidate text vector is calculated based on cosine similarity.

6. The system according to claim 1, characterized in that, The recall module converts the text into a text vector in the following manner: For the obtained word vector sequence, the word vectors are weighted by combining the smooth inverse frequency of words in the domain to generate the original text vector, wherein the smooth inverse frequency of words in the domain reflects the number of times the words appear in the domain literature.

7. A deep learning-based document retrieval method, comprising the following steps: For the query text input by the user, multiple candidate texts are generated based on pre-stored word vectors and text vectors; The query text input by the user and the multiple candidate texts are input into a trained long text encoder to obtain a query vector and candidate text vectors. The similarity between the query vector and the candidate text vectors is calculated to obtain the sorted retrieval results. The plurality of candidate texts are generated according to the following steps: For the query text input by the user, a legal-specific word segmenter is used to segment the text, and the word vector corresponding to each segmented word is found in the pre-stored first database. The word vector is then input into the uSIF algorithm to obtain the original vector of the query text. Subtract the pre-stored principal components from the original query text vector to obtain the final query text vector; The K vectors that are semantically most similar to the final query text vector and their corresponding text identifiers are retrieved using a pre-stored second database, and these are used as the multiple candidate texts to be recalled, where K is a set integer; The first and second databases are obtained according to the following steps: Existing case data is randomly selected, and a legal-specific word segmenter is used for word segmentation. The segmented text is then input into a text classifier to train word vectors, and the trained word vectors are stored in the first database for persistent storage. For all selected case data, a legal domain-specific word segmenter is used to segment the data, and the vector of each word is obtained from the first database. The resulting word vector matrix is ​​input into the uSIF algorithm to generate the original text vector of the sample case. Singular value decomposition is then performed on the original text vector of the sample case to obtain a set number of principal components. The original text vector of the sample case is then subtracted from the principal components to obtain the final text vector of the sample case. The final text vector of the sample case is then stored in the second database for persistence.

8. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the method according to claim 7.

Images