A case retrieval method

Abstract

The present invention discloses a case retrieval method. First, enterprise case documents are encoded by a stack-based encoding method. Based on the Transformer encoder, this method introduces a stack simulation layer to model the recursive state of long texts. Then, a dimensionality reduction algorithm for case document vectors based on structured information preservation is proposed. A deep learning model based on variational autoencoder and random quadruple loss is used to reduce the dimensionality of case document vectors, effectively retaining the structural information of case documents. At the same time, a vector dimensionality reduction evaluation scheme for structure preservation is designed. Finally, in order to improve the accuracy of user retrieval, a re-ranking method based on case documents and retrieval matching scores is proposed. This method takes into account the contextual information of other ranked candidate documents and uses a lightweight re-ranking framework of context to significantly enhance the effect of the retrieval model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a case retrieval method based on stack coding and structured information retention. Background Art

[0002] As power grid companies' businesses continue to expand and become more complex, case management has become an integral part of their operations. Power grid companies are involved in a wide range of cases, including contract disputes, intellectual property disputes, labor disputes, and environmental protection lawsuits. To effectively address these legal matters, power grid companies require efficient case retrieval methods to ensure timely access to relevant case information and materials when faced with legal risks and disputes. However, traditional case retrieval methods are often inefficient and fail to fully leverage modern technology to meet the needs of companies.

[0003] Traditional case retrieval methods typically rely on keyword matching and simple text retrieval, making them difficult to process complex legal texts and case structure. This approach not only suffers from low retrieval accuracy but also tends to overlook important implicit information within the case. With the advancement of natural language processing technology, existing methods typically use models such as Transformers to learn feature representations of complex case documents to enhance their understanding and analysis. They then use autoencoders to reduce the dimensionality of long document vectors to facilitate subsequent matching and scoring with user search vectors. Finally, they output a list of case and corporate documents in descending order of score. While these methods have improved retrieval accuracy and intelligence to some extent, they still face three major challenges: 1) Corporate case documents are lengthy, and traditional Transformer encoders employ a self-attention mechanism that lacks explicit recursive state tracking. Consequently, Transformer language models struggle to capture long-tail recursive structures, resulting in low efficiency when encoding long texts and poor generalization. 2) Autoencoders encode high-dimensional input data into low-dimensional latent representations by sampling from a latent space distribution and decode these representations into a reconstruction of the input data. During the training process, the autoencoder uses an additional quantity to quantify the Kullback-Leibler (KL) divergence between the latent space distribution and the latent prior as an additional loss term, and learns the parameters by reducing the reconstruction loss. Since the encoder and decoder networks are probabilistic, the distribution in the latent space cannot be controlled by the reconstruction loss alone, which makes it easy to lose the global structural information of the vector. 3) Existing methods use contrastive learning to train dense retrieval models. Contrastive learning uses (query, positive sample document, negative sample document) triplets as training samples. When training the model, it is required that the similarity score between the query vector and the true relevant (positive sample) document vector is higher than the similarity score between the query and the negative sample document vector. This type of method only considers positive and negative sample document pairs, does not compare a given query with multiple documents, and ignores the information between documents. Therefore, there is an urgent need for a new solution to solve the above technical problems. Summary of the Invention

[0004] The technical problem addressed by this invention is that corporate case documents are long. Traditional Transformer encoders are inefficient at encoding long texts and have weak generalization capabilities. Using autoencoders for dimensionality reduction fails to control the distribution in the latent space, easily losing the global structural information of the vectors. This method employs contrastive learning to train a dense retrieval model, considering only positive and negative document pairs without comparing a given query with multiple documents. This approach also ignores information between documents. The proposed method provides a case retrieval method based on stacked encoding and the preservation of structured information.

[0005] The technical solution adopted by the present invention to solve its technical problems is: a case retrieval method based on stack encoding and structured information retention. For a given user retrieval query, this method first encodes all enterprise case documents through a stack-based encoding method. On the basis of the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts. Then, a dimensionality reduction method for case document vectors based on structured information retention is proposed. A deep learning model based on variational autoencoder and random quadruple loss is used to reduce the dimensionality of case document vectors, effectively retaining the structural information of case documents. At the same time, a vector dimensionality reduction evaluation scheme for structure retention is designed. Finally, in order to improve the accuracy of user retrieval, a re-ranking method based on case documents and retrieval matching scores is proposed. This method takes into account the contextual information of other ranked candidate documents and uses a lightweight re-ranking framework of context to significantly enhance the effect of the retrieval model.

[0006] The case retrieval method based on stack coding and structured information retention of the present invention comprises the following steps:

[0007] The pre-trained language model BERT is used to encode the given user retrieval query to obtain the vector representation Q.

[0008] A stack-based encoding algorithm is used to encode all enterprise case documents and obtain the vector representation D=

[0009] {d1,d2,...,d n};

[0010] The obtained case document vector representation D is represented by a dimensionality reduction algorithm based on the case document vector that retains structured information, and the case document vector is reduced in dimension to obtain a low-dimensional case document vector representation D. ′ , to improve the efficiency of user retrieval and document matching;

[0011] For the proposed case document vector dimensionality reduction algorithm based on structured information preservation, a vector dimensionality reduction evaluation scheme oriented to structure preservation is designed;

[0012] A re-ranking algorithm based on the case document and user search matching scores is used for the obtained user search vector Q and the obtained case document vector D′ to improve the accuracy of user search, and the case document list is output in descending order of scores and the search results are returned.

[0013] As a further improvement to this invention, a pre-trained language model, BERT, is used to encode a given user search query, using the input format [CLS]query[SEP]. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After BERT pre-training, this classification identifier learns the representation information of the user search query, represented by a vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0014] As a further improvement of the present invention, for a certain enterprise case document, in the process of encoding using Transformer, like the ordinary self-attention mechanism, the hidden state sequence is obtained. Transformer language models are a class of neural sequence models that use a multi-head self-attention mechanism to obtain contextualized representations of tokens in a sequence and then use these representations to predict the next token.

[0015] As a further improvement to the present invention, a stack-based encoding method is used to encode all corporate case documents. Building on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts, enabling accurate encoding of corporate case documents. This stack simulation layer uses a stack tape to simulate a pushdown automaton that performs shift / reduce operations when tokens are predicted. The contents of the stack tape encode the recursive state by tracking the depth to which each token is reduced in the stack.

[0016] As a further improvement to the present invention, an algorithm is designed to automatically update the stack. When predicting the next word, the Transformer with the stack simulation layer selects a component to update the stack. This is achieved by selecting a word to be reduced from the candidate word-grams, which is a probabilistic additive decision derived from the attention operation on all word-grams. When finally calculating the attention score, for each word-gram, the present invention maps the contents of the stack to the depth vector of each layer. These depth vectors are added to the attention key, resulting in a local additive modulation of the attention score.

[0017] As a further improvement to the present invention, a dimensionality reduction method for case document vectors based on structured information preservation was designed. This method leverages the concepts of structure preservation at both the local and global levels to create a deep learning model based on a variational autoencoder and a randomized quadruple loss to reduce the dimensionality of case document vectors. This model, consisting of only a variational encoder and sampler (without a decoder), uses a "group loss" function to create vectors. Compared to traditional variational autoencoders, this model exhibits less global structural distortion while maintaining model parameterization and architectural flexibility.

[0018] As a further improvement of the present invention, a deep learning model based on a variational autoencoder and a random quadruple loss is trained by minimizing the group loss and the prior KL divergence. The group loss adopts the concept of a quadruple loss function and calculates the quadruple normalized distance between the original point and the vector point from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable cost function of each randomly selected quadruple point. The present invention denotes the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j as ρ ij and d ij To calculate the group-normalized distance between two points in the same group (for quartets, quintuples, sextuples, etc.), the present invention uses all pairwise distances within the group. For high-dimensional and low-dimensional points, the present invention formulates the group-normalized distance formulas as follows:

[0019]

[0020]

[0021] Here, n is the number of points in each group.

[0022] The group normalized distance difference between the high-dimensional input point and the low-dimensional vector point should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0023]

[0024] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are dynamically grouped within each batch, with the group size (n) specified as a hyperparameter. The group loss for each point i in the training batch is specified as the cost of the group with i as the first point, and the group loss term for each batch is calculated by averaging over the entire batch.

[0025] As a further improvement of the present invention, the vector dimension reduction evaluation scheme for structure preservation is used in the present invention to use the neighborhood overlap curve R based on sorting. NX (Rank-based Neighborhood Overlap Curve) is used as a quality assessment indicator. This curve quantifies the overlap of the neighbor rankings of the high-dimensional vector and the low-dimensional vector relative to the reference point, covering all neighborhood sizes from 1 to (N-1) (the sample size is N) and taking the average value for all reference points. In order to design a vector dimensionality reduction evaluation scheme for structure preservation, this paper proposes an R-based NX Quality assessment method of curves.

[0026] R NXThe curve quantifies the neighborhood relationship between the document vector in the high-dimensional space and the document vector in the low-dimensional space after dimensionality reduction, thereby verifying whether the structural information of the original document is effectively preserved during the dimensionality reduction process. The specific design process is as follows:

[0027] First, the high-dimensional case document vectors are compared with the low-dimensional case document vectors after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined.

[0028] Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points.

[0029] Afterwards, plot R according to different neighborhood sizes. NX This curve quantifies the overlap between high-dimensional and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document.

[0030] The main purpose of the designed structure-preserving vector dimensionality reduction evaluation scheme is to verify and ensure the effectiveness of the dimensionality reduction method in preserving the structural information of case documents. Specifically, the evaluation scheme uses R NX The curve quantitatively analyzes the neighborhood overlap of document vectors before and after dimensionality reduction, ensuring that the dimensionality reduction operation retains the relative relationships and structural features between documents as much as possible when compressing document vectors.

[0031] The effectiveness of the reduced case document vector in preserving structural information is evaluated through a dimensionality reduction scheme oriented towards structure preservation. The scheme uses a sorting-based neighborhood overlap curve R NX To quantify the neighborhood overlap between high-dimensional and low-dimensional vectors, ensure that the dimensionality reduction operation retains the structural information of the document as much as possible, thereby improving the accuracy of document matching and retrieval.

[0032] As a further improvement of the present invention, a rearrangement method based on the matching scores of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are used as input, and the matching score between each enterprise case document and the user retrieval is calculated using the scoring function τ(·). During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D of the document vector. ′ To adapt to different scoring functions and match the dimension of the retrieval vector Q. This method can adaptively handle different retrieval tasks and ensure that the retrieval relevance is maximized when sorting the retrieval results.

[0033] Algorithm Output: After applying this re-ranking algorithm, the system sorts the case documents based on the calculated matching scores, outputting a list of case documents sorted in descending order of matching scores. Ultimately, users will see the case documents most relevant to their search intent at the top, improving search accuracy and efficiency.

[0034] As a further improvement of the present invention, in order to maximize the use of the joint scoring of all enterprise case documents retrieved by each user, the present invention uses the so-called ranked list optimization method (ListNet) as the loss function, which is the predicted score of all candidate enterprise case documents and the true relevance label of the same group of candidate enterprise case documents provided by the dataset. KL divergence between the distributions (the relevance score of a positive document is a positive scalar, while the relevance score of a negative document or a document whose label is not well defined is set to -∞):

[0035]

[0036] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

[0037] Compared to existing technologies, the present invention has the following advantages: 1) Existing methods use Transformer encoders to encode case documents. Traditional Transformer encoders are inefficient when encoding long texts and have weak generalization capabilities. This present invention, however, builds on the Transformer encoder by introducing a stacked simulation layer to model the recursive state of long texts. Furthermore, a structure-preserving vector dimensionality reduction evaluation scheme is designed. 2) Existing methods use autoencoders to reduce the dimensionality of learned document representation vectors. This method cannot control the distribution in the latent space and is prone to losing the vector's global structural information. This present invention proposes a structure-preserving case document vector dimensionality reduction method. This method uses a deep learning model based on a variational autoencoder and a random quadruple loss to reduce the dimensionality of case document vectors, effectively preserving the structural information of the case documents. Furthermore, a structure-preserving vector dimensionality reduction evaluation scheme is designed. 3) Existing methods use contrastive learning to train intensive retrieval models, considering only positive and negative document pairs. This method does not compare a given query with multiple documents and ignores information between documents. The present invention proposes a re-ranking method based on case documents and retrieval matching scores, which takes into account the contextual information of other ranked candidate documents and uses a lightweight re-ranking framework of context to significantly enhance the effect of the retrieval model.

[0038] Experimental analysis has shown that the case retrieval method based on stack coding and structured information retention proposed by this method has improved the case retrieval model of power grid enterprises and greatly improved the performance and accuracy of the retrieval model. In addition, the present invention improves the encoding ability of recursive structures by adding stack storage to the Transformer language model, which can better understand and generate language structures with complex recursive characteristics. This method does not require the calculation of the k-nearest neighbor graph of the input data, which greatly reduces the computational complexity. Less data is required during training to achieve performance similar to or better than that of the standard Transformer model. It does not change the output space of the underlying sequence model, nor does it impose hard constraints on the attention mechanism, allowing the model to learn to use syntactic structures through gradient descent. The above characteristics make this method widely applicable in downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a basic process schematic diagram of the present invention;

[0040] Figure 2 It is a model framework diagram of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0042] Example 1: See Figure 1 、 Figure 2 To address the above issues and improve the performance and accuracy of power grid enterprise case retrieval, this paper proposes a case retrieval method based on stack encoding and structured information preservation. For the first problem, based on the Transformer encoder, this paper introduces a stack simulation layer to model the recursive state of long texts. This is a new self-attention layer that simulates the recursive state through a stack band. This simulation layer autoregressively predicts new tokens while simultaneously updating the stack band, thereby using the stack band to gently modulate attention on tokens. For the second problem, to preserve the structured information of enterprise case documents, this paper adopts the concepts of structure preservation at both local and global levels. It creates a deep learning model based on a variational autoencoder and a randomized quadruple loss from the SQuadMDS algorithm to reduce the dimensionality of case document vectors. This model uses a "group loss" function to create vectors, which has less global structural distortion than traditional VAEs while maintaining the model's parameterization and architectural flexibility. For the third problem, this paper designs a re-ranking method based on the matching score between case documents and retrieval. This method takes into account the contextual information of other ranked candidate documents and jointly scores a large number of negative samples, allowing the model to directly optimize information retrieval indicators and improve the performance and accuracy of user retrieval.

[0043] The case retrieval method based on stack coding and structured information retention of the present invention includes the following steps:

[0044] 1) Use the pre-trained language model BERT to encode the given user search query and obtain the vector representation Q. BERT is based on a bidirectional Transformer structure and uses a masked language model to achieve integrated feature fusion. It can model polysemy and generate deep bidirectional language representations. Therefore, the present invention uses BERT encoding, and the specific input form is [CLS]query[SEP]. Specifically, it represents the beginning of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by the vector Q. The separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0045] 2) A stack-based encoding method is used to encode all enterprise case documents and obtain a vector representation D = {d1, d2, ..., d |D| This paper designs a stack-based encoding method. Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts to achieve accurate encoding of corporate case documents.

[0046] First, for a certain corporate case document, in the process of using Transformer for encoding, like the ordinary self-attention mechanism, the hidden state sequence is obtained Transformer language models are a type of neural sequence model that uses a multi-head self-attention mechanism to obtain contextual representations of tokens in a sequence and then use these representations to predict the next token. Specifically, for a certain corporate case document, the input sequence X = {x1,…,x n}.set up is the hidden state of the i-th word in the k-th attention block. Then, the hidden vector representation of the i-th word is updated as:

[0047]

[0048] in is a learned matrix. FFN is a module that combines feedforward neural network, residual network and layer normalization. f is the fth self-attention head. Each attention head performs a weighted average of its input,

[0049]

[0050] where α ij is the attention weight assigned by the i-th word to the j-th word.

[0051] Then, the stack simulation layer proposed by the present invention uses a stack with S j ∈{0, k} k to simulate a pushdown automaton that performs shift / reduce operations as tokens are predicted. The contents of the stack encode the recursive state by tracking the depth to which each token is reduced in the stack. Specifically, when observing the prefix X ≤j ={x1, x2, ..., x j )After that, S j [k] = 0 means word x k is not reduced with any other tokens, and S j [k] = q represents x k appears in q reduction operations, resulting in the word x in the component k At depth q.

[0052] Next, the present invention updates the stack. Specifically, given the prefix X <j , the Transformer with stacked simulation layers predicts the next word x j And the stack with S j-1 This is done by updating the candidate word {x1, x2, ..., x j} select a word t to be reduced j This selection is achieved by where O is the final layer of the Transformer and is the new predicted word x j The vector representation of is obtained as This vector is a probabilistic additive decision resulting from the attention operation on all tokens.

[0053]

[0054] Among them, MLP(·) is a multi-layer perceptron, is a learned parameter matrix. The present invention uses these probabilities to select word t j =argmaxp(c|x ＜j ;S j-1 ) is reduced and the stack is updated by the following steps:

[0055] 1) Initialization and input: the state S at the previous moment j-1 , current index j, target t associated with j j And the current stack as input.

[0056] 2) Status update: First, update the current status S j Initialized to the same state as the previous state S j-1same.

[0057] 3) Create constituent elements: define a list named struct, which initially contains the current index j.

[0058] 4) Stack operation judgment: Check whether t is equal to j. If so, push the struct onto the stack and the algorithm ends.

[0059] 5) Loop reduction processing: If t is not equal to j, enter the loop and perform the reduction operation:

[0060] - Pop the top element Top from the stack.

[0061] -Update struct and add the popped Top to the front of the struct.

[0062] -Update the state Sj, and for each index i in the struct, increase the value of Sj[i] by 1.

[0063] 6) Check the termination condition: After each loop, check whether the popped Top is equal to the target tj:

[0064] - If yes, end the loop.

[0065] -If not, continue the loop and pop the next top element of the stack.

[0066] 7) Push and end: Push the updated struct back to the stack to complete the update of the entire stack.

[0067] In this manner, the algorithm continuously processes the contents of the stack, combining and reducing them according to the input goal until the termination condition is met. This processing method allows the algorithm to dynamically adjust the state of the stack to adapt to the given structure and goal. Note that the present invention makes additional component decisions by calculating the additional score of the rightmost word in the component.

[0068] Finally, the attention score is calculated. For each word unit c∈{0, 1, ..., j}, the present invention converts S j The content is mapped to the depth vector of each layer These depth vectors are added to the attention key, producing a local additive modulation of the attention score,

[0069]

[0070] Since these logits are themselves part of a softmax and nonlinearity, the overall effect can be arbitrarily nonlinear.

[0071] 3) The case document vector obtained in step 2) is represented as D = {d1, d2, ..., d |D|The present invention designs a dimensionality reduction method for case document vectors based on structured information preservation, performs dimensionality reduction on case document vectors, and obtains a low-dimensional case document vector representation D′={d′1,d′2,.....,d′ |D|} to improve the efficiency of user retrieval and document matching.

[0072] Traditional dimensionality reduction methods typically use autoencoders, which encode high-dimensional input data into a low-dimensional latent representation by sampling from a latent space distribution and decode it into a reconstruction of the input data. During training, the autoencoder uses an additional loss term to quantify the KL divergence between the latent space distribution and the latent prior, learning parameters by reducing the reconstruction loss. Because the encoder and decoder networks are probabilistic, the reconstruction loss alone cannot control the distribution in the latent space, which can easily lose the global structural information of the vector.

[0073] To preserve the structured information of corporate case documents, this paper employs both local and global structure-preserving concepts to create a deep learning model based on a variational autoencoder and the stochastic quadruple multidimensional scaling algorithm (SQuadMDS) to reduce the dimensionality of case document vectors. This model, constructed using a "group loss" function, exhibits less global structural distortion than traditional VAEs while maintaining model parameterization and architectural flexibility.

[0074] like Figure 2 As shown in the dimensionality reduction module in , the model only contains a variational encoder and sampler (no decoder), and minimizes the group loss and the prior KL divergence during training. The group loss adopts the concept of the quadruple loss function, and calculates the quadruple normalized distance between the original point and the vector point from the SQuadMDS algorithm. The normalized distance is used to calculate the differentiable cost function of each randomly selected quadruple point. The present invention denotes the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j as ρ ij and d ij To calculate the group normalized distance between two points in the same group (for quadruple

[0075] For high-dimensional and low-dimensional points, the present invention formulates the group normalized distance formula as follows:

[0076]

[0077]

[0078] Here, n is the number of points in each group.

[0079] The group normalized distance difference between the high-dimensional input point and the low-dimensional vector point should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0080]

[0081] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are dynamically grouped within each batch, with the group size (n) specified as a hyperparameter. The group loss for each point i in the training batch is specified as the cost of the group with i as the first point, and the group loss term for each batch is calculated by averaging over the entire batch.

[0082] Therefore, the model imposes constraints on the distribution in the latent space instead of using reconstruction loss to calculate weight updates, which preserves the global structure information of the vector to some extent.

[0083] 4) Regarding the dimensionality reduction method of the case document vector based on structured information preservation proposed in step 3), in order to prove the effectiveness of the dimensionality reduction method proposed in the present invention in preserving document structure information, the present invention designs a vector dimensionality reduction evaluation scheme for structure preservation. NX The curve is used as a quality assessment metric. This curve quantifies the overlap of the neighbor ordering of the high-dimensional vector with the low-dimensional vector relative to the reference point, covering all neighborhood sizes from 1 to (N-1) (with a sample size of N) and averaged over all reference points.

[0084] To this end, the present invention expresses the neighborhood ranking of point j (neighbor) relative to point i (reference point) in the high-dimensional vector and the low-dimensional vector as HD ij and LD ij The non-self-neighborhoods of high-dimensional vectors and low-dimensional vector points are expressed as and Where E is the neighborhood size. For a dataset of size N, Q for a specific value of E NX The value is calculated as follows:

[0085]

[0086] To get the full Q NX The present invention calculates the score of E from 1 to (N-1). The results show that random vectors will lead to With Q NX In comparison, R NX After correcting for chance, the calculation formula is:

[0087]

[0088] The present invention uses vector RNX The structure-preserving information is quantified by the area under the curve (AUC) of the curve. Specifically, local structure-preserving information is the AUC of the curve with the neighborhood size E rescaled by the logarithm lnE, so as to give more weight to local neighborhoods while not imposing a hard boundary between local and global. Furthermore, global structure-preserving information is the AUC of the line with E scaled linearly, so as to de-emphasize the local neighborhood of the reference point. In both cases, a higher score for structure-preserving information is better.

[0089] 5) For the user search vector Q obtained in step 1) and the enterprise case document vector D′ obtained in step 3), a re-ranking method based on the matching scores between enterprise case documents and user searches is proposed to improve the accuracy of user searches. A list of case documents is output in descending order of scores, and the search results are returned.

[0090] Specifically, if Figure 2 As shown in the matching scoring module in [1], the user retrieval vector Q and the reduced-dimensional enterprise case document vector D′ are used as inputs, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and the user retrieval. During the calculation process, the use of learnable linear projection can potentially change the dimension D′ of the document vector.

[0091] To adapt to different scoring functions and match the dimension of the search vector Q. The specific calculation formula is as follows:

[0092]

[0093] Among them, θ s is a parameter of the scoring function. While a variety of functions can be used as the scoring function, the present invention only utilizes a simple inner product. In addition to computational efficiency, the main advantage of the above function is that it can be used directly with the fine-tuned user retrieval encoder for dense retrieval (single-stage) via a fast approximate nearest neighbor search. In contrast, the nonlinear scoring module only allows the model to be used in a two-stage retrieval setting (candidate retrieval followed by re-ranking).

[0094] A key difference between the re-ranking method based on the matching scores of enterprise case documents and user retrieval proposed in the present invention and all dense retrieval methods is that for each query, in addition to k positive (real) documents, the model is also trained to jointly score the top |D|-k candidate documents retrieved by a certain candidate retrieval method. The present invention notes that the candidate retrieval method does not need to be the same as the basic method that provides the document vector D′ and the retrieval vector Q encoder. Therefore, it is allowed to utilize methods with different characteristics (for example, methods with higher recall or precision, or methods based on vocabulary overlap / sparse representation, such as BM25).

[0095] In order to maximize the use of the joint scoring of |D| enterprise case documents retrieved by each user, the present invention uses the so-called ranked list optimization method (ListNet) as the loss function, which is the predicted score of |D| candidate enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. KL divergence between the distributions (the relevance score of a positive document is a positive scalar, while the relevance score of a negative document or a document whose label is not well defined is set to -∞):

[0096]

[0097] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function. This invention differs from existing dense search methods by combining the joint scoring of user searches and enterprise case documents with the KL divergence loss for the first time. This combination establishes and utilizes context for each user search, further improving search performance and accuracy. The benefits of this invention are expected to be even greater for datasets containing multiple search document labels, especially those defined across multiple relevance levels.

[0098] Example 2: A case retrieval device, comprising:

[0099] The encoding module is used to encode the given user search query using the pre-trained language model BERT to obtain the vector representation Q, and to encode the enterprise case document using a stack-based encoding algorithm to obtain the vector representation D = {d1, d2, ..., d n The obtained case document vector representation D is reduced in dimension by a dimensionality reduction algorithm based on case document vectors that preserves structured information, and a low-dimensional case document vector representation D′ is obtained. The dimensionality reduction module is used to design a structure-preserving oriented vector dimensionality reduction evaluation scheme for the proposed dimensionality reduction algorithm for case document vectors that preserves structured information.

[0100] Rearrangement module, used to get the user retrieval vector Q and the case document vector D ′ ,Design a re-ranking method based on the matching score between case documents and user retrieval, and return the retrieval results;

[0101] The re-ranking method based on the matching scores between case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are used as inputs, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and the user retrieval. During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D of the document vector. ′To adapt to different scoring functions and match the dimension of the retrieval vector Q.

[0102] The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After BERT pre-training, the classification identifier will learn the representation information of the user search query, represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0103] A stack-based encoding algorithm is used to encode corporate case documents. For corporate case documents, a transformer network is used to encode and obtain a hidden state sequence. Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts; the deep learning model based on the variational autoencoder and random quadruple loss is trained by minimizing the group loss and the prior KL divergence. Among them, the group loss adopts the concept of the quadruple loss function, and calculates the quadruple normalized distance between the original point and the vector point from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point, and the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows:

[0104]

[0105]

[0106] Where n is the number of points in each group,

[0107] The group normalized distance difference between the high-dimensional input point and the low-dimensional vector point should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0108]

[0109] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are dynamically grouped in each batch, with the group size (n) specified as a hyperparameter. The group loss for each point i in the training batch is specified as the cost of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

[0110] For the proposed case document vector dimensionality reduction algorithm based on structured information preservation, a vector dimensionality reduction evaluation scheme oriented to structure preservation is designed;

[0111] Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows:

[0112] First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined.

[0113] Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points.

[0114] Afterwards, plot R according to different neighborhood sizes. NX The R_{NX} curve quantifies the overlap between high-dimensional and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the document's structural information. Finally, by analyzing the R_{NX} curve, we evaluate the effectiveness of the dimensionality reduction method in preserving structured information. If the overlap is high, it indicates that the dimensionality reduction process has well preserved the document's structural information. Otherwise, the dimensionality reduction method may need to be adjusted to better preserve the document's structural information.

[0115] The loss function is the so-called sorted list optimization method, which is the predicted score of all candidate power grid enterprise case documents and the true relevance label y∈ of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞:

[0116]

[0117] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

[0118] The above embodiments are only preferred implementations of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and equivalent substitutions can be made without departing from the principles of the present invention. These technical solutions after improvements and equivalent substitutions to the claims of the present invention all fall within the scope of protection of the present invention.

Claims

1. A case retrieval method, characterized in that: The method comprises the following steps: Use the pre-trained language model BERT to encode the given user retrieval query and obtain the vector representation Q. The enterprise case documents are encoded using a stack-based encoding algorithm to obtain a vector representation D = {d1, d2, ..., d n }, The obtained case document vector representation D is reduced in dimension using a case document vector dimensionality reduction algorithm based on structured information preservation to obtain a low-dimensional case document vector representation D′. A re-ranking method based on the case document and user retrieval matching scores is designed for the obtained user retrieval vector Q and the obtained case document vector D′, and the retrieval results are returned. The re-ranking method based on the matching scores between case documents and user retrievals is as follows: the user retrieval vector Q and the dimensionally reduced enterprise case document vector D′ are used as inputs. The scoring function τ(·) is used to calculate the matching score between each enterprise case document and the user retrieval. During the calculation process, a learnable linear projection function is used to adjust the document vector, potentially changing the dimension D′ of the document vector to adapt to different scoring functions while matching the dimension of the retrieval vector Q. A stack-based encoding algorithm is used to encode corporate case documents. For corporate case documents, a transformer network is used to encode and obtain a hidden state sequence. Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts; Aiming at the dimensionality reduction algorithm of case document vectors that retains structured information, a vector dimensionality reduction evaluation scheme oriented to structure preservation is designed; Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows: First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined. Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points. Afterwards, plot R according to different neighborhood sizes. NX The curve quantifies the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document. Finally, by NX The analysis of the curve is used to evaluate the effectiveness of the dimensionality reduction method in retaining structured information. If the overlap is high, it means that the dimensionality reduction process has better retained the structural information of the document. Otherwise, the dimensionality reduction method needs to be adjusted to better retain the structured information of the document.

2. The case retrieval method according to claim 1, characterized in that: The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After BERT pre-training, the classification identifier will learn the representation information of the user search query, represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

3. The case retrieval method according to claim 2, characterized in that: When performing dimensionality reduction on the case document vector, a deep learning model based on variational autoencoders and random quadruple loss is trained by minimizing the group loss and the prior KL divergence. The group loss adopts the concept of quadruple loss function, and the quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point. The Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows: Where n is the number of points in each group, The group normalized distance difference between the high-dimensional input point and the low-dimensional vector point should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows: The model is trained on shuffled batches of input data using the adaptive moment estimation optimizer Adam. Points are dynamically grouped in each batch, where n is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost value of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

4. The case retrieval method according to claim 1, wherein: The matching score of each enterprise case document and the user search is as follows. The loss function is called the sorted list optimization method. This loss function is the predicted score of all candidate power grid enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞: Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

5. A case search device, characterized in that: For implementing the case retrieval method according to any one of claims 1 to 4, the retrieval device comprises: The encoding module is used to encode the given user search query using the pre-trained language model BERT to obtain the vector representation Q, and to encode the enterprise case document using a stack-based encoding algorithm to obtain the vector representation D = {d1, d2, ..., d n }, the obtained case document vector representation D is represented by a dimensionality reduction algorithm based on the case document vector that retains structured information, and the case document vector is represented by a low-dimensional case document vector representation D′. A re-ranking module is used to design a re-ranking method based on the matching scores between the case documents and the user retrieval vector Q and the case document vector D′, and return the retrieval results; Among them, the rearrangement method based on the matching scores of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are taken as input, and the scoring function τ(·) is used to calculate the matching score of each enterprise case document and user retrieval respectively. During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D′ of the document vector to adapt to different scoring functions, while matching the dimension of the retrieval vector Q.

6. The case search device according to claim 5, characterized in that: The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After BERT pre-training, the classification identifier will learn the representation information of the user search query, represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

7. The case search device according to claim 6, characterized in that: A stack-based encoding algorithm is used to encode corporate case documents. For corporate case documents, a transformer network is used to encode and obtain a hidden state sequence.

8. The case search device according to claim 7, characterized in that: Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts; The deep learning model based on variational autoencoder and random quadruple loss is trained by minimizing group loss and prior KL divergence. The group loss adopts the concept of quadruple loss function, and the quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point, and the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ. ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows: Where n is the number of points in each group, The group normalized distance difference between the high-dimensional input point and the low-dimensional vector point should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows: The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are dynamically grouped in each batch, where n is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost value of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

9. The case search device according to claim 8, characterized in that: For the proposed case document vector dimensionality reduction algorithm based on structured information preservation, a vector dimensionality reduction evaluation scheme oriented to structure preservation is designed; Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows: First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined. Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to N-1, where N is the total number of samples, and the average is taken for all reference points. Afterwards, plot R according to different neighborhood sizes. NX The curve quantifies the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document. Finally, by NX The analysis of the curves can be used to evaluate the effectiveness of the dimensionality reduction method in retaining structured information. If the overlap is high, it means that the dimensionality reduction process has better retained the structural information of the document. Otherwise, the dimensionality reduction method may need to be adjusted to better retain the structured information of the document.

10. The case search device according to claim 6, characterized in that The matching score of each enterprise case document and the user search is as follows. The loss function is called the sorted list optimization method. This loss function is the predicted score of all candidate power grid enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞: Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

Images