Case retrieval method

Abstract

The invention discloses a case retrieval method, which comprises the following steps of: firstly, encoding an enterprise case document through a stack-based encoding method, and introducing a stack simulation layer to model a recursive state of a long text on the basis of a Transform encoder; and then a dimensionality reduction algorithm of a case document vector based on structured information retention is provided, a deep learning model based on a variational auto-encoder and random quadruple loss is used to carry out dimensionality reduction representation on the case document vector, and the structured information of the case document is effectively retained. Meanwhile, a vector dimension reduction evaluation scheme oriented to structure reservation is designed. Finally, in order to improve the accuracy of user retrieval, a reordering method based on case documents and retrieval matching scores is provided, the context information of other sorted candidate documents is considered in the method, and the effect of a retrieval model is remarkably enhanced by utilizing a lightweight reordering framework of contexts.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a case retrieval method based on stack coding and structured information retention. Background Art

[0002] As the business of power grid enterprises continues to expand and become more complex, case management has become an indispensable part of their operations. Power grid enterprises are involved in a wide range of cases, including contract disputes, intellectual property disputes, labor disputes, environmental protection lawsuits, etc. In order to effectively deal with the above legal affairs, power grid enterprises need an efficient case retrieval method to ensure that they can obtain relevant case information and materials in a timely manner when facing legal risks and disputes. However, traditional case retrieval methods are often inefficient and cannot fully utilize modern technical means to meet the needs of enterprises.

[0003] Traditional case retrieval methods usually rely on keyword matching and simple text retrieval, which is difficult to handle complex legal texts and case structure information. This method not only has low retrieval accuracy, but also tends to ignore important information implicit in the case. With the development of natural language processing technology, existing methods usually use models such as Transformer to learn the feature representation of complex case documents to enhance their understanding and analysis, and then use autoencoders to reduce the dimensionality of long document vectors to facilitate subsequent matching and scoring with user retrieval vectors, and finally output the case enterprise document list in descending order of scores. This type of method has improved the accuracy and intelligence of retrieval to a certain extent, but there are still three major problems: 1) Enterprise case documents are long, and traditional Transformer encoders use self-attention mechanisms and lack explicit recursive state tracking mechanisms. Therefore, the Transformer language model is difficult to capture long-tail recursive structures well, is less efficient when encoding long texts, and has weak generalization ability. 2) The autoencoder encodes high-dimensional input data into low-dimensional latent representations by sampling from the latent space distribution, and decodes it into a reconstruction of the input data. During the training process, the autoencoder uses an additional quantity to quantify the Kullback-Leibler (KL) divergence between the latent space distribution and the latent prior as an additional loss term to learn parameters by reducing the reconstruction loss. Since the encoder and decoder networks are probabilistic, the distribution in the latent space cannot be controlled by the reconstruction loss alone, which makes it easy to lose the global structural information of the vector. 3) Existing methods use contrastive learning to train dense retrieval models. Contrastive learning uses (query, positive sample document, negative sample document) triplets as training samples. When training the model, it is required that the similarity score between the query vector and the true relevant (positive sample) document vector is higher than the similarity score between the query and the negative sample document vector. This type of method only considers positive and negative sample document pairs, does not compare a given query with multiple documents, and ignores the information between documents. Therefore, there is an urgent need for a new solution to solve the above technical problems. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the length of enterprise case documents is long, and the traditional Transformer encoder is inefficient in encoding long texts and has weak generalization ability. Using an automatic encoder for dimensionality reduction cannot control the distribution in the latent space, so it is easy to lose the global structural information of the vector. Contrastive learning is used to train a dense retrieval model, only considering positive and negative sample document pairs, without comparing a given query with multiple documents, and ignoring the information between documents, providing a case retrieval method based on stack encoding and structured information retention.

[0005] The technical solution adopted by the present invention to solve its technical problems is: a case retrieval method based on stack encoding and structured information retention. For a given user retrieval query, this method first encodes all corporate case documents through a stack-based encoding method. On the basis of the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts. Then, a dimensionality reduction method for case document vectors based on structured information retention is proposed. A deep learning model based on variational autoencoder and random quadruple loss is used to reduce the dimension of case document vectors, effectively retaining the structural information of case documents. At the same time, a vector dimensionality reduction evaluation scheme for structure retention is designed. Finally, in order to improve the accuracy of user retrieval, a re-ranking method based on case documents and retrieval matching scores is proposed. This method takes into account the contextual information of other ranked candidate documents, and uses a lightweight re-ranking framework of context to significantly enhance the effect of the retrieval model.

[0006] The case retrieval method based on stack coding and structured information retention of the present invention comprises the following steps:

[0007] The pre-trained language model BERT is used to encode the given user retrieval query to obtain the vector representation Q.

[0009] A stack-based encoding algorithm is used to encode all corporate case documents to obtain a vector representation D =

[0010] {d1,d2,...,d n};

[0011] The obtained case document vector representation D is represented by a dimension reduction algorithm based on the case document vector that retains structured information to reduce the dimension of the case document vector and obtain a low-dimensional case document vector representation D. ′ , to improve the efficiency of user retrieval and document matching;

[0012] For the proposed case document vector dimension reduction algorithm based on structured information preservation, a vector dimension reduction evaluation scheme oriented to structure preservation is designed;

[0013] A re-ranking algorithm based on the case document and user search matching scores is used for the obtained user search vector Q and the obtained case document vector D′ to improve the accuracy of user search, and the case document list is output in descending order of scores to return the search results.

[0014] As a further improvement of the present invention, a pre-trained language model BERT is used to encode a given user search query, and the input form is [CLS]query[SEP]. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by a vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0015] As a further improvement of the present invention, for a certain enterprise case document, in the process of encoding using Transformer, like the ordinary self-attention mechanism, the hidden state sequence is obtained Transformer language models are a class of neural sequence models that use a multi-head self-attention mechanism to obtain contextualized representations of tokens in a sequence and then use these representations to predict the next token.

[0016] As a further improvement of the present invention, a stack-based encoding method is used to encode all corporate case documents. Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts to achieve accurate encoding of corporate case documents. The stack simulation layer proposed by the present invention uses a stack band to simulate a push-down automaton that performs shift / reduction operations when a word is predicted. The content of the stack band encodes the recursive state by tracking the depth to which each word is reduced in the stack.

[0017] As a further improvement of the present invention, an algorithm is designed to automatically update the stack. When predicting the next word, the Transformer with the stack simulation layer selects a component to update the stack. This is achieved by selecting a word to be reduced from the candidate word, which is a probabilistic additional decision obtained by performing an attention operation on all word-units. When the attention score is finally calculated, for each word-unit, the present invention maps the content of the stack to the depth vector of each layer. These depth vectors are added to the attention key, resulting in a local additive modulation of the attention score.

[0018] As a further improvement of the present invention, a dimensionality reduction method for case document vectors based on structured information preservation is designed, which adopts the concept of structure preservation at the local and global levels, creates a deep learning model based on variational autoencoders and random quadruple losses, and reduces the dimensionality of case document vectors. The model only contains a variational encoder and sampler (no decoder), and the model uses a "group loss" function to create vectors. Compared with traditional variational autoencoders, its global structure is less distorted, while maintaining the flexibility of the model's parameterization and architecture.

[0019] As a further improvement of the present invention, a deep learning model based on a variational autoencoder and a random quadruple loss is trained by minimizing the group loss and the prior KL divergence. The group loss adopts the concept of a quadruple loss function, and the quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable cost function of each randomly selected quadruple point. The present invention denotes the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j as ρ ij and d ij To calculate the group normalized distance between two points in the same group (for quartets, quintuples, sextuples, etc.), the present invention uses all pairwise distances within the group. For high-dimensional and low-dimensional points, the present invention formulates the group normalized distance formulas as follows:

[0020]

[0021]

[0022] Here, n is the number of points in each group.

[0023] The group normalized distance difference between the high-dimensional input points and the low-dimensional vector points should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0024]

[0025] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are grouped dynamically in each batch, and the size of the group (n) is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost of the group with i as the first point, and the group loss term for each batch is calculated by averaging over the entire batch.

[0026] As a further improvement of the present invention, the vector dimension reduction evaluation scheme for structure preservation uses a sorted neighborhood overlap curve R NX (Rank-based Neighborhood Overlap Curve) is used as a quality assessment indicator. This curve quantifies the overlap of the neighbor rankings of high-dimensional vectors and low-dimensional vectors relative to the reference point, covering all neighborhood sizes from 1 to (N-1) (sample size is N), and taking the average value for all reference points. In order to design a vector dimensionality reduction evaluation scheme for structure preservation, this paper proposes an R-based NX Quality assessment method of curves.

[0027] R NXThe curve quantifies the neighborhood relationship between the document vector in the high-dimensional space and the document vector in the low-dimensional space after dimensionality reduction, thereby verifying whether the structural information of the original document is effectively maintained during the dimensionality reduction process. The specific design process is as follows:

[0028] First, the high-dimensional case document vectors are compared with the low-dimensional case document vectors after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined.

[0029] Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points.

[0030] After that, we plot R according to different neighborhood sizes. NX} curve. This curve quantifies the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document.

[0031] The main purpose of the designed structure-preserving vector dimensionality reduction evaluation scheme is to verify and ensure the effectiveness of the dimensionality reduction method in preserving the structural information of case documents. Specifically, the evaluation scheme uses R NX The curve quantitatively analyzes the neighborhood overlap of document vectors before and after dimensionality reduction, ensuring that the dimensionality reduction operation retains the relative relationship and structural characteristics between documents as much as possible when compressing document vectors.

[0032] Through a structure-preserving dimensionality reduction scheme, the effectiveness of the reduced case document vector in preserving structural information is evaluated. The scheme uses a sort-based neighborhood overlap curve R NX To quantify the neighborhood overlap between high-dimensional and low-dimensional vectors, ensure that the dimensionality reduction operation retains the structural information of the document as much as possible, thereby improving the accuracy of document matching and retrieval.

[0033] As a further improvement of the present invention, a rearrangement method based on the matching scores of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are used as input, and the matching score between each enterprise case document and the user retrieval is calculated using the scoring function τ(·). During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D of the document vector. ′ To adapt to different scoring functions and match the dimension of the retrieval vector Q. This method can adaptively handle different retrieval tasks and ensure that the retrieval relevance can be maximized when sorting the retrieval results.

[0034] Algorithm output: After using this re-ranking algorithm, the system will sort the case documents according to the calculated matching scores and output a list of case documents sorted in descending order of matching scores. Ultimately, users will see the case documents most relevant to their search intent at the top, thereby improving the accuracy and efficiency of the search.

[0035] As a further improvement of the present invention, in order to maximize the use of the joint scoring of all enterprise case documents retrieved by each user, the present invention uses a so-called sorted list optimization method (ListNet) as a loss function, which is a loss function that compares the predicted scores of all candidate enterprise case documents with the true relevance labels of the same group of candidate enterprise case documents provided by the dataset. KL divergence between the distributions (the relevance score for a positive document is a positive scalar, while the relevance score for a negative document or a document whose label is not well defined is set to -∞):

[0036]

[0037] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

[0038] Compared with the prior art, the present invention has the following advantages: 1) The existing method uses the Transformer encoder to encode the case document. The traditional Transformer encoder is inefficient when encoding long texts and has weak generalization ability. On the basis of the Transformer encoder, the present invention introduces a stack simulation layer to model the recursive state of long texts. At the same time, a vector dimensionality reduction evaluation scheme for structure preservation is designed. 2) The existing method uses an autoencoder to reduce the dimensionality of the learned document representation vector. This method cannot control the distribution in the latent space, so it is easy to lose the global structural information of the vector. The present invention proposes a dimensionality reduction method for case document vectors based on structured information preservation, using a deep learning model based on variational autoencoders and random quadruple losses to reduce the dimensionality of case document vectors, effectively retaining the structural information of case documents. At the same time, a vector dimensionality reduction evaluation scheme for structure preservation is designed. 3) The existing method uses contrastive learning to train intensive retrieval models, only considering positive and negative sample document pairs, without comparing a given query with multiple documents, and ignoring the information between documents. The present invention proposes a re-ranking method based on case documents and retrieval matching scores, taking into account the contextual information of other ranked candidate documents, and utilizing a lightweight re-ranking framework of context to significantly enhance the effect of the retrieval model.

[0039] Experimental analysis has proved that the case retrieval method based on stack coding and structured information retention proposed by this method has played an improved role in the case retrieval model of power grid enterprises, greatly improving the performance and accuracy of the retrieval model. In addition, the present invention improves the encoding ability of recursive structures by adding stack storage to the Transformer language model, which can better understand and generate language structures with complex recursive characteristics. This method does not need to calculate the k-nearest neighbor graph of the input data, which greatly reduces the computational complexity. Less data is required during training to achieve similar or better performance than the standard Transformer model. It does not change the output space of the underlying sequence model, nor does it impose hard constraints on the attention mechanism, so that the model can learn to use syntactic structures through gradient descent. The above characteristics make this method widely applicable in downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a basic process schematic diagram of the present invention;

[0041] Figure 2 It is a model framework diagram of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the embodiments and the accompanying drawings.

[0043] Example 1: See Figure 1 , Figure 2 In order to solve the above problems and improve the performance and accuracy of case retrieval for power grid enterprises, the present invention proposes a case retrieval method based on stack encoding and structured information retention. For problem one, based on the Transformer encoder, the present invention introduces a stack simulation layer to model the recursive state of long texts. This is a new self-attention layer that simulates the recursive state through a stack band. The simulation layer updates the stack band synchronously while predicting new words in an autoregressive manner, and then uses the stack band to gently modulate the attention to the word. For problem two, in order to retain the structured information of enterprise case documents, the present invention adopts the concept of structure preservation at the local and global levels, and creates a deep learning model based on a variational autoencoder and a random quadruple loss from the SQuadMDS algorithm to reduce the dimension of case document vectors. The model uses a "group loss" function to create vectors, which has less global structural distortion than traditional VAEs, while maintaining the flexibility of the model's parameterization and architecture. For problem three, the present invention designs a rearrangement method based on case document and retrieval matching scores. This method takes into account the contextual information of other ranked candidate documents and jointly scores a large number of negative samples, allowing the model to directly optimize information retrieval indicators and improve the performance and accuracy of user retrieval.

[0044] The case retrieval method based on stack coding and structured information retention of the present invention comprises the following steps:

[0045] 1) Use the pre-trained language model BERT to encode the given user search query and obtain the vector representation Q. BERT is based on a bidirectional Transformer structure and uses a masked language model to achieve integrated feature fusion. It can model polysemy and generate deep bidirectional language representations. Therefore, the present invention uses BERT encoding, and the specific input form is [CLS]query[SEP]. Specifically, it represents the beginning of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by the vector Q. The separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0046] 2) A stack-based encoding method is used to encode all enterprise case documents to obtain a vector representation D = {d1, d2, ..., d |D| The present invention designs a stack-based encoding method. On the basis of the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts to achieve accurate encoding of corporate case documents.

[0047] First, for a certain corporate case document, in the process of encoding using Transformer, like the ordinary self-attention mechanism, the hidden state sequence is obtained Transformer language models are a type of neural sequence model that uses a multi-head self-attention mechanism to obtain contextual representations of tokens in a sequence and then use these representations to predict the next token. Specifically, for a certain corporate case document, the input sequence X = {x1,…,x n}.set up is the hidden state of the i-th word in the k-th attention block. Then, the hidden vector representation of the i-th word is updated as:

[0048]

[0049] in is a learned matrix. FFN is a module that combines feedforward neural network, residual network and layer normalization. f is the fth self-attention head. Each attention head performs a weighted average on its input,

[0050]

[0051] where α ij is the attention weight assigned by the i-th word to the j-th word.

[0052] Then, the stack simulation layer proposed by the present invention uses a stack with S j ∈{0,k} k to simulate a pushdown automaton that performs shift / reduce operations as tokens are predicted. The contents of the stack encode the recursive state by tracking the depth to which each token is reduced in the stack. Specifically, when observing the prefix X ≤j ={x1, x2, ..., x j )After that, S j [k] = 0 means word x k is not reduced with any other tokens, and S j [k] = q represents x k appears in q reduction operations, resulting in the word x in the component k At depth q.

[0053] Next, the present invention updates the stack. Specifically, given the prefix X <j , the Transformer with stacked analog layers predicts the next token x j And the stack with S j-1 This is done by updating the candidate word {x1, x2, ..., x j} select a word t to be reduced j This selection is implemented by , where O is the final layer of the Transformer and is the new predicted word x j The vector representation of is obtained as This vector is a probabilistic additional decision resulting from the attention operation on all tokens.

[0054]

[0055] Among them, MLP(·) is a multi-layer perceptron, is a learned parameter matrix. The present invention uses these probabilities to select word t j =argmaxp(c|x ＜j ; S j-1 ) is reduced and the stack is updated by the following steps:

[0056] 1) Initialization and input: The state S at the previous moment j-1 , current index j, target t associated with j j And the current stack as input.

[0057] 2) Status update: First, update the current status S j Initialized to the same state as the previous state S j-1same.

[0058] 3) Create constituent elements: define a list named struct, which initially contains the current index j.

[0059] 4) Stack operation judgment: Check whether t is equal to j. If so, push the struct into the stack and the algorithm ends.

[0061] 5) Loop reduction processing: If t is not equal to j, enter the loop and perform reduction operations:

[0062] - Pop the top element Top from the stack.

[0063] -Update struct, adding the pop-up Top to the front of the struct.

[0064] -Update the state Sj, and for each index i in the struct, increase the value of Sj[i] by 1.

[0065] 6) Check the termination condition: After each loop, check whether the popped-up Top is equal to the target tj:

[0066] - If yes, end the loop.

[0067] -If not, continue the loop and pop the next top element of the stack.

[0068] 7) Push and end: Push the updated struct back to the stack to complete the update of the entire stack.

[0069] In this way, the algorithm continuously processes the contents of the stack, combining and reducing them according to the input goals until the end condition is met. This processing method enables the algorithm to dynamically adjust the state of the stack to adapt to the given structure and goals. Note that the present invention makes additional decisions about components by calculating the additional score of the rightmost word in the component.

[0070] Finally, the attention score is calculated. For each word unit c∈{0, 1, ..., j}, the present invention converts S j The content is mapped to the depth vector of each layer These depth vectors are added to the attention key, producing a local additive modulation of the attention score,

[0071]

[0072] Since these logits are themselves part of a softmax and nonlinearity, the overall effect can be arbitrarily nonlinear.

[0073] 3) The case document vector obtained in step 2) is represented as D = {d1, d2, ..., d |D|The present invention designs a dimensionality reduction method for case document vectors based on structured information preservation, performs dimensionality reduction on case document vectors, and obtains a low-dimensional case document vector representation D′={d′1,d′2,.....,d′ |D|} to improve the efficiency of user retrieval and document matching.

[0074] Traditional dimensionality reduction methods usually use autoencoders to encode high-dimensional input data into low-dimensional latent representations by sampling from the latent space distribution and decode them into reconstructions of the input data. During training, the autoencoder uses an additional quantity to quantify the KL divergence between the latent space distribution and the latent prior as an additional loss term to learn parameters by reducing the reconstruction loss. Since the encoder and decoder networks are probabilistic, the distribution in the latent space cannot be controlled by the reconstruction loss alone, which makes it easy to lose the global structural information of the vector.

[0075] In order to preserve the structural information of enterprise case documents, this paper adopts the concept of structure preservation at the local and global levels, and creates a deep learning model based on variational autoencoders and SQuadMDS from the random quadruple multidimensional scaling algorithm to reduce the dimension of case document vectors. The model is created using a "group loss" function, which has less global structural distortion than traditional VAEs while maintaining the flexibility of the model's parameterization and architecture.

[0076] like Figure 2 As shown in the dimensionality reduction module in , the model only contains a variational encoder and sampler (no decoder), and minimizes the group loss and the prior KL divergence during training. The group loss adopts the concept of a quadruple loss function, and calculates the quadruple normalized distance between the original point and the vector point from the SQuadMDS algorithm. The normalized distance is used to calculate the differentiable cost function of each randomly selected quadruple point. The present invention denotes the Euclidean distance between any high-dimensional input point i or low-dimensional vector point j as ρ ij and d ij To calculate the group normalized distance between two points in the same group (for quadruple

[0077] For high-dimensional and low-dimensional points, the present invention formulates the group normalized distance formula as follows:

[0078]

[0079]

[0080] Here, n is the number of points in each group.

[0081] The group normalized distance difference between the high-dimensional input points and the low-dimensional vector points should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0082]

[0083] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are grouped dynamically in each batch, and the size of the group (n) is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost of the group with i as the first point, and the group loss term for each batch is calculated by averaging over the entire batch.

[0084] Therefore, the model imposes constraints on the distribution in the latent space instead of using reconstruction loss to calculate the weight update, which preserves the global structure information of the vector to some extent.

[0085] 4) For the case document vector dimensionality reduction method based on structural information preservation proposed in step 3), in order to prove the effectiveness of the dimensionality reduction method proposed in the present invention in preserving document structural information, the present invention designs a vector dimensionality reduction evaluation scheme for structure preservation. NX The curve is used as a quality assessment metric. This curve quantifies the overlap of the neighbor ordering of the high-dimensional vector with the low-dimensional vector relative to the reference point, covering all neighborhood sizes from 1 to (N-1) (with a sample size of N), and is averaged over all reference points.

[0086] To this end, the present invention represents the neighborhood ranking of point j (neighbor) relative to point i (reference point) in the high-dimensional vector and the low-dimensional vector as HD ij and LD ij The non-self-neighborhoods of high-dimensional vectors and low-dimensional vector points are expressed as and where E is the neighborhood size. For a dataset of size N, Q for a particular value of E NX The value is calculated as follows:

[0087]

[0088] To get the full Q NX The present invention calculates the score of E from 1 to (N-1). The results show that random vectors will lead to With Q NX In comparison, R NX Corrected for chance, the calculation formula is:

[0089]

[0090] The present invention uses vector RNX The structure preserving information is quantified by the area under the curve of the curve. Specifically, the local structure preserving information is the area under the curve of the curve with the neighborhood size E rescaled by the logarithm lnE so as to give more weight to the local neighborhood while not setting a hard boundary between local and global. In addition, the global structure preserving information is the area under the line with E scaled linearly so as not to emphasize the local neighborhood of the reference point. In both cases, the higher the structure preserving information score, the better.

[0091] 5) For the user search vector Q obtained in step 1) and the enterprise case document vector D′ obtained in step 3), a rearrangement method based on the matching score between the enterprise case document and the user search is proposed to improve the accuracy of the user search. The case document list is output in descending order of the score and the search result is returned.

[0092] Specifically, Figure 2 As shown in the matching scoring module in , the user retrieval vector Q and the reduced-dimensional enterprise case document vector D′ are taken as input, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and the user retrieval. During the calculation process, the use of learnable linear projection can potentially change the dimension D′ of the document vector

[0093] To adapt to different scoring functions and match the dimension of the search vector Q. The specific calculation formula is as follows:

[0094]

[0095] Among them, θ s are the parameters of the scoring function. Although a variety of functions can be used as scoring functions, the present invention only utilizes a simple inner product. In addition to computational efficiency, the main advantage of the above function is that it can be used directly for dense retrieval (single stage) using a fine-tuned user retrieval encoder via a fast approximate nearest neighbor search. In contrast, the non-linear scoring module only allows the model to be used in a two-stage retrieval setting (candidate retrieval followed by re-ranking).

[0096] A key difference between the re-ranking method based on the matching scores of enterprise case documents and user retrieval proposed in the present invention and all dense retrieval methods is that for each query, in addition to k positive (real) documents, the model is also trained to jointly score the top |D|-k candidate documents retrieved by a certain candidate retrieval method. The present invention notes that the candidate retrieval method does not need to be the same as the basic method that provides the document vector D′ and the retrieval vector Q encoder. Therefore, it is allowed to utilize methods with different characteristics (for example, methods with higher recall or precision, or methods based on vocabulary overlap / sparse representation, such as BM25).

[0097] In order to maximize the use of the joint scoring of |D| enterprise case documents retrieved by each user, the present invention uses the so-called ranked list optimization method (ListNet) as the loss function, which is the predicted score of |D| candidate enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. KL divergence between the distributions (the relevance score for a positive document is a positive scalar, while the relevance score for a negative document or a document whose label is not well defined is set to -∞):

[0098]

[0099] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function. Different from the existing dense retrieval methods, the present invention combines the joint scoring of user retrieval and enterprise case documents with KL divergence loss for the first time. This combination establishes and utilizes context for each user retrieval, further improving the performance and accuracy of retrieval. For data sets containing multiple retrieval document labels, especially labels defined across multiple relevance levels, the expected benefits of the present invention will be greater.

[0100] Embodiment 2: A case search device, comprising:

[0101] The encoding module is used to encode the given user search query using the pre-trained language model BERT to obtain the vector representation Q, and to encode the enterprise case document using a stack-based encoding algorithm to obtain the vector representation D = {d1, d2, ..., d n}, for the obtained case document vector representation D, the case document vector is reduced in dimension through the case document vector dimensionality reduction algorithm based on structured information retention, and a low-dimensional case document vector representation D′ is obtained. The dimensionality reduction module is used to design a vector dimensionality reduction evaluation scheme for structure retention based on the proposed case document vector dimensionality reduction algorithm based on structured information retention;

[0102] The rearrangement module is used to obtain the user search vector Q and the case document vector D ′ ,Design a re-ranking method based on the matching scores between case documents and user retrieval, and return the retrieval results;

[0103] The rearrangement method based on the matching scores of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are used as input, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and the user retrieval. During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D of the document vector. ′To adapt to different scoring functions and match the dimension of the retrieval vector Q.

[0104] The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

[0105] The stack-based encoding algorithm is used to encode the enterprise case documents. For the enterprise case documents, the transformer network Transformer is used to encode the hidden state sequence. Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts. The deep learning model based on the variational autoencoder and random quadruple loss is trained by minimizing the group loss and the prior KL divergence. The group loss adopts the concept of the quadruple loss function. The quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point. The Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows:

[0106]

[0107]

[0108] Where n is the number of points in each group,

[0109] The group normalized distance difference between the high-dimensional input points and the low-dimensional vector points should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows:

[0110]

[0111] The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are grouped dynamically in each batch. The size of the group (n) is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost value of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

[0112] For the proposed case document vector dimension reduction algorithm based on structured information preservation, a vector dimension reduction evaluation scheme oriented to structure preservation is designed;

[0113] Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows:

[0114] First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined.

[0115] Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points.

[0116] After that, we plot R according to different neighborhood sizes. NX The R_{NX} curve is used to quantify the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively retains the structural information of the document. Finally, the R_{NX} curve is used to evaluate the effectiveness of the dimensionality reduction method in retaining structural information. If the overlap is high, it means that the dimensionality reduction process has better preserved the structural information of the document. Otherwise, the dimensionality reduction method may need to be adjusted to better preserve the structural information of the document.

[0117] The so-called sorted list optimization method is used as the loss function, which is the predicted score of all candidate power grid enterprise case documents and the true relevance label y∈ of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞:

[0118]

[0119] Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

[0120] The above embodiments are only preferred implementation modes of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and equivalent substitutions without departing from the principles of the present invention. These technical solutions after improvements and equivalent substitutions to the claims of the present invention all fall within the protection scope of the present invention.

Claims

1. A case retrieval method, characterized in that: The method comprises the following steps: The pre-trained language model BERT is used to encode the given user search query to obtain the vector representation Q. The stack-based encoding algorithm is used to encode the enterprise case document to obtain the vector representation D = {d1, d2, ..., d n }, the obtained case document vector representation D is represented by a dimension reduction algorithm based on the case document vector with structured information retention, and the case document vector is represented by a low-dimensional case document vector representation D′. For the obtained user search vector Q and the obtained case document vector D′, a rearrangement method based on the matching scores of case documents and user searches is designed, and the search results are returned; Among them, the rearrangement method based on the matching score of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are taken as input, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and user retrieval respectively. During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D′ of the document vector to adapt to different scoring functions, while matching the dimension of the retrieval vector Q.

2. The case retrieval method according to claim 1, characterized in that: The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

3. The case retrieval method according to claim 1, characterized in that: The stack-based encoding algorithm is used to encode the enterprise case documents. For the enterprise case documents, the transformer network Transformer is used to encode the hidden state sequence.

4. The case retrieval method according to claim 3, characterized in that: Based on the Transformer encoder, a stack simulation layer is introduced to model the recurrent state of long texts.

5. The case retrieval method according to claim 4, characterized in that: The deep learning model based on variational autoencoder and random quadruple loss is trained by minimizing group loss and prior KL divergence. The group loss adopts the concept of quadruple loss function. The quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point. The Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows: Where n is the number of points in each group, The group normalized distance difference between the high-dimensional input points and the low-dimensional vector points should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows: The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are grouped dynamically in each batch. The size of the group (n) is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost value of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

6. The case retrieval method according to claim 1, characterized in that: For the proposed case document vector dimension reduction algorithm based on structured information preservation, a vector dimension reduction evaluation scheme oriented to structure preservation is designed; Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows: First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined. Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points. Afterwards, R is plotted according to different neighborhood sizes. NX } curve, which quantifies the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document. Finally, by analyzing the R_{NX} curve, we evaluate the effectiveness of the dimensionality reduction method in retaining structured information. If the overlap is high, it means that the dimensionality reduction process has better retained the structural information of the document. Otherwise, the dimensionality reduction method may need to be adjusted to better retain the structured information of the document.

7. The case retrieval method according to claim 1, characterized in that: The loss function is the so-called sorted list optimization method, which is the predicted score of all candidate power grid enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞: Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

8. A case search device, characterized in that: Used to implement the case retrieval method according to any one of claims 1 to 7, the retrieval device comprises: The encoding module is used to encode the given user search query using the pre-trained language model BERT to obtain the vector representation Q, and to encode the enterprise case document using a stack-based encoding algorithm to obtain the vector representation D = {d1, d2, ..., d n }, for the obtained case document vector representation D, the case document vector is reduced in dimension through a dimensionality reduction algorithm based on the case document vector that retains structured information, and a low-dimensional case document vector representation D′ is obtained; a rearrangement module is used to design a rearrangement method based on the case document and user retrieval matching score for the obtained user retrieval vector Q and the obtained case document vector D′, and return the retrieval result; Among them, the rearrangement method based on the matching score of case documents and user retrieval is as follows: the user retrieval vector Q and the enterprise case document vector D′ after dimensionality reduction are taken as input, and the scoring function τ(·) is used to calculate the matching score between each enterprise case document and user retrieval respectively. During the calculation process, the document vector is adjusted using a learnable linear projection function, potentially changing the dimension D′ of the document vector to adapt to different scoring functions, while matching the dimension of the retrieval vector Q.

9. The case search device according to claim 8, characterized in that: The pre-trained language model BERT is used to encode a given user search query. Specifically, a specific classification identifier [CLS] is inserted at the beginning to indicate the start of the input sentence. After the BERT pre-training process, the classification identifier will learn the representation information of the user search query, which is represented by the vector Q. A separator [SEP] is inserted at the end to indicate the end of the input sentence.

10. The case search device according to claim 9, characterized in that: The stack-based encoding algorithm is used to encode the enterprise case documents. For the enterprise case documents, the transformer network Transformer is used to encode the hidden state sequence.

11. The case search device according to claim 10, characterized in that: Based on the Transformer encoder, a stack simulation layer is introduced to model the recursive state of long texts; The deep learning model based on variational autoencoder and random quadruple loss is trained by minimizing group loss and prior KL divergence. The group loss adopts the concept of quadruple loss function. The quadruple normalized distance between the original point and the vector point is calculated from the random quadruple multidimensional scaling algorithm SQuadMDS. The normalized distance is used to calculate the differentiable loss function of each randomly selected quadruple point. The Euclidean distance between any high-dimensional input point i or low-dimensional vector point j is denoted as ρ ij and d ij , for high-dimensional and low-dimensional points, the group normalized distance formulas are formulated as follows: Where n is the number of points in each group, The group normalized distance difference between the high-dimensional input points and the low-dimensional vector points should be minimized to calculate the cost function of the group. The group normalized distance difference is calculated as follows: The model is trained on shuffled batches of input data using an adaptive moment estimation optimizer (Adam). Points are grouped dynamically in each batch. The size of the group (n) is specified as a hyperparameter. The group loss value for each point i in the training batch is specified as the cost value of the group with i as the first point. The group loss term for each batch is calculated by taking the average over the entire batch.

12. The case search device according to claim 8, characterized in that: For the proposed case document vector dimension reduction algorithm based on structured information preservation, a vector dimension reduction evaluation scheme oriented to structure preservation is designed; Evaluation of structure-preserving vector dimensionality reduction schemes using R NX The curve is used as a quality evaluation indicator, and the specific design process is as follows: First, the high-dimensional case document vector is compared with the low-dimensional case document vector after dimensionality reduction. For each document vector, the vector representation of each document is selected as a reference point and its neighborhood is determined. Second, for each document vector, the overlap between its high-dimensional space and the neighbor ranking in the low-dimensional space is calculated, covering different cases from neighborhood size 1 to (N-1), where N is the total number of samples, and the average is taken for all reference points. Afterwards, R is plotted according to different neighborhood sizes. NX } curve, which quantifies the overlap between high-dimensional vectors and low-dimensional vectors at different neighborhood sizes, reflecting whether the dimensionality reduction operation effectively preserves the structural information of the document. Finally, by analyzing the R_{NX} curve, we evaluate the effectiveness of the dimensionality reduction method in retaining structured information. If the overlap is high, it means that the dimensionality reduction process has better retained the structural information of the document. Otherwise, the dimensionality reduction method may need to be adjusted to better retain the structured information of the document.

13. The case search device according to claim 8, characterized in that: The loss function is the so-called sorted list optimization method, which is the predicted score of all candidate power grid enterprise case documents and the true relevance label of the same set of candidate enterprise case documents provided by the dataset. The KL divergence between the distributions, the relevance score of a positive document is a positive scalar, and the relevance score of a negative document or a document whose label is not well defined is set to -∞: Among them, D KL (·) is the KL divergence calculation function, and σ(·) represents the sigmoid function.

Images