Document retrieval method and system, storage medium and electronic equipment

Abstract

The invention discloses a document retrieval method and system, a storage medium and electronic equipment. The method comprises the steps of obtaining a target query text input by a target object; a pre-trained generative document retrieval model is utilized to retrieve a target document identification lexical element sequence with the highest semantic relevance with the target query text from a target database, and the target database comprises a plurality of documents and document identification lexical element sequences corresponding to the documents; the generative document retrieval model is obtained by training the sequence generation model by using a comparative learning technology and a progressive learning strategy; and feeding back a target document corresponding to the target document identification lexical element sequence to the target object. According to the method and the device, the technical problem that the related retrieval model cannot quickly and accurately retrieve the document related to the query text semantics in the complex corpus is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a document retrieval method and system, a storage medium and an electronic device. BACKGROUND

[0002] In the current field of document retrieval, especially for efficient retrieval of large-scale unstructured data, traditional retrieval methods such as sparse retrieval and standard dense retrieval face significant technical challenges. Among them:

[0003] Sparse retrieval relies on keyword matching and statistical models, and its main advantage is fast processing speed and easy implementation, but its limitations are also very obvious, namely the lack of deep semantic understanding of documents. This means that when faced with documents containing similar keywords but different semantics, sparse retrieval methods are prone to misleading retrieval results, reducing retrieval accuracy.

[0004] And dense retrieval uses neural network encoding to capture semantic information of text to make up for the lack of semantic understanding of sparse retrieval to some extent, especially in capturing deep semantic relationships between texts. However, dense retrieval methods also have limitations, mainly including high computational cost, especially when processing large-scale document collections, which requires pre-computing and maintaining a large vector index, which not only consumes a lot of storage resources, but also may affect the timeliness of retrieval due to low index update frequency. In addition, in order to train a high-performance dense retrieval model, high-quality negative examples are often needed, and improper negative example selection can cause bias in the model training process, affecting the model's generalization ability and discrimination ability, especially when faced with high-similarity competitive documents, the model is easy to confuse, reducing retrieval performance.

[0005] Therefore, the problems of the above two retrieval methods in processing large-scale document collections limit their retrieval accuracy and efficiency on complex corpora, resulting in poor retrieval accuracy and robustness. SUMMARY

[0006] The embodiments of the present application provide a document retrieval method and system, a storage medium and an electronic device to at least solve the technical problem that related retrieval models cannot quickly and accurately retrieve documents related to the semantics of the query text in complex corpora.

[0007] According to an aspect of an embodiment of the present application, a document retrieval method is provided, including: obtaining a target query text input by a target object; retrieving, by using a pre-trained generative document retrieval model, a target document token sequence with the highest semantic relevance to the target query text from a target database, wherein the target database includes a plurality of documents and a document token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model by using a contrast learning technique and a progressive learning strategy; and feeding back a target document corresponding to the target document token sequence to the target object.

[0008] Optionally, the training process of the generative document retrieval model includes: obtaining a plurality of sets of training sample data, wherein each set of training sample data includes: a query text, a document token sequence corresponding to a positive sample document with a semantic relevance higher than a first threshold value to the query text, and a document token sequence corresponding to a hard negative sample document with a semantic relevance lower than a second threshold value to the query text, and the first threshold value is much higher than the second threshold value; determining a sequence generation model, wherein the sequence generation model at least includes an encoder, a decoder, and an output layer; and iteratively training the sequence generation model by using a contrast learning technique and a progressive learning strategy based on the plurality of sets of training sample data to obtain the generative document retrieval model.

[0009] Optionally, obtaining the plurality of sets of training sample data includes: determining a plurality of documents in the target database, and performing feature extraction on each document to obtain a corresponding document embedding vector; randomly sampling a plurality of sets of continuous word groups from each document, and taking the plurality of sets of continuous word groups in each document as a plurality of query texts; clustering the document embedding vectors of each document by using a hierarchical k-means algorithm to obtain a clustering tree, and sequentially determining a clustering path of each document from bottom to top of the clustering tree, and determining a document token sequence of each document according to the clustering path; and for each document, the plurality of sets of training sample data are composed of the plurality of query texts corresponding to the document, the document token sequence of the document as a positive sample document, and the document token sequences of other documents as hard negative sample documents.

[0010] Optionally, the hierarchical k-means algorithm is used to cluster the document embedding vectors to obtain a clustering tree, including: taking the document embedding vectors of the plurality of documents as initial clustering clusters, and randomly selecting k document embedding vectors from the initial clustering clusters as k initial clustering centers; using the k-means algorithm to assign the plurality of document embedding vectors in the initial clustering clusters to the nearest initial clustering center to obtain k sub-clustering clusters, and re-determine the clustering centers of the k sub-clustering clusters; traversing each sub-clustering cluster to determine the number of document embedding vectors contained in the current sub-clustering cluster, and in the case that the number of document embedding vectors contained in the current sub-clustering cluster is higher than a preset node number threshold, using the k-means algorithm to recursively cluster the plurality of document embedding vectors in the sub-clustering cluster until the number of document embedding vectors contained in the recursively clustered sub-clustering cluster is not higher than the node number threshold; taking the clustering centers of the initial clustering clusters as root nodes, taking the clustering centers of the k sub-clustering clusters obtained in the first round of clustering as first-layer sub-nodes, taking the clustering centers of the recursively clustered sub-clustering clusters and the document embedding vectors contained in each sub-clustering cluster as deeper sub-nodes, and obtaining a multi-level clustering tree.

[0011] Optionally, the clustering paths of the document embedding vectors are sequentially determined from bottom to top in the clustering tree, and the document token sequences of the documents are determined according to the clustering paths, including: for each document embedding vector, sequentially determining the node sequence numbers corresponding to the clustering clusters to which the document embedding vector is assigned at each level clustering and the node sequence number corresponding to the clustering cluster to which the document embedding vector is finally assigned from the root node of the clustering tree; adding the node sequence numbers corresponding to the clustering clusters to which the document embedding vector is assigned at each level clustering to the product sum of the node number threshold and the layer number corresponding to the corresponding level clustering to form the document token sequence of the document.

[0012] Optionally, based on the plurality of sets of training sample data, the sequence generation model is iteratively trained by using a contrastive learning technique and a progressive learning strategy to obtain the generative document retrieval model, including: in the first number of iterative training rounds, using the sequence generation model to retrieve, from the target database, a plurality of documents with a semantic relevance to the query text higher than a third threshold value but lower than a first threshold value within a training batch corresponding to the first number of iterative training rounds as soft negative sample documents, and determining a document token sequence corresponding to each soft negative sample document, and reconstituting a plurality of sets of new training sample data from the plurality of sets of training sample data, the plurality of soft negative sample documents, and the corresponding document token sequences; based on the plurality of sets of new training sample data, the sequence generation model is subjected to contrastive learning to adjust model parameters of the sequence generation model, wherein the third threshold value is higher than the second threshold value; in the second number of iterative training rounds, using the sequence generation model to retrieve, from the target database, a first number of documents with a semantic relevance to the query text higher than a fourth threshold value but lower than the first threshold value within a training batch corresponding to the second number of iterative training rounds as soft negative sample documents, and determining a document token sequence corresponding to each soft negative sample document, and reconstituting a plurality of sets of new training sample data from the plurality of sets of training sample data, the first number of soft negative sample documents, and the corresponding document token sequences; based on the plurality of sets of new training sample data, the sequence generation model is subjected to contrastive learning to adjust model parameters of the sequence generation model, wherein the fourth threshold value is higher than the third threshold value; in the third number of iterative training rounds, using the sequence generation model to retrieve, from the target database, a second number of documents with a semantic relevance to the query text higher than the fourth threshold value but lower than the first threshold value within a training batch corresponding to the third number of iterative training rounds as soft negative sample documents, and determining a document token sequence corresponding to each soft negative sample document, and reconstituting a plurality of sets of new training sample data from the plurality of sets of training sample data, the second number of soft negative sample documents, and the corresponding document token sequences; based on the plurality of sets of new training sample data, the sequence generation model is subjected to contrastive learning to adjust model parameters of the sequence generation model until the model parameters converge, and a trained generative document retrieval model is obtained, wherein the second number is less than the first number.

[0013] Optionally, the sequence generation model is subjected to contrastive learning based on multiple sets of new training sample data to adjust model parameters of the sequence generation model, including: for each set of new training sample data, inputting query text in the new training sample data into an encoder in the sequence generation model to obtain query text embedding vectors output by the encoder; performing the following steps in a loop until a preset decoding termination condition is met to obtain a predicted document token sequence matching the query text in the new training sample data: in the i-th decoding process, inputting the query text embedding vector and the predicted document token sequence output in the previous i-1 decoding processes into a decoder in the sequence generation model to obtain a hidden state vector output by the decoder, wherein when i=1, the input of the decoder is the query text embedding vector and a preset initial document token; converting the hidden state vector into a predicted probability distribution by using an output layer in the sequence generation model, and selecting a predicted document token with the maximum probability as the input of the decoder in the i+1 decoding process according to the predicted probability distribution; constructing a loss function of the sequence generation model according to the predicted document token sequence matching the query text in the new training sample data, the document token sequence corresponding to the positive sample document, the document token sequence corresponding to the hard negative sample document, and the document token sequence corresponding to the soft negative sample document:

[0014]

[0015] wherein, represents the loss function, represents the prediction loss function, represents the encoder boundary loss function, represents the decoder contrastive loss function, represents the document-aware label mapping loss function, wherein the prediction loss function is expressed as:

[0016]

[0017] wherein, d k represents the k-th predicted document token in the predicted document token sequence matching the query text q, d <k represents a sequence composed of the first k document tokens in the predicted document token sequence matching the query text q, |d| represents the total number of tokens in the predicted document token sequence matching the query text q, p θ represents the model parameter, and p θ =Softmax(Decoder(Encoder(q),d <k )·W lm ), W lmThe output layer weight matrix is ​​represented by Encoder, the decoder by Decoder, and Softmax by the output layer activation function; the encoder boundary loss function is also represented by Softmax. Expression writing:

[0018]

[0019] In the formula, α and γ represent weighting coefficients, γ represents the preset boundary value, and K... pos The query text vector ε represents the query text q. q With the corresponding positive sample document embedding vector Vector similarity between them, K neg The query text vector ε represents the query text q. q With the corresponding soft negative sample document embedding vector Vector similarity between them, K rnd The query text vector ε represents the query text q. q With the corresponding hard negative sample document embedding vector Vector similarity between them; decoder contrastive loss function Expression writing:

[0020]

[0021] In the formula, They represent The value of the exponential function, and in Represents the query text vector ε q The embedding representation in the i-th decoding process, This represents the embedding representation of the positive sample document corresponding to the query text q during the i-th decoding process, where T represents the transpose operation. in Let represent the embedding representation of the soft negative sample document corresponding to the query text q during the i-th decoding process. in The document-aware label mapping loss function represents the embedding representation of the hard negative sample document corresponding to the query text q during the i-th decoding process. Expression writing:

[0022]

[0023] In the formula, S represents lmpos , The value of the exponential function, and in represents the label mapping representation of the positive sample document corresponding to the query text q in the i-th decoding process; the model parameters are adjusted based on the loss function through back propagation and gradient descent algorithm.

[0024] According to another aspect of the embodiments of the present application, a document retrieval system is further provided, comprising: an acquisition module configured to acquire a target query text input by a target object; a retrieval module configured to retrieve, by using a pre-trained generative document retrieval model, a target document token sequence with the highest semantic relevance to the target query text from a target database, wherein the target database comprises a plurality of documents and a document token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model by using a contrastive learning technique and a progressive learning strategy; and a feedback module configured to feed back a target document corresponding to the target document token sequence to the target object.

[0025] According to another aspect of the embodiments of the present application, a computer program product is further provided, comprising: a computer program, wherein the computer program is executed by a processor to implement the above-mentioned document retrieval method.

[0026] According to another aspect of the embodiments of the present application, an electronic device is further provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned document retrieval method by using the computer program.

[0027] In the embodiments of the present application, the generative document retrieval model obtained by training the sequence generation model by using the contrastive learning technique and the progressive learning strategy directly retrieves the target document token sequence with the highest semantic relevance and the best matching to the target query text from the target database. Through such end-to-end learning, the direct generation of query-to-document matching is realized, the retrieval efficiency and accuracy are greatly improved, and the external index maintenance is not needed. The target document corresponding to the target document token sequence retrieved is fed back to the target object, the fast and accurate document retrieval and delivery are realized, the purpose of improving the user retrieval experience and the system retrieval performance is achieved, and further the technical problem that the related retrieval model cannot quickly and accurately retrieve the document with semantic relevance to the query text in the complex corpus is solved. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0029] Figure 1 is a flow schematic diagram of an optional document retrieval method according to an embodiment of the present application;

[0030] Figure 2 is a structural schematic diagram of an optional document retrieval system according to an embodiment of the present application;

[0031] Figure 3 is a structural schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable persons skilled in the art to better understand the schemes of the present application, the technical schemes in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should belong to the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] In order to better understand the embodiments of the present application, first, some nouns or terms appearing in the description of the embodiments of the present application are translated and explained as follows:

[0035] Contrastive Learning: It is a special unsupervised learning method, which aims to learn data representation by maximizing the similarity between relevant samples and minimizing the similarity between irrelevant samples. Usually, a high degree of freedom, self-defined rule is used to generate positive and negative samples.

[0036] Progressive Learning Strategy: It is a learning strategy that gradually increases the learning task or learning difficulty, and its core idea is to gradually advance the learning task from simple to complex, from easy to difficult, so that the model can have better generalization ability and stability when facing complex tasks. In machine learning and deep learning, the progressive learning strategy is often used to deal with difficult tasks, domain adaptation, transfer learning, etc.

[0037] Sequence Generation Models: A class of deep learning models that focus on problems where both the input and output are sequences of data. They learn the temporal dependencies and patterns in the sequences to generate new sequences of data, commonly used in natural language processing, speech synthesis, and music composition. These models include Recurrent Neural Networks, Long Short-Term Memory networks, and Transformers.

[0038] Embodiment 1

[0039] According to the embodiments of the present application, a document retrieval method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0040] Figure 1 is a flowchart of a document retrieval method provided according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0041] Step S102, obtaining the target query text input by the target object.

[0042] Step S104, using a pre-trained generative document retrieval model to retrieve the target document token sequence most relevant to the target query text from the target database. Wherein, the generative document retrieval model is obtained by training the sequence generation model using the contrastive learning technique and the progressive learning strategy.

[0043] Step S106, feeding the target document corresponding to the target document token sequence to the target object.

[0044] Based on the scheme defined by the above steps S102 to S106, it can be known that in the embodiments of the present application, the generative document retrieval model obtained by training the sequence generation model using the contrastive learning technique and the progressive learning strategy directly retrieves the target document token sequence most relevant and matching to the target query text from the target database. Through such end-to-end learning, the direct generation of query to document matching is realized, which greatly improves the retrieval efficiency and accuracy, and achieves the purpose of no need for external index maintenance. The target document corresponding to the target document token sequence retrieved is fed to the target object, realizing fast and accurate document retrieval and delivery, and achieving the purpose of improving user retrieval experience and system retrieval performance.

[0045] The steps of the document retrieval method are described below in conjunction with the specific implementation process.

[0046] In the technical solution provided in step S102, the retrieval system can receive a query request from a target object (such as a user or an application program) through a user interface or other data interface, wherein the query request carries at least a target query text, which is usually a question or a set of keywords in natural language form; then, the retrieval system can perform preliminary processing on the received target query text, parse its format, remove unnecessary symbols and stop words, and perform basic text preprocessing (such as uniformizing case, normalizing numbers and special characters, etc.) to ensure the cleanliness and consistency of the text, and prepare it as input for the model; then, the system can convert the parsed target query text into a format suitable for processing by the machine learning model, such as word segmentation, stemming, and word vectorization, etc. These preprocessing steps help the model better understand the semantic features of the query text, and prepare for subsequent feature extraction and retrieval.

[0047] Then, the retrieval system calls the pre-trained generative document retrieval model to analyze the target query text to retrieve the target document identification token sequence with the highest semantic relevance to the target query text from the target database, and the specific flow implementation process is as follows:

[0048] First, the retrieval system inputs the target query text into the encoder in the generative document retrieval model, uses the multi-layer attention mechanism of the encoder to perform deep semantic analysis on the target query text, and extracts the target query text embedding vector that can represent the core information and context of the text.

[0049] Then, the decoder in the generative document retrieval model generates the target document identification token sequence with the highest semantic relevance to the target query text based on the target query text embedding vector generated by the encoder, wherein the generation at each step depends on the target query text embedding vector and the document identification token sequence generated in the previous step, to ensure that the generated document identification token sequence is not only related to the target query text, but also reflects the hierarchical structure of the document in the corpus.

[0050] Finally, the corresponding target document in the target database is located according to the position and value of each token in the target document identification token sequence.

[0051] Specifically, regarding the above-mentioned generative document retrieval model, its training process includes:

[0052] Step S1, obtaining a plurality of sets of training sample data.

[0053] Wherein, each group of training sample data includes: query text, document identification token sequence corresponding to positive sample document with high semantic relevance to the query text, and document identification token sequence corresponding to hard negative sample document with low semantic relevance to the query text, and the first threshold value is much higher than the second threshold value, to construct clear positive and negative sample comparison.

[0054] In the technical solution provided in the above step S1, regarding the acquisition of training sample data, the following steps can be implemented:

[0055] Step S11, determine a plurality of documents in the target database, and perform feature extraction on each document to obtain the corresponding document embedding vector. Regarding the document embedding vector, the pre-trained BERT model or other deep learning model can be used to convert each document in the target database into a dense vector representation to obtain the document embedding vector of each document.

[0056] Step S12, randomly sample multiple groups of continuous word groups from each document, and use the multiple groups of continuous word groups in each document as multiple query texts. Wherein, the word group length can be set to 16, 32, etc. to ensure that the key semantic fragments in the document can be covered. Using these continuous word groups as simulated query texts can enhance the learning of the model, so that the model can understand the content of the document from multiple perspectives from local to overall.

[0057] Step S13, use the hierarchical k-means algorithm to cluster the document embedding vectors of each document to obtain a clustering tree, and determine the clustering path of each document from bottom to top in the clustering tree, and determine the document identification token sequence of each document according to the clustering path. Wherein, each parent node in the clustering tree represents a cluster, and the child node corresponds to the documents contained in the cluster.

[0058] Specifically, regarding the clustering of the document embedding vector, it can be implemented in the following way:

[0059] First step: use the document embedding vectors of multiple documents as initial clustering clusters, and randomly select k document embedding vectors from the initial clustering clusters as k initial clustering centers.

[0060] Second step: use the k-means algorithm to assign the multiple document embedding vectors in the initial clustering cluster to the nearest initial clustering center to obtain k sub-clustering clusters, and re-determine the clustering centers of the k sub-clustering clusters.

[0061] Third step: traverse each sub-cluster, determine the number of document embedding vectors contained in the current sub-cluster, and if the number of document embedding vectors contained in the current sub-cluster is higher than the preset node number threshold (denoted as c), recursively cluster the multiple document embedding vectors in the sub-cluster using the k-means algorithm to form smaller sub-clusters, until the number of document embedding vectors contained in the recursively clustered sub-cluster is not higher than the node number threshold, thereby realizing hierarchical clustering of the documents. For the sub-cluster whose number of document embedding vectors is not higher than the node number threshold, the documents in the cluster can be sequentially numbered from 0 to c-1.

[0062] Fourth step: take the cluster center of the initial cluster as the root node, take the cluster centers of the k sub-clusters obtained in the first round of clustering as the sub-nodes of the first layer, and take the cluster centers of the recursively clustered sub-clusters and the document embedding vectors contained in each sub-cluster as deep sub-nodes, to obtain a multi-level clustering tree.

[0063] The multi-level clustering tree obtained by the above method organizes the documents into tree nodes with a root node, and provides a clear path for the generation of the document token sequence of each document, i.e., each document can be associated through the path between the root node and the leaf node of the clustering tree, where each path node represents a cluster at a different level in the clustering tree, and the final leaf node corresponds to a specific document, thereby mapping the document embedding vector of any document to a tree-like path identifier sequence.

[0064] In addition, the determination of the document token sequence of the document embedding vector can be realized in the following way: for each document embedding vector, start from the root node (r0) of the clustering tree and determine the node sequence number corresponding to the cluster to which the document embedding vector is assigned at each level of clustering and the node sequence number corresponding to the cluster to which the document embedding vector is finally assigned. This step can be understood as follows: during the recursive clustering process, whenever a document embedding vector is assigned to a new cluster (sub-node), record its clustering number i and determine the node sequence number (r i ) in the clustering tree of the new cluster. The node sequence number (r i ) reflects the specific position of the document in the clustering tree; then, add the node sequence number corresponding to the cluster to which the document embedding vector is assigned at each level of clustering to the product of the node number threshold and the number of levels corresponding to the corresponding level of clustering, to form the document token sequence of the document. Specifically, for each clustering in the clustering tree (i.e., the i-th layer of the tree), add the node sequence number (r i ) corresponding to the cluster to which the document embedding vector is assigned to the node number threshold (c) and the number of levels corresponding to the level of clustering (i) as the encoding result of the clustering index of this layer.

[0065] For example, if the document embedding vector of a certain document is in the hierarchical clustering process, its clustering path in the clustering tree is "root -> cluster 1 -> cluster 6 -> cluster 17" (i.e. the node number corresponding to the cluster to which the 1st layer is clustered is 1, the node number corresponding to the cluster to which the 21st layer is clustered is 6, and the node number corresponding to the cluster to which the 3rd layer is clustered is 17), then the path index of the document embedding vector in the clustering tree can be represented as 1-6-17; if the node number threshold is 30, using the above encoding method, each token in the path index can be uniquely encoded by position to convert [1, 6, 17] to [1+1*30, 6+2*30, 17+3*30] = [31, 66, 107]. Therefore, the embodiments of the present application construct a unique document identification token sequence (also known as a token sequence of docID) for each document by concatenating the path index from the root node of the clustering tree to the finally assigned leaf node.

[0066] In step S14, for each document, a plurality of training sample data groups are composed of the plurality of query texts corresponding to the document, the document identification token sequence of the positive sample document corresponding to the document, and the document identification token sequence of the other document selected from the target database as the hard negative sample document.

[0067] In addition, regarding the acquisition of training sample data, the query text input by the object history and the document identification token sequence of the positive sample document most semantically associated with the query text and the document identification token sequence of the hard negative sample document least semantically associated with the query text retrieved by manual search can also be composed into a plurality of training sample data groups.

[0068] In step S2, a sequence generation model is determined. The sequence generation model at least includes: an encoder for converting a query text into a semantic vector representation, a decoder for predicting the embedding representation of the next predicted token based on the output of the encoder, and an output layer for converting the hidden state vector of the decoder into the probability distribution of each token in a predetermined vocabulary.

[0069] In step S3, based on the plurality of training sample data groups, the sequence generation model is iteratively trained using a contrastive learning technique and a progressive learning strategy to obtain a generative document retrieval model.

[0070] The contrastive learning technique optimizes the model parameters by pulling the positive samples and pushing away the negative samples, thereby enhancing the model's ability to distinguish between semantically related and unrelated documents. The progressive learning strategy gradually adjusts the difficulty of the negative samples according to the model training progress, starting with easy negative samples and gradually introducing difficult samples that are closer to the positive samples, to avoid the model converging to a local optimal solution too early and improve the model's ability to distinguish between high similarity documents.

[0071] Therefore, in the technical solution provided in the above step S3, when iterative training is performed using the progressive learning strategy, it can be divided into the following three stages:

[0072] The first stage is an initial training stage (including a first number of iterative training rounds), and the training process includes:

[0073] First, the sequence generation model is used to retrieve, from the target database, a plurality of documents with a semantic relevance to the query text higher than a third threshold value but lower than a first threshold value in the training batch corresponding to the first number of iterative training rounds as soft negative sample documents, and determine the document token sequence corresponding to each soft negative sample document. The soft negative sample document refers to a document with a certain semantic relevance to the given query text, but the relevance degree is lower than that of the positive sample document, in other words, the soft negative sample document is a document set between the positive sample (highly relevant documents) and the hard negative sample (completely irrelevant documents). Therefore, the third threshold value is higher than the second threshold value, so as to ensure that the soft negative sample document has a certain similarity with the positive sample document, but is not sufficient to be regarded as a positive sample document.

[0074] Then, a plurality of groups of training sample data in the training batch corresponding to the first number of iterative training rounds, a plurality of soft negative sample documents and corresponding document token sequences are recombined to form a plurality of groups of new training sample data.

[0075] Based on the plurality of groups of new training sample data, the sequence generation model is subjected to contrastive learning to adjust the model parameters of the sequence generation model.

[0076] The second stage is a mid-term training stage (including a second number of iterative training rounds), and the training process includes:

[0077] The sequence generation model is used to retrieve, from the target database, a plurality of query text semantic relevance higher than the fourth threshold value but lower than the first threshold value in the training batch corresponding to the second number of iterative training rounds as soft negative sample documents, and determine the document token sequence corresponding to each soft negative sample document. Here, the fourth threshold value is higher than the third threshold value but lower than the first threshold value, so as to strictly screen out soft negative sample documents. The first number can be determined by probability, such as sampling negative sample documents from the Top5 documents output by the model according to a preset probability p as the model iteratively trains, and t represents the current iterative training round, and T1 represents the second number.

[0078] The multiple groups of new training sample data are reconstituted from the multiple groups of training sample data in the training batches corresponding to the first number of iterative training rounds and the second number of soft negative sample documents and the corresponding document identification token sequences.

[0079] The sequence generation model is subjected to comparative learning based on the multiple groups of new training sample data to adjust the model parameters of the sequence generation model, wherein the fourth threshold value is higher than the third threshold value.

[0080] The third stage: advanced training stage (including a third number of iterative training rounds), the training process includes:

[0081] The sequence generation model is used to retrieve, from the target database, a second number of documents with a higher semantic relevance to the multiple query texts than the fourth threshold value but lower than the first threshold value in the training batches corresponding to the third number of iterative training rounds as soft negative sample documents, and determine the document identification token sequences corresponding to each soft negative sample document. Here, the second number can also be determined by probability, for example, after the second number of iterative training rounds, the model can sample negative sample documents from the Top5 documents output by the model according to a probability of 0.75. Therefore, the second number is less than the first number.

[0082] The multiple groups of new training sample data are reconstituted from the multiple groups of training sample data in the training batches corresponding to the third number of iterative training rounds and the second number of soft negative sample documents and the corresponding document identification token sequences.

[0083] The sequence generation model is subjected to comparative learning based on the multiple groups of new training sample data to adjust the model parameters of the sequence generation model, until the model parameters converge, to obtain the trained generative document retrieval model.

[0084] In the iterative process of the above different stages, different threshold values are set to dynamically adjust the difficulty of the soft negative sample documents faced by the model in different stages, that is, documents with lower relevance are used as soft negative sample documents in the initial stage, and then the threshold value is gradually increased, and documents similar to the positive samples but not completely matched are introduced as soft negative sample documents, thereby helping the model gradually improve its ability to distinguish similar documents, while avoiding the model focusing on specific patterns too early and ignoring global information, reducing the risk of overfitting. In addition, the number of soft negative sample documents is increased at the initial node, and the model can learn the basic patterns and features faster, speeding up the training process, and as the training deepens, the number of soft negative sample documents is reduced, which can ensure that the model focuses more on optimizing the distinction of highly relevant documents in the advanced stage, improving the efficiency and pertinence of the training.

[0085] Further, in each of the above stages, based on the existing capability of the model, the sequence generation model can be subjected to contrastive learning based on multiple sets of new training sample data to adjust the model parameters of the sequence generation model according to the following steps:

[0086] Step 1: For each set of new training sample data, input the query text within the new training sample data into the encoder in the sequence generation model to obtain the query text embedding vector output by the encoder.

[0087] Step 2: The following steps are executed in a loop until a preset decoding termination condition is met (including: the length of the generated document representation token sequence reaches a preset standard length or reaches a certain termination condition such as the generation <eos>until the end of the token), obtaining a predicted document identification token sequence matching the query text in the new training sample data: in the i-th decoding process, inputting the query text embedding vector and the predicted document identification token sequence output by the previous i-1 decoding processes into the decoder in the sequence generation model to obtain the hidden state vector output by the decoder (which contains the context information and historical decoding information in the current decoding process), wherein when i = 1, the input of the decoder is the query text embedding vector and a preset initial document identification token (such as <sos>token, for starting the decoding process); converting the hidden state vector into a prediction probability distribution (which contains the prediction probability of each document identifier token in the preset vocabulary) by using the output layer in the sequence generation model, which means that the output layer maps the hidden state of the decoder to a probability space, in which each document identifier token in the preset vocabulary is assigned a probability value to represent the possibility of the document identifier token as the next predicted document identifier token; selecting the predicted document identifier token with the maximum probability as the input of the decoder in the i+1th decoding process according to the prediction probability distribution, wherein the predicted document identifier token with the maximum probability also constitutes part of the predicted document identifier token sequence.

[0088] Step 3: constructing the loss function of the sequence generation model as shown below according to the predicted document identifier token sequence matched by the query text in the new training sample data, the document identifier token sequence corresponding to the positive sample document, the document identifier token sequence corresponding to the hard negative sample document, and the document identifier token sequence corresponding to the soft negative sample document:

[0089]

[0090] In the formula, d represents the loss function, represents the prediction loss function, represents the encoder boundary loss function, represents the decoder contrast loss function, represents the document-aware label mapping loss function, wherein:

[0091] The expression of the prediction loss function is written as:

[0092]

[0093] In the formula, d k represents the kth predicted document identifier token in the predicted document identifier token sequence matched by the query text q, d <k represents the sequence composed of the first k document identifier tokens in the predicted document identifier token sequence matched by the query text q, |d| represents the total number of tokens in the predicted document identifier token sequence matched by the query text q, p θ represents the model parameter, and p θ = Softmax(Decoder(Encoder(q), d <k · W lm ), W lm represents the weight matrix of the output layer, Encoder represents the encoder, Decoder represents the decoder, and Softmax represents the activation function of the output layer;

[0094] Encoder boundary loss function It is defined based on the boundary loss between the query text and the corresponding positive sample document, hard negative sample document, and soft negative sample document. Therefore, the encoder boundary loss function... Expression writing:

[0095]

[0096] In the formula, α and γ represent weighting coefficients, with γ representing a preset boundary value (used to control the degree of separation between positive and negative sample documents). The query text vector ε represents the query text q. q With the corresponding positive sample document embedding vector Vector similarity between The query text vector ε represents the query text q. q With the corresponding soft negative sample document embedding vector Vector similarity between them The query text vector ε represents the query text q. q With the corresponding hard negative sample document embedding vector The vector similarity between them. Furthermore, ε q ε represents the average pooling output of the encoder on the query text q, hence ε q =Linear(AvgPool(Encoder(q))), where AvgPool represents the average pooling of all tokens corresponding to the query text q output by the encoder, resulting in a fixed-dimensional vector, while Linear represents the transformation of the average-pooled vector through a linear layer (fully connected layer) to map it to the target dimension. About The determination method is ε q The method of determination is the same, so it will not be explained further.

[0097] Decoder contrast loss function This is achieved by constructing an embedding representation of the decoder output and query text with their corresponding positive, hard, and soft negative sample documents, and then calculating the list loss through the dot product during each decoding process. Therefore, the decoder's contrastive loss function... Expression writing:

[0098]

[0099] In the formula, They represent The value of the exponential function; It represents the query text vector ε q Embedded representation in the decoder the similarity between the embedding representation of the corresponding positive sample document in the decoder wherein denotes the query text vector ε q the embedding representation in the i-th decoding process, denotes the embedding representation of the corresponding positive sample document of the query text q in the i-th decoding process, T denotes the transpose operation; denotes the query text vector ε q the embedding representation in the decoder the similarity between the embedding representation of the corresponding soft negative sample document in the decoder wherein denotes the embedding representation of the corresponding soft negative sample document of the query text q in the i-th decoding process; denotes the query text vector ε q the embedding representation in the decoder the similarity between the embedding representation of the corresponding hard negative sample document in the decoder wherein denotes the embedding representation of the corresponding hard negative sample document of the query text q in the i-th decoding process.

[0100] Document-aware label mapping loss function is written as:

[0101]

[0102] wherein, denote the exponential function values of S lmpos , and denotes the embedding representation of the corresponding positive sample document of the query text q in the decoder the similarity between the label mapping representation of the output layer corresponding to the embedding representation of the positive sample document in the decoder , denotes the label mapping representation of the output layer corresponding to the embedding representation of the corresponding positive sample document of the query text q in the i-th decoding process, which is the weight matrix W lm of the output layer; denotes the embedding representation of the corresponding soft negative sample document of the query text q in the decoder the similarity between the label mapping representation of the output layer corresponding to the embedding representation of the positive sample document in the decoder ; denotes the embedding representation of the corresponding hard negative sample document of the query text q in the decoder a label mapping representation corresponding to an embedding representation of the positive sample document in the decoder a similarity between.

[0103] It should be noted that in order to simplify the above analysis process, the above document samples (including positive document samples, hard negative document samples, and soft negative document samples) can be replaced by the context of the document sample to simplify the analysis difficulty, and the context of the document sample can be composed of multiple continuous word groups in the document, such as using 10 continuous 32-word groups to form the context of the document, thereby ensuring the maximum overlap with the query text.

[0104] Fourth step: adjusting the model parameters based on the loss function through back propagation and gradient descent algorithm. That is, the gradient of the loss function with respect to the model parameters is calculated through back propagation, and the model parameters are updated according to the opposite direction of the calculated gradient and the preset learning rate using the gradient descent algorithm, so that the model parameters are adjusted towards minimizing the loss method to improve the performance of the generative document retrieval model.

[0105] In addition, the generative document retrieval model can be loaded into the memory, for example, the original data of the generative document retrieval model can be loaded from the non-volatile memory to the volatile memory to enable the processor to run the first neural network. The original data of the generative document retrieval model refers to the data that has not been processed, which usually includes the parameters and structure data of the first neural network. The structure data can be a calculation relationship based on the parameters, such as the forward propagation calculation relationship between the intermediate layers and the neurons. Specifically, the structure data can include code related to the structure of the generative document retrieval model, such as code for performing related calculations between the intermediate layers and the neurons.

[0106] In an embodiment, a region for loading the generative document retrieval model can be divided in the memory, which can include a structure data storage area and a parameter storage area. The structure data storage area is used to store the structure-related code, and the parameters referred to by the structure-related code can be pointed to the address of a specific parameter in the parameter storage area. During the training process of the generative document retrieval model, the parameters may need to be frequently updated, and then the parameter values in the parameter storage area can be updated.

[0107] Therefore, the embodiments of the present application train the generative document retrieval model through the above document perception contrast learning, progressive learning, and document perception label mapping strategies, and have the following technical advantages:

[0108] (1) Through the encoder-decoder joint contrast, the model can not only understand the global semantics of the document, but also perform fine-grained position-by-position contrast learning at the decoder level, thereby enhancing the ability of the model to capture the semantic similarity between the query and the document.

[0109] (2) The negative sample document sampling strategy from simple to difficult helps the model to quickly learn the basic knowledge in the early stage, and gradually increases the learning difficulty, avoids the model from falling into local optimum, and speeds up the overall convergence speed. At the same time, with the deepening of training, the introduction of soft negative sample documents increases the learning difficulty of the model, prompting the model to make accurate judgments when facing high confusion documents, and improving the robustness of the model in complex retrieval scenarios.

[0110] (3) The document-aware label mapping method enables generative retrieval to learn from the advantages of dense retrieval, achieving more efficient data management and retrieval performance.

[0111] In summary, the above-mentioned generative document retrieval model can provide high-quality and high-relevance retrieval results when processing complex queries and document sets, thereby improving user experience and satisfaction.

[0112] Finally, in the technical solution provided in the above step S106, the retrieval system can parse the target document identifier token sequence to convert the target document identifier token sequence into the original semantics; then use the parsed document identifier to perform reverse lookup in the original database to locate the target document corresponding to the document identifier; the retrieved target document is presented to the target object in a preset manner, such as display in summary form, provide full-text link, or directly load the document content in the application program, etc.

[0113] Embodiment 2

[0114] According to the embodiments of the present application, a document retrieval system for implementing the document retrieval method in Embodiment 1 is also provided, as shown in Figure 2 The document retrieval system at least includes: an acquisition module 22, a retrieval module 24 and a feedback module 26, wherein:

[0115] The acquisition module 22 is configured to acquire the target query text input by the target object;

[0116] The retrieval module 24 is configured to retrieve the target document identifier token sequence with the highest semantic association with the target query text from the target database by using the pre-trained generative document retrieval model, wherein the target database includes a plurality of documents and a document identifier token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model using a contrast learning technique and a progressive learning strategy;

[0117] The feedback module 26 is configured to feed back the target document corresponding to the target document identifier token sequence to the target object.

[0118] It should be noted that the modules in the document retrieval system in the embodiments of the present application correspond one by one to the implementation steps of the document retrieval method in Embodiment 1. Since Embodiment 1 has been described in detail, the details not embodied in this embodiment can be referred to Embodiment 1, and will not be described in detail here.

[0119] Embodiment 3

[0120] According to the embodiments of the present application, a computer program product is also provided, which includes a computer program. When the computer program is executed by a processor, the document retrieval method in Embodiment 1 is implemented.

[0121] According to the embodiments of the present application, a non-volatile storage medium is also provided, which includes a stored computer program. The device in which the non-volatile storage medium is located executes the document retrieval method in Embodiment 1 by running the computer program.

[0122] According to the embodiments of the present application, a processor is also provided, which is used to run a computer program. When the computer program is run, the document retrieval method in Embodiment 1 is executed.

[0123] According to the embodiments of the present application, an electronic device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to execute the document retrieval method in Embodiment 1 by the computer program.

[0124] Specifically, when the computer program is run, the following steps are implemented: obtaining a target query text input by a target object; retrieving, by using a pre-trained generative document retrieval model, a target document identification token sequence with the highest semantic association with the target query text from a target database, wherein the target database includes a plurality of documents and a document identification token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model by using a contrast learning technique and a progressive learning strategy; and feeding back a target document corresponding to the target document identification token sequence to the target object.

[0125] As an optional implementation, the above electronic device can exist in the form of a mobile terminal, a computer terminal or a similar computing device. Figure 3 A hardware structure block diagram of an electronic device for implementing a document retrieval method is shown. As shown in FIG. 8, the electronic device includes a processor 801, a memory 802, a communication interface 803, and a communication bus 804. Figure 3 As shown, the electronic device 30 may include one or more processors 302 (shown as 302a, 302b, ..., 302n in the figure) (processor 302 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, electronic device 30 may also include... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0126] It should be noted that the aforementioned one or more processors 302 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element of the electronic device 30. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0127] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the document retrieval method in this embodiment. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, thereby implementing the aforementioned application vulnerability detection method. The memory 304 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 304 may further include memory remotely located relative to the processor 302, and these remote memories can be connected to the electronic device 30 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0128] The transmission device 306 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the electronic device 30. In one example, the transmission device 306 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

[0129] The display can be a liquid crystal display (LCD) that is touch screen type, for example, which can enable a user to interact with a user interface of the electronic device 30.

[0130] The above-mentioned embodiment numbers are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0131] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the system embodiments described above are only schematic. For example, the division of the units can be a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, units or modules, and can be electrical or other forms.

[0133] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0134] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0135] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0136] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.< / sos> < / eos>

Claims

1. A document retrieval method characterized by, The method comprises the following steps: obtaining a target query text input by a target object; retrieving a target document identification token sequence with the highest semantic relevance to the target query text from a target database by using a pre-trained generative document retrieval model, wherein the target database comprises a plurality of documents and a document identification token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model using a contrastive learning technique and a progressive learning strategy; feeding a target document corresponding to the target document identification token sequence to the target object.

2. The method of claim 1, wherein, The training process of the generative document retrieval model comprises the following steps: obtaining a plurality of sets of training sample data, wherein each set of training sample data comprises a query text, a document identification token sequence corresponding to a positive sample document with a semantic relevance higher than a first threshold value to the query text, and a document identification token sequence corresponding to a hard negative sample document with a semantic relevance lower than a second threshold value to the query text, and the first threshold value is much higher than the second threshold value; determining the sequence generation model, wherein the sequence generation model comprises at least an encoder, a decoder, and an output layer; iteratively training the sequence generation model using a contrastive learning technique and a progressive learning strategy based on the plurality of sets of training sample data to obtain the generative document retrieval model.

3. The method of claim 2, wherein, Obtaining a plurality of sets of training sample data comprises the following steps: determining a plurality of documents in the target database, and performing feature extraction on each document to obtain a corresponding document embedding vector; randomly sampling a plurality of continuous word groups from each document, and using the plurality of continuous word groups in each document as a plurality of query texts; clustering the document embedding vectors of each document using a hierarchical k-means algorithm to obtain a clustering tree, and determining a clustering path for each document from bottom to top in the clustering tree, and determining a document identification token sequence for each document based on the clustering path; for each document, a plurality of sets of training sample data are composed of the plurality of query texts corresponding to the document, the document identification token sequence of the document as a positive sample document, and the document identification token sequences of other documents as hard negative sample documents.

4. The method of claim 3, wherein, Clustering the document embedding vectors using a hierarchical k-means algorithm to obtain a clustering tree comprises the following steps: using the document embedding vectors of the plurality of documents as initial clustering clusters, and randomly selecting k document embedding vectors from the initial clustering clusters as k initial clustering centers; assigning the plurality of document embedding vectors in the initial clustering clusters to the nearest initial clustering center using a k-means algorithm to obtain k sub-clustering clusters, and re-determining the clustering centers of the k sub-clustering clusters; determining, for each of the sub-clustering clusters, a number of document embedding vectors contained in a current sub-clustering cluster, and performing recursive clustering on the plurality of document embedding vectors in the sub-clustering cluster using the k-means algorithm until the number of document embedding vectors contained in the sub-clustering cluster after the recursive clustering is not higher than the node number threshold, in a case where the number of document embedding vectors contained in the current sub-clustering cluster is higher than the node number threshold; obtaining a multi-level clustering tree by taking the clustering centers of the initial clustering clusters as root nodes, taking the clustering centers of the k sub-clustering clusters obtained in the first round of clustering as sub-nodes of a first level, and taking the clustering centers of the sub-clustering clusters after the recursive clustering and the document embedding vectors contained in each of the sub-clustering clusters as sub-nodes of deeper levels.

5. The method of claim 3, wherein, determining, from the clustering tree, a clustering path of each of the document embedding vectors in turn from bottom to top, and determining a document token sequence of each of the documents according to the clustering path, including: for each of the document embedding vectors, determining, from bottom to top, a node sequence number of a clustering cluster to which the document embedding vector is assigned when clustering at each level and a node sequence number of a clustering cluster to which the document embedding vector is finally assigned, starting from a root node of the clustering tree; adding, to the node sequence number of the clustering cluster to which the document embedding vector is assigned when clustering at each level, a product of a node number threshold and a number of levels corresponding to clustering at the corresponding level, to obtain the document token sequence of the document.

6. The method of claim 2, wherein, iteratively training the sequence generation model using a contrast learning technique and a progressive learning strategy based on the plurality of sets of training sample data, to obtain the generative document retrieval model, including: in a first number of iteration training rounds, using the sequence generation model to retrieve, from a target database, a plurality of documents whose semantic relevance to a plurality of query texts in a training batch corresponding to the first number of iteration training rounds is higher than a third threshold value but lower than the first threshold value as soft negative sample documents, determining a document token sequence corresponding to each of the soft negative sample documents, and reconstituting a plurality of sets of new training sample data from the plurality of sets of training sample data in the training batch corresponding to the first number of iteration training rounds, the plurality of soft negative sample documents, and the document token sequences corresponding to the plurality of soft negative sample documents; and performing contrast learning on the sequence generation model based on the plurality of sets of new training sample data, to adjust model parameters of the sequence generation model, wherein the third threshold value is higher than the second threshold value. In the second number of iterative training rounds, the sequence generation model is used to retrieve, from the target database, a first number of documents with a semantic relevance to the query text that is higher than a fourth threshold value but lower than the first threshold value in a training batch corresponding to the second number of iterative training rounds as soft negative sample documents, and determine a document identification token sequence corresponding to each of the soft negative sample documents, and reconstitute a plurality of new training sample data groups from the plurality of training sample data groups in the training batch corresponding to the first number of iterative training rounds, the first number of soft negative sample documents, and the corresponding document identification token sequences; and perform contrastive learning on the sequence generation model based on the plurality of new training sample data groups to adjust model parameters of the sequence generation model, wherein the fourth threshold value is higher than the third threshold value. In the third number of iterative training rounds, the sequence generation model is used to retrieve, from the target database, a second number of documents with a semantic relevance to the query text that is higher than the fourth threshold value but lower than the first threshold value in a training batch corresponding to the third number of iterative training rounds as soft negative sample documents, and determine a document identification token sequence corresponding to each of the soft negative sample documents, and reconstitute a plurality of new training sample data groups from the plurality of training sample data groups in the training batch corresponding to the third number of iterative training rounds, the second number of soft negative sample documents, and the corresponding document identification token sequences; and perform contrastive learning on the sequence generation model based on the plurality of new training sample data groups to adjust model parameters of the sequence generation model until the model parameters converge, thereby obtaining the trained generative document retrieval model, wherein the second number is less than the first number.

7. The method of claim 6, wherein, Performing contrastive learning on the sequence generation model based on the plurality of new training sample data groups to adjust model parameters of the sequence generation model comprises: For each of the new training sample data groups, input the query text in the new training sample data group into an encoder in the sequence generation model to obtain a query text embedding vector output by the encoder; Cyclically perform the following steps until a preset decoding termination condition is met, thereby obtaining a predicted document identification token sequence that matches the query text in the new training sample data group: in an i-th decoding process, input the query text embedding vector and a predicted document identification token sequence output in a previous i-1 decoding process into a decoder in the sequence generation model to obtain a hidden state vector output by the decoder, wherein when i = 1, the input of the decoder is the query text embedding vector and a preset initial document identification token; convert the hidden state vector into a predicted probability distribution using an output layer in the sequence generation model, and select a predicted document identification token with the highest probability as the input of the decoder in an (i+1)-th decoding process according to the predicted probability distribution; and Cyclically perform the following steps until a preset decoding termination condition is met, thereby obtaining a predicted document identification token sequence that matches the query text in the new training sample data group: in an i-th decoding process, input the query text embedding vector and a predicted document identification token sequence output in a previous i-1 decoding process into a decoder in the sequence generation model to obtain a hidden state vector output by the decoder, wherein when i = 1, the input of the decoder is the query text embedding vector and a preset initial document identification token; convert the hidden state vector into a predicted probability distribution using an output layer in the sequence generation model, and select a predicted document identification token with the highest probability as the input of the decoder in an (i+1)-th decoding process according to the predicted probability distribution; According to the loss function of the sequence generation model constructed by the predicted document identification token sequence matched with the query text in the new training sample data, the document identification token sequence corresponding to the positive sample document, the document identification token sequence corresponding to the hard negative sample document, and the document identification token sequence corresponding to the soft negative sample document, the loss function of the sequence generation model is constructed: wherein, denotes the loss function, denotes the prediction loss function, denotes the encoder boundary loss function, denotes the decoder contrast loss function, denotes the document-aware label mapping loss function, wherein: the prediction loss function is written as: where d k represents the kth predicted document identification token in the predicted document identification token sequence matched by the query text q, d <k represents a sequence consisting of the first k document identification tokens in the predicted document identification token sequence matched by the query text q, |d| represents the total number of tokens in the predicted document identification token sequence matched by the query text q, p θ represents a model parameter, and p θ = Softmax(Decoder(Encoder(q), d <k )· W lm ), W lm represents a weight matrix of the output layer, Encoder represents the encoder, Decoder represents the decoder, and Softmax represents an activation function of the output layer; and the encoder boundary loss function is written as: In the formula, α,β represent weight coefficients, γ represents a preset boundary value, K pos represents the vector similarity between the query text vector ε corresponding to the query text q and the corresponding positive sample document embedding vector q represents the vector similarity between the query text vector ε corresponding to the query text q and the corresponding soft negative sample document embedding vector neg q represents the vector similarity between the query text vector ε corresponding to the query text q and the corresponding hard negative sample document embedding vector rnd q ; the expression of the decoder contrast loss function is written as:​​​​​ wherein respectively represent the exponential function value of and wherein denotes the embedding representation of the query text vector q denotes the embedding representation of the query text q in the i-th decoding process, denotes the embedding representation of the positive sample document corresponding to the query text q in the i-th decoding process, and T denotes the transpose operation, wherein denotes the embedding representation of the soft negative sample document corresponding to the query text q in the i-th decoding process, wherein denotes the embedding representation of the hard negative sample document corresponding to the query text q in the i-th decoding process; and the document-aware label mapping loss function is written as: wherein respectively represent exponential function values of wherein represents the label mapping representation of the positive sample document corresponding to the query text q in the i-th decoding process; By back propagation and gradient descent algorithm, the model parameters are adjusted based on the loss function.

8. A document retrieval system characterized by, The method comprises the steps of: An acquisition module is configured to acquire a target query text input by a target object; A retrieval module is configured to retrieve a target document identification token sequence with the highest semantic relevance to the target query text from a target database by using a pre-trained generative document retrieval model, wherein the target database comprises a plurality of documents and a document identification token sequence corresponding to each document, and the generative document retrieval model is obtained by training a sequence generation model using a contrastive learning technique and a progressive learning strategy; A feedback module is configured to feed back a target document corresponding to the target document identification token sequence to the target object.

9. A computer program product, characterised in that, The computer program is executed by a processor to implement the document retrieval method of any one of claims 1 to 7. The memory and the processor, wherein the memory stores a computer program, and the processor is configured to execute the document retrieval method of any one of claims 1 to 7 by the computer program.

10. An electronic device, comprising: The memory and the processor, wherein the memory stores a computer program, and the processor is configured to execute the document retrieval method of any one of claims 1 to 7 by the computer program. ​

Images