Document retrieval method and device, computer device, storage medium and product

Abstract

The application relates to the technical field of artificial intelligence, and relates to a document retrieval method and device, computer equipment, a storage medium and a product. The method comprises the following steps: obtaining a to-be-retrieved document; extracting a target vector of the to-be-retrieved document; the target vector is obtained based on a title vector, a content vector and a sentence vector of the to-be-retrieved document; according to the target vector of the to-be-retrieved document and target vectors of index documents in an index library, the to-be-retrieved document is retrieved in the index library to obtain a retrieval result of the to-be-retrieved document; the target vector of the index document is obtained based on a title vector, a content vector and a sentence vector of the index document. The method can improve the accuracy of the document retrieval result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a document retrieval method, apparatus, computer equipment, storage medium, and product. Background Technology

[0002] With the development of information retrieval technology, document retrieval is being used more and more widely. The process of document retrieval involves inputting the document to be retrieved into an index database for matching or similarity calculation, thereby obtaining the retrieval results for the document to be retrieved.

[0003] Traditional techniques represent the document to be retrieved as a first vector and each document in the index as a second vector. Then, based on the similarity between the first vector and each of the second vectors, the documents in the index are sorted to obtain a ranking result, and the retrieval result for the document to be retrieved is determined based on the ranking result.

[0004] However, traditional document retrieval methods based on a first vector and multiple second vectors suffer from low accuracy in the retrieval results. Summary of the Invention

[0005] Therefore, it is necessary to provide a document retrieval method, apparatus, computer equipment, storage medium, and product that can improve the accuracy of document retrieval results in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a document retrieval method, including:

[0007] Retrieve the document to be searched;

[0008] Extract the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector and sentence vector of the document to be retrieved;

[0009] Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is retrieved in the index library to obtain the retrieval results of the document to be retrieved; the target vector of the indexed document is obtained based on the title vector, content vector and sentence vector of the indexed document.

[0010] In one embodiment, extracting the target vector of the document to be retrieved includes:

[0011] Extract the title vector, content vector, and sentence vector of the document to be retrieved;

[0012] A preset clustering algorithm is used to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved.

[0013] The title vector, content vector, and centroid vector of the document to be retrieved are fused to obtain the target vector of the document to be retrieved.

[0014] In one embodiment, extracting the title vector, content vector, and sentence vector of the document to be retrieved includes:

[0015] The title vector of the document to be retrieved is extracted using a first preset semantic representation model; the first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents;

[0016] The content vector of the document to be retrieved is extracted using a preset text vector model; the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents;

[0017] The sentence vector of the document to be retrieved is extracted using a second preset semantic representation model; the second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents.

[0018] In one embodiment, the method further includes:

[0019] Multiple historical title data are input into the initial semantic representation model for vector transformation to generate multiple predicted title vectors;

[0020] For each of the multiple predicted title vectors, the predicted title vectors are randomly deactivated to generate a first positive sample pair, and the predicted title vectors that have not been randomly deactivated are taken as the first negative sample pair that matches the first positive sample pair.

[0021] Based on the first positive sample pair and the first negative sample pair of each predicted title vector, the value of the first loss function is calculated, and the model parameters of the initial semantic representation model are updated based on the value of the first loss function to obtain the first preset semantic representation model.

[0022] In one embodiment, the method further includes:

[0023] Multiple historical content data are input into the initial text vector model for vector transformation to generate multiple predicted content vectors;

[0024] For each of the multiple predicted content vectors, the predicted content vectors are randomly deactivated to generate a second positive sample pair, and the predicted content vectors that have not been randomly deactivated are taken as the second negative sample pair that matches the second positive sample pair.

[0025] Based on the second positive sample pairs and the second negative sample pairs of each predicted content vector, the value of the second loss function is calculated, and the model parameters of the initial text vector model are updated based on the value of the second loss function to obtain the preset text vector model.

[0026] In one embodiment, the method further includes:

[0027] Multiple historical sentence data are input into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors;

[0028] For each of the multiple predicted sentence vectors, the predicted sentence vectors are randomly deactivated to generate a third positive sample pair, and the predicted sentence vectors that have not been randomly deactivated are taken as the third negative sample pair that matches the third positive sample pair.

[0029] Based on the third positive sample pair and the third negative sample pair of each predicted sentence vector, the value of the third loss function is calculated, and the model parameters of the initial semantic representation model are updated based on the value of the third loss function to obtain the second preset semantic representation model.

[0030] In one embodiment, the method further includes:

[0031] For each indexed document in the index library, extract the title vector, content vector, and sentence vector of the indexed document;

[0032] The sentence vectors of each index document are clustered using the preset clustering algorithm to obtain the centroid vector of each index document;

[0033] The title vector, content vector, and centroid vector of each indexed document are fused to obtain the target vector of each indexed document.

[0034] Secondly, this application also provides a document retrieval device, comprising:

[0035] The retrieval module is used to retrieve the documents to be searched.

[0036] The target vector extraction module is used to extract the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector and sentence vector of the document to be retrieved;

[0037] The retrieval module is used to retrieve the document to be retrieved in the index library based on the target vector of the document to be retrieved and the target vectors of each index document in the index library, and to obtain the retrieval results of the document to be retrieved; the target vector of the index document is obtained based on the title vector, content vector and sentence vector of the index document.

[0038] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.

[0039] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0040] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0041] The aforementioned document retrieval method, apparatus, computer equipment, storage medium, and product acquire a document to be retrieved; extract a target vector from the document to be retrieved; the target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved; based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is retrieved in the index library to obtain the retrieval results; the target vector of the indexed document is obtained based on the title vector, content vector, and sentence vector of the indexed document. Since this application can extract the target vector of the document to be retrieved from the document to be retrieved based on the title vector, content vector, and sentence vector of the document to be retrieved, this application comprehensively considers the title vector, content vector, and sentence vector of the document to be retrieved during the extraction process, thus obtaining a more accurate target vector of the document to be retrieved. Similarly, since this application can pre-determine the target vectors of each indexed document in the index library based on the title vector, content vector, and sentence vector of the indexed document, this application can obtain more accurate target vectors for each indexed document. Then, based on the more accurate target vector of the document to be retrieved and the more accurate target vector of each indexed document in the index, the document to be retrieved is searched in the index, and more accurate search results for the document to be retrieved can be obtained. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a diagram illustrating the application environment of a document retrieval method in one embodiment.

[0044] Figure 2 This is a flowchart illustrating a document retrieval method in one embodiment;

[0045] Figure 3 This is a flowchart illustrating the steps for obtaining the target vector of a document to be retrieved in one embodiment.

[0046] Figure 4 This is a flowchart illustrating the steps for extracting the title vector, content vector, and sentence vector of a document to be retrieved in one embodiment.

[0047] Figure 5 This is a flowchart illustrating the steps for obtaining the target vectors of each indexed document in another embodiment;

[0048] Figure 6 This is a flowchart illustrating a document retrieval method in one optional embodiment;

[0049] Figure 7 This is a schematic diagram of the overall process of a document retrieval method in one embodiment;

[0050] Figure 8 This is a structural block diagram of a document retrieval device in one embodiment;

[0051] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] With the development of information retrieval technology, document retrieval is being used more and more widely. The process of document retrieval involves inputting the document to be retrieved into an index database for matching or similarity calculation, thereby obtaining the retrieval results for the document to be retrieved.

[0054] Traditional techniques represent the document to be retrieved as a first vector and each document in the index as a second vector. Then, based on the similarity between the first vector and each of the second vectors, the documents in the index are sorted to obtain a ranking result, and the retrieval result for the document to be retrieved is determined based on the ranking result.

[0055] However, the accuracy of the first vector and multiple second vectors obtained by traditional methods is low. Therefore, traditional document retrieval methods based on the first vector and multiple second vectors suffer from low accuracy in document retrieval results.

[0056] The document retrieval method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 retrieves the document to be retrieved from terminal 102; server 104 extracts the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved; server 104 searches the index library for the document to be retrieved based on the target vector of the document to be retrieved and the target vectors of each index document in the index library, obtaining the retrieval results for the document to be retrieved; the target vector of the index document is obtained based on the title vector, content vector, and sentence vector of the index document. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0057] In one exemplary embodiment, such as Figure 2 As shown, a document retrieval method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes steps 220 to 260. Wherein:

[0058] Step 220: Obtain the document to be searched.

[0059] The document to be retrieved refers to the document for which a document retrieval is required. Optionally, the terminal 102 may actively send the document to be retrieved to the server 104, and the server 104 may receive the document to be retrieved sent by the terminal 102. Alternatively, the server 104 may also actively obtain the document to be retrieved from the terminal 102. Of course, the embodiments of this application do not limit the method of obtaining the document to be retrieved.

[0060] Step 240: Extract the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector and sentence vector of the document to be retrieved.

[0061] For example, server 104 can extract the title vector, content vector, and sentence vector of the document to be retrieved from the acquired document. Therefore, server 104 can determine the target vector of the document to be retrieved based on the title vector, content vector, and sentence vector. For example, the process of extracting the title vector, content vector, and sentence vector of the document to be retrieved can be as follows: server 104 can input the title data, content data, and sentence data of the document to be retrieved into a unified preset vector conversion model for vector conversion to obtain the title vector, content vector, and sentence vector of the document to be retrieved.

[0062] In this context, the title vector of the document to be retrieved refers to the vector obtained after vectorizing the title data of the document to be retrieved; the content vector of the document to be retrieved refers to the vector obtained after vectorizing the content data of the document to be retrieved; and the sentence vector of the document to be retrieved refers to the vector obtained after vectorizing the sentence data of the document to be retrieved. The target vector of the document to be retrieved refers to the vector obtained after vectorizing the document to be retrieved, and the target vector of the document to be retrieved is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved. The unified preset vector transformation model can include, but is not limited to, the Term Frequency-Inverse Document Frequency (TF-IDF) model, word to vector (word2vec), semantic representation (Sentence-BERT, SBERT) model, and text vector (paragraph vector, doc2vec) model, etc.

[0063] Step 260: Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is retrieved in the index library to obtain the retrieval results of the document to be retrieved; the target vector of the indexed document is obtained based on the title vector, content vector and sentence vector of the indexed document.

[0064] Optionally, since multiple index documents are pre-stored in the index library, the server 104 can extract the title vector, content vector, and sentence vector of each index document from the index library. Therefore, the server 104 can determine the target vector of each index document based on these vectors. Then, the server 104 can input the target vector of the document to be retrieved into the index library to perform a retrieval and obtain the retrieval results. For example, the document retrieval process can be as follows: the server 104 can calculate the similarity between the target vector of the document to be retrieved and the target vectors of each index document in the index library, obtaining the similarity calculation result between the target vector of the document to be retrieved and the target vectors of each index document in the index library. Therefore, the server 104 can sort the multiple index documents in the index library according to the similarity calculation result, obtain the sorting result, and determine the retrieval result of the document to be retrieved based on the sorting result.

[0065] The target vector of the indexed document refers to the vector obtained after vector transformation of the indexed document. The target vector of the indexed document is obtained based on the title vector, content vector, and sentence vector of the indexed document. The retrieval results for the document to be retrieved refer to the document information of a preset number of indexed documents in the index database that have a high similarity to the document to be retrieved, output in similarity order. Document information may include, but is not limited to, document identifiers, document titles, and document content. Of course, this embodiment does not limit the preset number.

[0066] In the above document retrieval method, the document to be retrieved is obtained; the target vector of the document to be retrieved is extracted; the target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved; based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is retrieved in the index library to obtain the retrieval results of the document to be retrieved; the target vector of the indexed document is obtained based on the title vector, content vector, and sentence vector of the indexed document. Since this application can extract the target vector of the document to be retrieved from the document to be retrieved based on the title vector, content vector, and sentence vector of the document to be retrieved, this application comprehensively considers the title vector, content vector, and sentence vector of the document to be retrieved during the extraction process, thus obtaining a more accurate target vector of the document to be retrieved. Similarly, since this application can pre-determine the target vector of each indexed document in the index library based on the title vector, content vector, and sentence vector of the indexed document, this application can obtain a more accurate target vector of each indexed document. Then, based on the more accurate target vector of the document to be retrieved and the more accurate target vector of each indexed document in the index, the document to be retrieved is searched in the index, and more accurate search results for the document to be retrieved can be obtained.

[0067] The above embodiments involve extracting the target vector of the document to be retrieved. The specific methods are described below. In one embodiment, such as... Figure 3 As shown, S240 includes:

[0068] Step 320: Extract the title vector, content vector, and sentence vector of the document to be retrieved.

[0069] Optionally, server 104 can input the title data, content data, and sentence data of the document to be retrieved into a unified document retrieval model for vector transformation to obtain the title vector, content vector, and sentence vector of the document to be retrieved. Alternatively, server 104 can input the title data of the document to be retrieved into a preset vector transformation model corresponding to the title data for vector transformation to obtain the title vector of the document to be retrieved; input the content data of the document to be retrieved into a preset vector transformation model corresponding to the content data for vector transformation to obtain the content vector of the document to be retrieved; and input the sentence data of the document to be retrieved into a preset vector transformation model corresponding to the sentence data for vector transformation to obtain the sentence vector of the document to be retrieved.

[0070] Among them, the unified preset vector transformation model, the preset vector transformation model corresponding to the title data, the preset vector transformation model corresponding to the content data, and the preset vector transformation model corresponding to the sentence data can all include, but are not limited to, the Term Frequency-Inverse Document Frequency (TF-IDF) model, the word to vector (word2vec) model, the Sentence-Bert (SBERT) semantic representation model, and the Paragraph Vector (doc2vec) model.

[0071] Step 340: Use a preset clustering algorithm to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved.

[0072] Optionally, since sentence data in a document is more complex than title and content data, with a document containing a large number of sentences and thus a large number of sentence vectors, server 104 can employ a preset clustering algorithm to cluster the sentence vectors of the document to be retrieved, obtaining the centroid vector of the document. The preset clustering algorithm can include, but is not limited to, any one of the following algorithms: K-means, Mean-Shift, DBSCAN, and Expectation-Maximization (EM).

[0073] In this embodiment, the preset clustering algorithm is the Mean-Shift algorithm. The Mean-Shift algorithm is a density-based non-parametric clustering algorithm that can automatically determine the number of clusters and does not require prior knowledge. The basic idea of ​​the Mean-Shift algorithm is to treat each data point as a data point and, by continuously moving the center of the data points, cluster them into regions with the highest local density. The execution process of the Mean-Shift algorithm is as follows: First, a data point is randomly selected as the starting point. Second, the density around the data point is calculated, and the data point is moved to the center of the region with the highest density, and the center point of the region with the highest density is determined. Then, using the new center point as the starting point, the above process is repeated until the data converges. Finally, data points that converge to the same center point are grouped into the same class. Compared to the K-means algorithm, the Mean-Shift algorithm can automatically discover the number of clusters without manual intervention.

[0074] For example, server 104 can use the Mean-Shift algorithm to cluster the sentence vectors of the document to be retrieved, thereby obtaining the centroid vector of the document. The calculation formula of the Mean-Shift algorithm is shown below.

[0075]

[0076] Among them, S c (x) is the centroid vector corresponding to the current point, and x is the current point (i.e., the center point). In formula (1), x i Let n be the sentence vector, G be the kernel function, and h be the bandwidth parameter. Formula (1) means that the centroid vector of the current point is the weighted average of all sample points (i.e., sentence vectors) within a circular region centered at the current point with a bandwidth of h.

[0077] Therefore, when the data converges, server 104 can obtain the centroid vector set S, which includes n centroid vectors S = {S...} c1 ,S c2 ,S c3 ,...,S cn}. Among them, S c1 S c2 S c3 S cn These represent the centroid vectors respectively.

[0078] Step 360: The title vector, content vector, and centroid vector of the document to be retrieved are fused to obtain the target vector of the document to be retrieved.

[0079] Optionally, server 104 can determine the fusion weight of the document to be retrieved based on the number of centroid vectors. A higher number of centroid vectors indicates a more dispersed distribution of the document's center points; therefore, it is necessary to increase the influence of the centroid vectors corresponding to sentence vectors on the final target vector, i.e., to increase the fusion weight. The formula for calculating the fusion weight is shown below.

[0080]

[0081] Where β represents the fusion weight; n represents the number of centroid vectors; M represents the average number of centroid vectors in each historical document. In this embodiment, M is 8 by default; when n>M, β=0.8.

[0082] Therefore, server 104 can fuse the title vector, content vector, and centroid vector of the document to be retrieved according to the fusion weights to obtain the target vector of the document to be retrieved. The calculation formula for fusing the title vector, content vector, and centroid vector of the document to be retrieved is shown in the following formula.

[0083]

[0084] Among them, F i≤nT represents the target vector; the number of target vectors in a document is equal to the number of centroid vectors. embeding Represents the title vector, C embeding This represents the content vector, and in this embodiment, the weight ratio between the two is maintained at 1:2.

[0085] In this embodiment, the title vector, content vector, and sentence vector of the document to be retrieved are extracted. A preset clustering algorithm is used to cluster the sentence vector of the document to be retrieved to obtain the centroid vector of the document. The title vector, content vector, and centroid vector of the document to be retrieved are then fused to obtain the target vector of the document. Because the target vector of the document to be retrieved can be obtained by fusing the title vector, content vector, and sentence vector of the document, this application comprehensively considers the title vector, content vector, and sentence vector of the document in the process of extracting the target vector, thus obtaining a more accurate target vector of the document.

[0086] The above embodiments involve extracting the title vector, content vector, and sentence vector of the document to be retrieved. The specific methods are described below. In one embodiment, such as... Figure 4 As shown, S320 includes:

[0087] Step 420: Extract the title vector of the document to be retrieved using the first preset semantic representation model; the first preset semantic representation model is obtained by training the initial semantic representation model based on the historical title data corresponding to multiple historical documents.

[0088] Optionally, server 104 can first extract the title data of the document to be retrieved from the document to be retrieved. Then, server 104 can input the title data of the document to be retrieved into a first preset semantic representation model for vector transformation to obtain the title vector of the document to be retrieved. Here, the title data of the document to be retrieved refers to the data corresponding to the title in the document to be retrieved. The first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents. Historical documents refer to the training data used for model training. Historical title data refers to the training data used to train the initial semantic representation model to obtain the first preset semantic representation model. The initial semantic representation model refers to the untrained Sentence-Bert model.

[0089] For example, server 104 can use natural language processing (NLP) techniques to extract the title data of the document to be retrieved. NLP techniques may include, but are not limited to, part-of-speech tagging methods and dependency parsing. The formula corresponding to the first preset semantic representation model is shown below.

[0090] T embeding=B_S(x i (4)

[0091] Among them, T embeding B_S(x) represents the title vector. i ) represents the process of converting title data into title vectors using the first preset semantic representation model (Sentence-BERT), where x in formula (4) i This represents the title data.

[0092] Step 440: Extract the content vector of the document to be retrieved using a preset text vector model; the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents.

[0093] Optionally, server 104 can first extract the content data of the document to be retrieved. Then, server 104 can input the content data of the document to be retrieved into a preset text vector model for vector transformation to obtain the content vector of the document to be retrieved. Here, the content data of the document to be retrieved refers to the data corresponding to the content in the document. The preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents. Historical content data refers to the training data used to train the initial text vector model to obtain the preset text vector model. The initial text vector model refers to an untrained doc2vec model.

[0094] For example, server 104 can use natural language processing (NLP) techniques to extract the content data of the document to be retrieved. NLP techniques may include, but are not limited to, part-of-speech tagging methods and dependency parsing. The formula corresponding to the preset text vector model is shown below.

[0095] C embeding =G_Doc2Vec(y i (5)

[0096] Among them, C embeding Represents a content vector, G_Doc2Vec(y i ) represents the process of converting content data into content vectors using a preset text vector model (Paragraph Vector, doc2vec). In formula (5), y i This represents content data.

[0097] Step 460: Extract sentence vectors of the document to be retrieved using the second preset semantic representation model; the second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents.

[0098] Optionally, server 104 can first extract sentence data from the document to be retrieved. Then, server 104 can input the sentence data of the document to be retrieved into a second preset semantic representation model for vector transformation to obtain the sentence vector of the document to be retrieved. Here, the sentence data of the document to be retrieved refers to the data corresponding to sentences in the document, and one document to be retrieved includes multiple sentence data. The second preset semantic representation model is obtained by training an initial semantic representation model based on historical sentence data corresponding to multiple historical documents. Historical sentence data refers to the training data used to train the initial semantic representation model to obtain the second preset semantic representation model. The initial semantic representation model refers to the untrained Sentence-Bert model.

[0099] For example, server 104 can use natural language processing (NLP) techniques to extract sentence data from the document to be retrieved. NLP techniques may include, but are not limited to, part-of-speech tagging methods and dependency parsing. The process of converting sentence data into sentence vectors using the second preset semantic representation model (Sentence-BERT) is similar to the process of converting title data into title vectors using the first preset semantic representation model; therefore, the above embodiments can be referred to, and will not be repeated here.

[0100] In this embodiment, a first preset semantic representation model is used to extract the title vector of the document to be retrieved; a preset text vector model is used to extract the content vector of the document to be retrieved; and a second preset semantic representation model is used to extract the sentence vector of the document to be retrieved. For the title vector, content vector, and sentence vector, respectively, relatively accurate models corresponding to the title data, content data, and sentence data can be used for vector transformation, thereby obtaining relatively accurate title vectors, relatively accurate content vectors, and relatively accurate sentence vectors respectively.

[0101] In the above embodiments, the first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents. The specific method for obtaining the first preset semantic representation model is described below. In one embodiment, the above document retrieval method further includes:

[0102] Multiple historical title data are input into the initial semantic representation model for vector transformation to generate multiple predicted title vectors.

[0103] For each predicted title vector among multiple predicted title vectors, random deactivation is performed on the predicted title vector to generate the first positive sample pair, and the predicted title vector among the multiple predicted title vectors that have not undergone random deactivation is taken as the first negative sample pair that matches the first positive sample pair.

[0104] Based on the first positive sample pair and the first negative sample pair of each predicted title vector, the value of the first loss function is calculated, and the model parameters of the initial semantic representation model are updated according to the value of the first loss function to obtain the first preset semantic representation model.

[0105] Optionally, in the first step, server 104 can obtain an initial semantic representation model (Sentence-BERT). In the second step, server 104 can input multiple historical title data into the initial semantic representation model for vector transformation, generating multiple predicted title vectors. In the third step, for each predicted title vector, a random dropout process is performed twice to generate two different vectors, which are then used as the first positive sample pair; the predicted title vectors that have not undergone random dropout are used as the first negative sample pair matching the first positive sample pair. In the fourth step, server 104 can calculate the value of a first loss function based on the first positive and first negative sample pairs of each predicted title vector. The formula for calculating the value of the first loss function is shown below.

[0106]

[0107] Wherein, L in formula (6) i To determine the value of the first loss function for comparative learning, h in formula (6) i , For a first positive sample pair, h in formula (6) j The first negative sample pair is the one that matches the first positive sample pair, where T is the temperature coefficient.

[0108] Fifth, server 104 can update the model parameters of the initial semantic representation model based on the value of the first loss function, and obtain a new initial semantic representation model based on the updated model parameters. Sixth, the second and subsequent steps are executed iteratively using the new initial semantic representation model until the value of the first loss function is minimized, thereby obtaining the target model parameters of the initial semantic representation model, and obtaining the first preset semantic representation model based on the target model parameters of the initial semantic representation model.

[0109] The principle of contrastive learning (SimCSE) is as follows: by training with positive and negative sample pairs, the distance between similar sentences in the vector space can be reduced, while the distance between dissimilar sentences in the vector space can be increased. This makes similar sentences closer in the vector space and dissimilar sentences farther apart, thereby obtaining the value of the contrastive learning loss function to optimize the initial model.

[0110] In this embodiment, multiple historical title data can be input into the initial semantic representation model for vector transformation to generate multiple predicted title vectors. Then, for each predicted title vector, a random deactivation process is performed to obtain a first positive sample pair and a first negative sample pair matching the first positive sample pair. Therefore, the value of a first loss function can be calculated based on the first positive and first negative sample pairs of each predicted title vector, and the initial semantic representation model can be trained based on the value of the first loss function, thus obtaining a well-trained first preset semantic representation model based on the principle of contrastive learning.

[0111] In the above embodiments, the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents. The specific method for obtaining the preset text vector model is described below. In one embodiment, the above document retrieval method further includes:

[0112] Multiple historical content data are input into the initial text vector model for vector transformation, generating multiple predicted content vectors.

[0113] For each of the multiple predicted content vectors, a random deactivation process is performed on the predicted content vector to generate a second positive sample pair. The predicted content vector that has not undergone random deactivation is then used as the second negative sample pair to match the second positive sample pair.

[0114] Based on the second positive sample pair and the second negative sample pair of each predicted content vector, the value of the second loss function is calculated, and the model parameters of the initial text vector model are updated according to the value of the second loss function to obtain the preset text vector model.

[0115] Optionally, in the first step, server 104 can use a natural language processing library (gensim library) to train an initial text vector model (Paragraph Vector, doc2vec) based on existing corpora. In the second step, server 104 can input multiple historical content data into the initial text vector model for vector transformation, generating multiple predicted content vectors. In the third step, for each predicted content vector, a random dropout process is performed twice to generate two different vectors, which are then used as the second positive sample pair; the predicted content vectors that have not undergone random dropout are used as the second negative sample pair matching the second positive sample pair. In the fourth step, server 104 can calculate the value of the second loss function based on the second positive and second negative sample pairs of each predicted content vector. The formula for calculating the value of the second loss function is shown below.

[0116]

[0117] Wherein, L in formula (7) i For the value of the second loss function in contrastive learning, h in formula (7) i , For a second positive sample pair, h in formula (7) j The second negative sample pair is the one that matches the second positive sample pair, where T is the temperature coefficient.

[0118] Fifth, server 104 can update the model parameters of the initial text vector model based on the value of the second loss function, and obtain a new initial text vector model based on the updated model parameters. Sixth, the second and subsequent steps are executed iteratively using the new initial text vector model until the value of the second loss function is minimized, thereby obtaining the target model parameters of the initial text vector model, and obtaining a preset text vector model based on the target model parameters of the initial text vector model.

[0119] In this embodiment, multiple historical content data can be input into an initial text vector model for vector transformation to generate multiple predicted content vectors. Then, for each predicted content vector, a random deactivation process is performed to obtain a second positive sample pair and a second negative sample pair that matches the second positive sample pair. Therefore, the value of a second loss function can be calculated based on the second positive and second negative sample pairs of each predicted content vector, and the initial text vector model can be trained based on the value of the second loss function to obtain a pre-trained text vector model based on the principle of contrastive learning.

[0120] In the above embodiments, the second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents. The specific method for this is described below. In one embodiment, the above document retrieval method further includes:

[0121] Multiple historical sentence data are input into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors.

[0122] For each predicted sentence vector among multiple predicted sentence vectors, random deactivation is performed on the predicted sentence vector to generate a third positive sample pair, and the predicted sentence vector among multiple predicted sentence vectors that has not undergone random deactivation is used as the third negative sample pair to match the third positive sample pair.

[0123] Based on the third positive and third negative sample pairs of each predicted sentence vector, the value of the third loss function is calculated, and the model parameters of the initial semantic representation model are updated according to the value of the third loss function to obtain the second preset semantic representation model.

[0124] Optionally, in the first step, server 104 can obtain an initial semantic representation model (Sentence-BERT). In the second step, server 104 can input multiple historical sentence data into the initial semantic representation model for vector transformation, generating multiple predicted sentence vectors. In the third step, for each predicted sentence vector, a random dropout process is performed twice to generate two different vectors, which are then used as the third positive sample pair; the predicted sentence vectors that have not undergone random dropout are used as the third negative sample pair matched with the third positive sample pair. In the fourth step, server 104 can calculate the value of the third loss function based on the third positive and third negative sample pairs of each predicted sentence vector. The formula for calculating the value of the third loss function is shown below.

[0125]

[0126] Wherein, L in formula (8) i To determine the value of the third loss function for comparative learning, h in formula (8) i , For a third positive sample pair, h in formula (8) j The third negative sample pair is the one that matches the third positive sample pair, where T is the temperature coefficient.

[0127] Fifth, server 104 can update the model parameters of the initial semantic representation model based on the value of the third loss function, and obtain a new initial semantic representation model based on the updated model parameters. Sixth, the second and subsequent steps are executed iteratively using the new initial semantic representation model until the value of the third loss function is minimized, to obtain the target model parameters of the initial semantic representation model, and obtain a second preset semantic representation model based on the target model parameters of the initial semantic representation model.

[0128] In this embodiment, multiple historical sentence data can be input into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors. Then, for each predicted sentence vector, a random deactivation process is performed to obtain a third positive sample pair and a third negative sample pair that matches the third positive sample pair. Therefore, the value of the third loss function is calculated based on the third positive and third negative sample pairs of each predicted sentence vector, and the initial semantic representation model is trained based on the value of the third loss function, thus obtaining a well-trained second preset semantic representation model based on the principle of contrastive learning.

[0129] In the above embodiments, the target vector of the indexed document is obtained based on the title vector, content vector, and sentence vector of the indexed document. The specific methods for obtaining the target vector of each indexed document are described below. In one embodiment, such as... Figure 5 As shown, the above document retrieval method also includes:

[0130] Step 520: For each indexed document in the index library, extract the title vector, content vector, and sentence vector of the indexed document.

[0131] Optionally, for each indexed document in the index database, server 104 can input the title data, content data, and sentence data of the indexed document into a unified document retrieval model for vector conversion to obtain the title vector, content vector, and sentence vector of the indexed document. Alternatively, server 104 can input the title data of the indexed document into a preset vector conversion model corresponding to the title data for vector conversion to obtain the title vector of the indexed document; input the content data of the indexed document into a preset vector conversion model corresponding to the content data for vector conversion to obtain the content vector of the indexed document; and input the sentence data of the indexed document into a preset vector conversion model corresponding to the sentence data for vector conversion to obtain the sentence vector of the indexed document.

[0132] Among them, the unified preset vector transformation model, the preset vector transformation model corresponding to the title data, the preset vector transformation model corresponding to the content data, and the preset vector transformation model corresponding to the sentence data can all include, but are not limited to, the Term Frequency-Inverse Document Frequency (TF-IDF) model, the word to vector (word2vec) model, the Sentence-Bert (SBERT) semantic representation model, and the Paragraph Vector (doc2vec) model.

[0133] Step 540: The sentence vectors of each indexed document are clustered using a preset clustering algorithm to obtain the centroid vector of each indexed document.

[0134] Optionally, since sentence data in a document is more complex than title and content data, with a document containing a large number of sentences and thus a large number of sentence vectors, server 104 can employ a preset clustering algorithm to cluster the sentence vectors of each indexed document to obtain the centroid vector of each indexed document. The preset clustering algorithm can include, but is not limited to, any one of the following algorithms: K-means, Mean-Shift, DBSCAN, or Expectation-Maximization (EM).

[0135] In this embodiment, the preset clustering algorithm is the Mean-Shift algorithm. The Mean-Shift algorithm is a density-based non-parametric clustering algorithm that can automatically determine the number of clusters and does not require prior knowledge. The basic idea of ​​the Mean-Shift algorithm is to treat each data point as a data point and, by continuously moving the center of the data points, cluster them into regions with the highest local density. The execution process of the Mean-Shift algorithm is as follows: First, a data point is randomly selected as the starting point. Second, the density around the data point is calculated, and the data point is moved to the center of the region with the highest density, and the center point of the region with the highest density is determined. Then, using the new center point as the starting point, the above process is repeated until the data converges. Finally, data points that converge to the same center point are grouped into the same class. Compared to the K-means algorithm, the Mean-Shift algorithm can automatically discover the number of clusters without manual intervention.

[0136] For example, server 104 can use the Mean-Shift algorithm to cluster the sentence vectors of each indexed document to obtain the centroid vector of each indexed document. The calculation formula of the Mean-Shift algorithm can be referred to the above embodiment, and will not be repeated here.

[0137] Step 560: The title vector, content vector and centroid vector of each indexed document are fused to obtain the target vector of each indexed document.

[0138] Optionally, for each indexed document in the index database, the server 104 can determine the fusion weight of the indexed document based on the number of centroid vectors. A higher number of centroid vectors indicates a more dispersed center point for the document corresponding to the centroid vectors; therefore, it is necessary to increase the influence of the centroid vectors corresponding to sentence vectors on the final target vector, i.e., to increase the fusion weight. The calculation formula for the fusion weight and the calculation formula for fusing the title vector, content vector, and centroid vector of the indexed document can be found in the above embodiments and will not be repeated here. Thus, for each indexed document in the index database, the server 104 can fuse the title vector, content vector, and centroid vector of the indexed document according to the fusion weight to obtain the target vector of each indexed document.

[0139] In this embodiment, for each indexed document in the index database, the title vector, content vector, and sentence vector of the indexed document are extracted. A preset clustering algorithm is used to cluster the sentence vectors of each indexed document to obtain the centroid vector of each indexed document. The title vector, content vector, and centroid vector of each indexed document are then fused to obtain the target vector of each indexed document. Because the target vector of an indexed document can be obtained by fusing the title vector, content vector, and sentence vector, this application comprehensively considers the title vector, content vector, and sentence vector of the indexed document during the extraction of the target vector, thus obtaining a relatively accurate target vector for each indexed document.

[0140] In an optional embodiment, such as Figure 6 As shown, a document retrieval method is provided, applied to server 104, including:

[0141] Step 602: For each indexed document in the index database, extract the title vector, content vector, and sentence vector of the indexed document;

[0142] Step 604: Use a preset clustering algorithm to cluster the sentence vectors of each indexed document to obtain the centroid vector of each indexed document;

[0143] Step 606: Merge the title vector, content vector, and centroid vector of each indexed document to obtain the target vector of each indexed document;

[0144] Step 608: Obtain the document to be searched;

[0145] Step 610: Input multiple historical title data into the initial semantic representation model for vector transformation to generate multiple predicted title vectors;

[0146] Step 612: For each predicted title vector in the multiple predicted title vectors, perform random deactivation processing on the predicted title vectors to generate the first positive sample pair, and take the predicted title vectors in the multiple predicted title vectors that have not been randomly deactivated as the first negative sample pair that matches the first positive sample pair.

[0147] Step 614: Calculate the value of the first loss function based on the first positive sample pair and the first negative sample pair of each predicted title vector, and update the model parameters of the initial semantic representation model based on the value of the first loss function to obtain the first preset semantic representation model;

[0148] Step 616: Extract the title vector of the document to be retrieved using the first preset semantic representation model;

[0149] Step 618: Input multiple historical content data into the initial text vector model for vector transformation to generate multiple predicted content vectors;

[0150] Step 620: For each of the multiple predicted content vectors, perform random deactivation processing on the predicted content vectors to generate a second positive sample pair, and take the predicted content vectors that have not been randomly deactivated as the second negative sample pair that matches the second positive sample pair.

[0151] Step 622: Calculate the value of the second loss function based on the second positive sample pair and the second negative sample pair of each predicted content vector, and update the model parameters of the initial text vector model based on the value of the second loss function to obtain the preset text vector model;

[0152] Step 624: Extract the content vector of the document to be retrieved using a preset text vector model;

[0153] Step 626: Input multiple historical sentence data into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors;

[0154] Step 628: For each predicted sentence vector in the multiple predicted sentence vectors, perform random deactivation processing on the predicted sentence vectors to generate a third positive sample pair, and take the predicted sentence vectors in the multiple predicted sentence vectors that have not been randomly deactivated as the third negative sample pair that matches the third positive sample pair.

[0155] Step 630: Calculate the value of the third loss function based on the third positive sample pair and the third negative sample pair of each predicted sentence vector, and update the model parameters of the initial semantic representation model based on the value of the third loss function to obtain the second preset semantic representation model.

[0156] Step 632: Extract sentence vectors from the document to be retrieved using the second preset semantic representation model;

[0157] Step 634: Use a preset clustering algorithm to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved;

[0158] Step 636: The title vector, content vector, and centroid vector of the document to be retrieved are fused to obtain the target vector of the document to be retrieved;

[0159] Step 638: Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is searched in the index library to obtain the search results for the document to be retrieved.

[0160] Optional, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the overall process of a document retrieval method in one embodiment. S702, server 104 can acquire multiple historical documents. For example, firstly, server 104 can acquire publicly available document datasets and document data from certain professional fields. Secondly, server 104 can fuse and splice the publicly available document datasets and document data from certain professional fields to obtain multiple randomly distributed historical documents. Then, server 104 can determine or extract historical title data, historical content data, and historical sentence data from each historical document according to its identifier. The historical sentence data can be a set of historical sentence data divided according to punctuation marks such as periods, question marks, and semicolons. For example, historical title data can be represented as: historical document 1, historical title data 1; historical document 2, historical title data 2; ...; historical document n, historical title data n. Historical content data can be represented as: historical document 1, historical content data 1; historical document 2, historical content data 2; ...; historical document n, historical content data n. Historical sentence data can be represented as: historical document 1, historical sentence data 1; historical document 1, historical sentence data 2; historical document 1, historical sentence data 3; historical document 2, historical sentence data 2; ...; historical document n, historical sentence data n.

[0161] S704, Server 104 can train the model separately using the principle of contrastive learning to obtain a first preset semantic representation model, a preset text vector model, and a second preset semantic representation model. S706, Server 104 can extract the title vector of the document to be retrieved using the first preset semantic representation model; extract the content vector of the document to be retrieved using the preset text vector model; and extract the sentence vector of the document to be retrieved using the second preset semantic representation model. A preset clustering algorithm is then used to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved; the title vector, content vector, and centroid vector of the document to be retrieved are then fused to obtain the target vector of the document to be retrieved. S708, Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index database, the document to be retrieved is searched in the index database to obtain the search results. The target vector of the indexed document is obtained based on the title vector, content vector, and sentence vector of the indexed document.

[0162] In the aforementioned document retrieval method, since this application can extract the target vector of the document to be retrieved based on its title vector, content vector, and sentence vector, it comprehensively considers the title vector, content vector, and sentence vector during the extraction process, thus obtaining a more accurate target vector. Similarly, since this application can pre-determine the target vector of each indexed document in the index database based on its title vector, content vector, and sentence vector, it can obtain a more accurate target vector for each indexed document. Then, based on the more accurate target vector of the document to be retrieved and the more accurate target vectors of each indexed document in the index database, a search is performed on the document to be retrieved in the index database, resulting in a more accurate retrieval result.

[0163] Since the core elements of a document include a title, overall content, and each detailed sentence that makes up the article, the title represents the soul of the entire article, around which the document unfolds. The overall content allows for a macro-level exploration of the article, reflecting its overall direction, style, and domain. Sentences represent the details of a document, carrying specific information. Therefore, this application can decompose the document's elements and construct and train corresponding models for each element using different methods to maximize the effectiveness of each element's characteristics. Then, fusion weights can be set based on the number of centroids corresponding to sentence elements, and the models corresponding to each element can be fused according to these weights. This allows for adjusting the participation level of document details based on the centroids of sentence elements, ultimately yielding a more accurate fused target vector.

[0164] Therefore, this application provides a similar document retrieval method based on fused vectors. First, the document is divided into different parts (title, content, and each sentence). Second, for the title and content parts, a contrastive learning approach is used to train the model, representing the title and content as title vectors and content vectors, respectively. For the sentence part, a clustering algorithm is used to obtain the centroid vectors corresponding to the sentences. Finally, the title vector, content vector, and centroid vector are fused to obtain the final target vector. The above method constructs models from three perspectives: title, overall document content, and each sentence, and then fuses the three models to obtain the final target vector. This effectively solves the shortcomings of traditional methods, such as the need for labeling large amounts of data and poor performance with long texts, thus obtaining a more accurate document vector representation and greatly improving the accuracy of similar document retrieval. In addition, since contrastive learning is a self-supervised learning method, no manual labeling is required when training the model using contrastive learning, making it more convenient and faster.

[0165] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0166] Based on the same inventive concept, this application also provides a document retrieval device for implementing the document retrieval method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more document retrieval device embodiments provided below can be found in the limitations of the document retrieval method described above, and will not be repeated here.

[0167] In one exemplary embodiment, such as Figure 8 As shown, a document retrieval device 800 is provided, including: an acquisition module 820, a target vector extraction module 840, and a retrieval module 860, wherein:

[0168] The acquisition module 820 is used to acquire the document to be retrieved.

[0169] The target vector extraction module 840 is used to extract the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector and sentence vector of the document to be retrieved.

[0170] The retrieval module 860 is used to retrieve the document to be retrieved in the index database based on the target vector of the document to be retrieved and the target vectors of each index document in the index database, and obtain the retrieval results of the document to be retrieved; the target vector of the index document is obtained based on the title vector, content vector and sentence vector of the index document.

[0171] In one embodiment, the target vector extraction module 840 includes:

[0172] The first extraction unit is used to extract the title vector, content vector, and sentence vector of the document to be retrieved;

[0173] The first clustering processing unit is used to cluster the sentence vectors of the document to be retrieved using a preset clustering algorithm to obtain the centroid vector of the document to be retrieved.

[0174] The first fusion unit is used to fuse the title vector, content vector, and centroid vector of the document to be retrieved to obtain the target vector of the document to be retrieved.

[0175] In one embodiment, the extraction unit includes:

[0176] The title vector extraction subunit is used to extract the title vector of the document to be retrieved using a first preset semantic representation model; the first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents;

[0177] The content vector extraction subunit is used to extract the content vector of the document to be retrieved using a preset text vector model; the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents;

[0178] The sentence vector extraction subunit is used to extract the sentence vector of the document to be retrieved using the second preset semantic representation model; the second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents.

[0179] In one embodiment, the document retrieval device 800 further includes:

[0180] The first vector conversion module is used to input multiple historical title data into the initial semantic representation model for vector conversion, and generate multiple predicted title vectors.

[0181] The first random deactivation processing module is used to perform random deactivation processing on each of the multiple predicted title vectors to generate a first positive sample pair, and to take the predicted title vectors that have not been randomly deactivated as the first negative sample pair that matches the first positive sample pair.

[0182] The first preset semantic representation model acquisition module is used to calculate the value of the first loss function based on the first positive sample pair and the first negative sample pair of each predicted title vector, and update the model parameters of the initial semantic representation model based on the value of the first loss function to obtain the first preset semantic representation model.

[0183] In one embodiment, the document retrieval device 800 further includes:

[0184] The second vector conversion module is used to input multiple historical content data into the initial text vector model for vector conversion, generating multiple predicted content vectors.

[0185] The second random deactivation processing module is used to perform random deactivation processing on each of the multiple predicted content vectors to generate a second positive sample pair, and to take the predicted content vectors that have not been randomly deactivated as the second negative sample pair that matches the second positive sample pair.

[0186] The preset text vector model acquisition module is used to calculate the value of the second loss function based on the second positive sample pair and the second negative sample pair of each predicted content vector, and update the model parameters of the initial text vector model based on the value of the second loss function to obtain the preset text vector model.

[0187] In one embodiment, the document retrieval device 800 further includes:

[0188] The third vector transformation module is used to input multiple historical sentence data into the initial semantic representation model for vector transformation, generating multiple predicted sentence vectors;

[0189] The third random deactivation module is used to perform random deactivation processing on each of the multiple predicted sentence vectors to generate a third positive sample pair, and to take the predicted sentence vectors that have not been randomly deactivated as the third negative sample pair that matches the third positive sample pair.

[0190] The second preset semantic representation model acquisition module is used to calculate the value of the third loss function based on the third positive sample pair and the third negative sample pair of each predicted sentence vector, and update the model parameters of the initial semantic representation model based on the value of the third loss function to obtain the second preset semantic representation model.

[0191] In one embodiment, the document retrieval device 800 further includes:

[0192] The second extraction module is used to extract the title vector, content vector, and sentence vector of each indexed document in the index library;

[0193] The second clustering processing module is used to cluster the sentence vectors of each indexed document using a preset clustering algorithm to obtain the centroid vector of each indexed document.

[0194] The second fusion module is used to fuse the title vector, content vector, and centroid vector of each indexed document to obtain the target vector of each indexed document.

[0195] Each module in the aforementioned document retrieval device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0196] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores document retrieval data. The I / O interfaces allow the processor to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements a document retrieval method.

[0197] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0198] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0199] Retrieve the document to be searched;

[0200] Extract the target vector from the document to be retrieved; the target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved.

[0201] Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index, the document to be retrieved is retrieved in the index to obtain the retrieval results; the target vector of the indexed document is obtained based on the title vector, content vector and sentence vector of the indexed document.

[0202] In one embodiment, the target vector of the document to be retrieved is extracted, and the processor, when executing the computer program, further performs the following steps:

[0203] Extract the title vector, content vector, and sentence vector of the document to be retrieved;

[0204] A pre-defined clustering algorithm is used to cluster the sentence vectors of the document to be retrieved, thereby obtaining the centroid vector of the document to be retrieved.

[0205] The title vector, content vector, and centroid vector of the document to be retrieved are fused to obtain the target vector of the document to be retrieved.

[0206] In one embodiment, the title vector, content vector, and sentence vector of the document to be retrieved are extracted. When the processor executes the computer program, it also performs the following steps:

[0207] The title vector of the document to be retrieved is extracted using a first preset semantic representation model; the first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents;

[0208] The content vector of the document to be retrieved is extracted using a preset text vector model; the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents;

[0209] The sentence vectors of the document to be retrieved are extracted using a second preset semantic representation model and a preset clustering algorithm. The second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents.

[0210] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0211] Multiple historical title data are input into the initial semantic representation model for vector transformation to generate multiple predicted title vectors;

[0212] For each predicted title vector among multiple predicted title vectors, random deactivation is performed on the predicted title vector to generate the first positive sample pair, and the predicted title vector among multiple predicted title vectors that has not undergone random deactivation is taken as the first negative sample pair that matches the first positive sample pair.

[0213] Based on the first positive sample pair and the first negative sample pair of each predicted title vector, the value of the first loss function is calculated, and the model parameters of the initial semantic representation model are updated according to the value of the first loss function to obtain the first preset semantic representation model.

[0214] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0215] Multiple historical content data are input into the initial text vector model for vector transformation to generate multiple predicted content vectors;

[0216] For each of the multiple predicted content vectors, random deactivation is performed on the predicted content vector to generate a second positive sample pair, and the predicted content vector that has not been randomly deactivated is taken as the second negative sample pair that matches the second positive sample pair.

[0217] Based on the second positive sample pair and the second negative sample pair of each predicted content vector, the value of the second loss function is calculated, and the model parameters of the initial text vector model are updated according to the value of the second loss function to obtain the preset text vector model.

[0218] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0219] Multiple historical sentence data are input into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors;

[0220] For each predicted sentence vector in multiple predicted sentence vectors, random deactivation is performed on the predicted sentence vector to generate a third positive sample pair, and the predicted sentence vectors that have not undergone random deactivation are used as the third negative sample pair to match the third positive sample pair.

[0221] Based on the third positive and third negative sample pairs of each predicted sentence vector, the value of the third loss function is calculated, and the model parameters of the initial semantic representation model are updated according to the value of the third loss function to obtain the second preset semantic representation model.

[0222] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0223] For each indexed document in the index database, extract the title vector, content vector, and sentence vector of the indexed document;

[0224] A pre-defined clustering algorithm is used to cluster the sentence vectors of each indexed document to obtain the centroid vector of each indexed document;

[0225] The title vector, content vector, and centroid vector of each indexed document are fused to obtain the target vector of each indexed document.

[0226] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0227] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0228] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0229] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0230] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0231] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A document retrieval method, characterized in that, The method includes: Retrieve the document to be searched; Extract the target vector of the document to be retrieved; the target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved. Extracting the target vector of the document to be retrieved includes: extracting the title vector, content vector, and sentence vector of the document to be retrieved; using a preset clustering algorithm to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved. The centroid vector is the weighted average of the sentence vectors within a circular region centered at the center point and with a bandwidth of h; fusing the title vector, content vector, and centroid vector of the document to be retrieved to obtain the target vector of the document to be retrieved, where h is the bandwidth parameter. The fusion weight of the document to be retrieved is determined according to the number of centroid vectors of the document to be retrieved. The more centroid vectors there are, the greater the influence of the centroid vector corresponding to the sentence vector on the final target vector, and the greater the fusion weight. Based on the target vector of the document to be retrieved and the target vectors of each indexed document in the index library, the document to be retrieved is retrieved in the index library to obtain the retrieval results of the document to be retrieved; the target vector of the indexed document is obtained based on the title vector, content vector and sentence vector of the indexed document.

2. The method according to claim 1, characterized in that, The extraction of the title vector, content vector, and sentence vector of the document to be retrieved includes: The title vector of the document to be retrieved is extracted using a first preset semantic representation model; the first preset semantic representation model is obtained by training an initial semantic representation model based on historical title data corresponding to multiple historical documents; The content vector of the document to be retrieved is extracted using a preset text vector model; the preset text vector model is obtained by training an initial text vector model based on historical content data corresponding to multiple historical documents; The sentence vector of the document to be retrieved is extracted using a second preset semantic representation model; the second preset semantic representation model is obtained by training the initial semantic representation model based on historical sentence data corresponding to multiple historical documents.

3. The method according to claim 2, characterized in that, The method further includes: Multiple historical title data are input into the initial semantic representation model for vector transformation to generate multiple predicted title vectors; For each of the multiple predicted title vectors, the predicted title vectors are randomly deactivated to generate a first positive sample pair, and the predicted title vectors that have not been randomly deactivated are taken as the first negative sample pair that matches the first positive sample pair. Based on the first positive sample pair and the first negative sample pair of each predicted title vector, the value of the first loss function is calculated, and the model parameters of the initial semantic representation model are updated based on the value of the first loss function to obtain the first preset semantic representation model.

4. The method according to claim 2 or 3, characterized in that, The method further includes: Multiple historical content data are input into the initial text vector model for vector transformation to generate multiple predicted content vectors; For each of the multiple predicted content vectors, the predicted content vector is randomly deactivated to generate a second positive sample pair, and the predicted content vectors that have not been randomly deactivated are taken as the second negative sample pair that matches the second positive sample pair. Based on the second positive sample pairs and the second negative sample pairs of each predicted content vector, the value of the second loss function is calculated, and the model parameters of the initial text vector model are updated based on the value of the second loss function to obtain the preset text vector model.

5. The method according to claim 2 or 3, characterized in that, The method further includes: Multiple historical sentence data are input into the initial semantic representation model for vector transformation to generate multiple predicted sentence vectors; For each of the multiple predicted sentence vectors, the predicted sentence vectors are randomly deactivated to generate a third positive sample pair, and the predicted sentence vectors that have not been randomly deactivated are taken as the third negative sample pair that matches the third positive sample pair. Based on the third positive sample pair and the third negative sample pair of each predicted sentence vector, the value of the third loss function is calculated, and the model parameters of the initial semantic representation model are updated based on the value of the third loss function to obtain the second preset semantic representation model.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: For each indexed document in the index library, extract the title vector, content vector, and sentence vector of the indexed document; The sentence vectors of each index document are clustered using the preset clustering algorithm to obtain the centroid vector of each index document; The title vector, content vector, and centroid vector of each indexed document are fused to obtain the target vector of each indexed document.

7. A document retrieval device, characterized in that, The device includes: The retrieval module is used to retrieve the documents to be searched. The target vector extraction module is used to extract the target vector of the document to be retrieved. The target vector is obtained based on the title vector, content vector, and sentence vector of the document to be retrieved. Extracting the target vector of the document to be retrieved includes: extracting the title vector, content vector, and sentence vector of the document to be retrieved; using a preset clustering algorithm to cluster the sentence vectors of the document to be retrieved to obtain the centroid vector of the document to be retrieved. The centroid vector is the weighted average of the sentence vectors within a circular region with a bandwidth of h centered at the center point; fusing the title vector, content vector, and centroid vector of the document to be retrieved to obtain the target vector of the document to be retrieved, where h is the bandwidth parameter; and determining the fusion weight of the document to be retrieved based on the number of centroid vectors. The more centroid vectors there are, the greater the influence of the centroid vectors corresponding to the sentence vectors on the final target vector, thus increasing the fusion weight. The retrieval module is used to retrieve the document to be retrieved in the index library based on the target vector of the document to be retrieved and the target vectors of each index document in the index library, and to obtain the retrieval result of the document to be retrieved; the target vector of the index document is obtained based on the title vector, content vector and sentence vector of the index document.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images