Information retrieval method and device, computer equipment, medium and program product

Abstract

The invention relates to an information retrieval method and device, computer equipment, a medium and a program product. The method comprises the following steps: acquiring to-be-retrieved information of a user; based on the to-be-retrieved information, a multi-path recall strategy is adopted to retrieve the retrieval information base, and a plurality of candidate texts are obtained; based on the similarity between the to-be-retrieved information and all the candidate texts, determining semantic density; determining clustering parameters based on the semantic density; performing duplicate removal processing on all the candidate texts based on the clustering parameters to obtain a plurality of duplicate-removed candidate texts; and determining a retrieval result corresponding to the to-be-retrieved information based on all the candidate texts after duplicate removal. By adopting the method, the accuracy of the retrieval result can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information retrieval, and in particular to an information retrieval method and device, a computer device, a medium and a program product. BACKGROUND

[0002] With the development of artificial intelligence technology, retrieval-augmented generation (RAG) technology has emerged. The technology is a technical framework combining information retrieval and text generation, which retrieves relevant document fragments from a knowledge base as context to guide a large language model to generate more accurate answers. The current mainstream RAG technology mainly adopts a multi-modal hybrid recall architecture, such as a multi-path parallel mechanism using semantic vector retrieval, keyword retrieval, and hypothetical question recall. Then, the multi-path recall results are fused to obtain the final retrieval result. However, in the multi-path recall mode, different models are used in each recall mode. Since different models use different types of corpora during training, the same semantics is presented in different modalities in different models when different models are applied, and the results corresponding to different modalities are difficult to unify. Therefore, when using a multi-modal hybrid recall architecture, different models will perform repeated recall on the same document, resulting in redundant recall results.

[0003] The current conventional way to eliminate redundancy is to set a fixed deduplication parameter and use a deduplication algorithm to eliminate redundancy. However, setting a fixed deduplication parameter can easily lead to poor deduplication results, thereby affecting the accuracy of the retrieval results.

[0004] Therefore, how to perform information retrieval based on multi-path recall to improve the accuracy of the retrieval results is a problem to be solved. SUMMARY

[0005] Therefore, it is necessary to provide an information retrieval method, device, computer device, medium and program product to improve the accuracy of the retrieval results.

[0006] In a first aspect, the present application provides an information retrieval method, comprising:

[0007] obtaining user information to be retrieved;

[0008] based on the information to be retrieved, using a multi-path recall strategy to retrieve a retrieval information library respectively, to obtain a plurality of candidate texts;

[0009] based on the similarity between the information to be retrieved and all the candidate texts, determining a semantic density;

[0010] determining a clustering parameter based on the semantic density;

[0011] perform deduplication processing on all the candidate texts based on the clustering parameter to obtain a plurality of deduplicated candidate texts;

[0012] determine the search result corresponding to the to-be-searched information based on all the deduplicated candidate texts.

[0013] In one of the embodiments, the determining the semantic density based on the similarity between the to-be-searched information and all the candidate texts comprises:

[0014] determining the similarity between the to-be-searched information and each of the candidate texts;

[0015] determining the average value of all the similarities as the semantic density.

[0016] In one of the embodiments, the determining the similarity between the to-be-searched information and each of the candidate texts comprises:

[0017] determining the cosine similarity between the to-be-searched information and each of the candidate texts as the initial similarity of the corresponding candidate text;

[0018] adjusting the initial similarity of the corresponding candidate text based on the language category of each of the candidate texts to obtain the similarity of each of the candidate texts.

[0019] In one of the embodiments, the determining the semantic density based on the semantic density comprises:

[0020] determining a target clustering parameter generation rule from the clustering parameter rule based on the value range of the semantic density, wherein the clustering parameter rule comprises a clustering parameter generation rule corresponding to different value ranges of the semantic density;

[0021] generating the clustering parameter based on the target clustering parameter generation rule.

[0022] In one of the embodiments, the performing deduplication processing on all the candidate texts based on the clustering parameter to obtain a plurality of deduplicated candidate texts comprises:

[0023] determining a clustering radius based on the clustering parameter;

[0024] performing clustering on all the candidate texts based on the clustering radius by using a density algorithm of clustering analysis to obtain the plurality of deduplicated candidate texts.

[0025] In one of the embodiments, the determining the search result corresponding to the to-be-searched information based on all the deduplicated candidate texts comprises:

[0026] determining any of the plurality of deduplicated candidate texts as a deduplicated target candidate text;

[0027] Determine the time decay parameter of the deduplicated target candidate text;

[0028] Based on the time decay parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0029] Based on the comprehensive evaluation value of each deduplicated candidate text, the multiple deduplicated candidate texts are sorted to obtain the retrieval results corresponding to the information to be retrieved.

[0030] In one embodiment, determining the time-decrease parameter of the deduplicated target candidate text includes:

[0031] Based on the query time of the information to be retrieved and the generation time of the deduplicated target candidate text, the timeliness parameter of the deduplicated target candidate text is determined;

[0032] Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameter, the timeliness decay parameter of the deduplicated target candidate text is determined.

[0033] In one embodiment, determining the timeliness decay parameter of the deduplicated target candidate text based on the text type and corresponding timeliness parameter of the deduplicated target candidate text includes:

[0034] Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameter, the initial timeliness decay parameter of the deduplicated target candidate text is determined.

[0035] The language confidence level is determined based on the language type of the deduplicated target candidate text;

[0036] The initial time-decrease parameter is adjusted based on the language confidence level to obtain the time-decrease parameter.

[0037] In one embodiment, determining the comprehensive evaluation value of the deduplicated target candidate text based on the time-decrease parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved includes:

[0038] Determine the popularity value of the deduplicated target candidate text;

[0039] Based on the time-effect decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0040] In one embodiment, determining the comprehensive evaluation value of the deduplicated target candidate text based on the timeliness decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value includes:

[0041] Determine the recall path to which the deduplicated target candidate text belongs;

[0042] The path parameters of the deduplicated target candidate text are determined based on the path weight corresponding to the recall path.

[0043] Based on the time decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, the popularity value, and the path parameters, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0044] In one embodiment, before determining the path parameters of the deduplicated target candidate text based on the path weight corresponding to the recall path, the method further includes:

[0045] Determine the number of time-related keywords, the number of technical terminology keywords, and the total number of words in the information to be retrieved;

[0046] The timeliness sensitivity factor is determined based on the number of time-related keywords and the total number of words.

[0047] The technology depth factor is determined based on the number of technical terminology keywords and the total number of words.

[0048] Based on the time sensitivity factor and the technology depth factor, the path weight corresponding to each recall path in the multi-path recall strategy is determined.

[0049] Secondly, this application also provides an information retrieval device, comprising:

[0050] The information acquisition module is used to acquire the user's search information;

[0051] The retrieval module is used to search the retrieval information database separately based on the information to be retrieved, using a multi-path recall strategy to obtain multiple candidate texts;

[0052] A semantic density determination module is used to determine semantic density based on the similarity between the information to be retrieved and all the candidate texts;

[0053] A clustering parameter determination module is used to determine clustering parameters based on the semantic density;

[0054] The deduplication module is used to perform deduplication processing on all the candidate texts based on the clustering parameters to obtain multiple deduplicated candidate texts.

[0055] The retrieval result determination module is used to determine the retrieval result corresponding to the information to be retrieved based on all deduplicated candidate texts.

[0056] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the information retrieval method described in any of the above embodiments.

[0057] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the information retrieval method described in any of the above embodiments.

[0058] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the information retrieval method described in any of the above embodiments.

[0059] The aforementioned information retrieval methods, devices, computer equipment, media, and program products employ a multi-path recall strategy to retrieve information from the database. This effectively avoids the missed detection problem inherent in single-strategy retrieval, improving both recall and accuracy. Furthermore, semantic density is determined based on the similarity between the information to be retrieved and all candidate texts, and clustering parameters are determined based on this semantic density. This allows for adaptive adjustment of the clustering parameters according to the information to be retrieved and the retrieval results. The adaptively adjusted clustering parameters are then used to deduplicate all candidate texts, effectively avoiding the poor deduplication results caused by setting fixed deduplication parameters. Finally, the retrieval results corresponding to the information to be retrieved are determined based on all deduplicated candidate texts, effectively improving the accuracy of the retrieval results. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the application environment of an information retrieval method provided in an embodiment of this application;

[0062] Figure 2 This is a flowchart of an information retrieval method provided in an embodiment of this application;

[0063] Figure 3 This is a flowchart illustrating an application example of an information retrieval method provided in an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of the structure of an information retrieval device provided in an embodiment of this application;

[0065] Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0068] The information retrieval method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. Figure 1This is a schematic diagram of an application environment for an information retrieval method provided in this application embodiment, wherein terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104, or it can be placed on the cloud or other network servers. The data storage system can be a retrieval information database storing various types of knowledge. Terminal 102 can obtain the user's retrieval information and transmit it to server 104 via a communication network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0069] In one exemplary embodiment, Figure 2 This is a flowchart of an information retrieval method provided in an embodiment of this application, which is applied to... Figure 1 Taking server 104 as an example, the explanation is as follows: Figure 2 As shown, the process includes:

[0070] Step 201: Obtain the user's search information.

[0071] Step 202: Based on the information to be retrieved, a multi-path recall strategy is used to search the information database separately to obtain multiple candidate texts.

[0072] Step 203: Determine the semantic density based on the similarity between the information to be retrieved and all candidate texts.

[0073] Step 204: Determine clustering parameters based on semantic density.

[0074] Step 205: Perform deduplication on all candidate texts based on clustering parameters to obtain multiple deduplicated candidate texts.

[0075] Step 206: Determine the search results corresponding to the information to be searched based on all deduplicated candidate texts.

[0076] For example, when a user needs to retrieve knowledge from a knowledge base, they can input the information to be retrieved on the terminal. The input methods for this information include, but are not limited to, text input, voice, and handwriting. Furthermore, the terminal transmits the user-inputted information to the server, thereby enabling the server to obtain the user's retrieval information.

[0077] The server can deploy multiple retrieval engines, such as semantic vector retrieval engines, keyword retrieval engines, and hypothetical question recall retrieval engines, each with a corresponding retrieval model.

[0078] Specifically, the semantic vector retrieval engine can include BCE, BGE, and T2V models. The BCE model uses a 768-dimensional vector, trained on corpora from various knowledge bases, and calculates cosine similarity. The BGE model uses a 1024-dimensional vector, trained on general internet corpora, and measures L2 distance. The T2V model uses a 512-dimensional vector, trained on scientific literature corpora, and calculates inner product similarity. Vector storage in the semantic vector retrieval engine uses a Milvus database to build a sharded index (IVF_FLAT algorithm, nlist=2048). The keyword retrieval engine can include an improved BM25F algorithm based on ElasticSearch. The hypothetical question recall retrieval engine can generate 10 potential questions for each document block using a large model, constructing a two-layer index: 1) a question-document inverted index; 2) a question vector index (using the FAISS library to build an HNSW graph, M=32 parameters).

[0079] Furthermore, after the server obtains the user's search information, it can adopt a multi-path retrieval strategy to search the search information database based on the search information. That is, each search engine searches the search information database based on the search information. Each search method is executed independently and does not affect each other, so that each search engine can retrieve the corresponding candidate text. The search results of multiple search engines are merged to obtain multiple candidate texts.

[0080] Furthermore, the server can determine the semantic density based on the similarity between the information to be retrieved and all candidate texts, and then determine the clustering parameters based on the semantic density. Finally, the server can deduplicate all candidate texts by clustering according to these parameters, resulting in multiple deduplicated candidate texts. Finally, the multiple deduplicated candidate texts are ranked to obtain the retrieval results corresponding to the information to be retrieved.

[0081] In the above implementation process, a multi-path recall strategy is used to retrieve information from the database, effectively avoiding the missed detection problem of a single strategy and improving the recall and accuracy of the retrieval. Furthermore, the semantic density is determined based on the similarity between the information to be retrieved and all candidate texts, and clustering parameters are determined based on the semantic density. This allows for adaptive adjustment of the clustering parameters according to the information to be retrieved and the retrieval results. The adaptively adjusted clustering parameters are then used to deduplicate all candidate texts, effectively avoiding the poor deduplication effect caused by setting fixed deduplication parameters. Finally, the retrieval results corresponding to the information to be retrieved are determined based on all deduplicated candidate texts, effectively improving the accuracy of the retrieval results.

[0082] In one embodiment, determining semantic density based on the similarity between the information to be retrieved and all candidate texts may include the following steps:

[0083] Step 1: Determine the similarity between the information to be retrieved and each candidate text.

[0084] Step 2: Determine the semantic density by averaging all similarities.

[0085] For example, a word vector model can be used to first convert the information to be retrieved and each candidate text into corresponding vectors. Then, a cosine similarity algorithm can be used to calculate the similarity between the information to be retrieved and each candidate text. Specifically, the word vector model can be Word2Vec, BERT4, or other models; there are no restrictions here.

[0086] Furthermore, the average of all similarities is determined, and this average is used as the semantic density. As an example, the semantic density can be determined by the following expression:

[0087] (1)

[0088] in, is the semantic density; N is the total number of candidate texts; For the first Candidate texts and information to be retrieved The similarity.

[0089] In the above implementation process, the semantic density is determined by the average similarity between the information to be retrieved and all candidate texts, which can accurately determine the average similarity of all recall results.

[0090] In one embodiment, determining the similarity between the information to be retrieved and each of the candidate texts may include the following steps:

[0091] Step 1: Determine the cosine similarity between the information to be retrieved and each candidate text as the initial similarity of the corresponding candidate text.

[0092] Step 2: Adjust the initial similarity of each candidate text based on the language of each candidate text to obtain the similarity of each candidate text.

[0093] For example, when the retrieval results of different recall paths include candidate texts in multiple languages, the similarity determination results may be biased due to the differences in language types during the similarity determination process, which in turn affects the subsequent clustering results. Therefore, the similarity of the corresponding candidate texts can be adaptively adjusted according to the language type of each candidate text.

[0094] Specifically, a word vector model can be used to convert the information to be retrieved and each candidate text into corresponding vectors. Then, a cosine similarity algorithm can be used to calculate the similarity between the information to be retrieved and each candidate text, and this similarity is determined as the initial similarity between the information to be retrieved and the corresponding candidate text. .

[0095] Furthermore, a similarity adjustment coefficient is determined based on the language of the candidate texts, and then the initial similarity of the corresponding candidate texts is adjusted according to the similarity adjustment coefficient. Adjustments are made to obtain the similarity score for each candidate text. Specifically, the similarity score between the information to be retrieved and the candidate text can be determined by the following expression:

[0096] (2)

[0097] in, For the first Candidate texts and information to be retrieved The initial similarity can be determined based on the information to be retrieved and the first... The cosine similarity between candidate texts is determined. For the first The similarity adjustment coefficients corresponding to the language types of the candidate texts, and the similarity adjustment coefficients corresponding to different language types. Different, for example, when the first When the language of the candidate text is Chinese, the corresponding =1; when the first When the language of the candidate text is English, the corresponding It is 0.9; when the first When the language of the candidate text is Japanese, the corresponding The value is 0.8. It should be noted that this application only considers candidate texts in different languages. Using 1, 0.9, and 0.8 as examples, we will illustrate how different languages ​​correspond to different values ​​in practical applications. The values ​​can be adjusted adaptively according to the actual situation, and no restrictions are imposed here.

[0098] In the above implementation process, the similarity of candidate texts is adaptively adjusted according to different language types, so that the similarity determination process can be applied to multiple languages, which facilitates the improvement of the accuracy of subsequent clustering results.

[0099] In one embodiment, determining clustering parameters based on semantic density includes:

[0100] Step 1: Determine the target clustering parameter generation rule from the clustering parameter rules based on the range of values ​​of semantic density.

[0101] Among them, the clustering parameter rules include the clustering parameter generation rules corresponding to different ranges of semantic density.

[0102] Step 2: Generate clustering parameters based on the target clustering parameter generation rules.

[0103] For example, clustering parameters can be determined based on semantic density. Specifically, clustering parameter rules can be set in the server, which can include clustering parameter generation rules corresponding to different ranges of semantic density values. As an example, the clustering parameter rules can be determined by the following expression:

[0104] (3)

[0105] Where ε is the clustering parameter.

[0106] Based on the clustering parameter rules described above, the target clustering parameter generation rule can be determined from the clustering parameter rules according to the range of values ​​for the semantic density. Furthermore, the value of the clustering parameter ε is determined based on the target clustering parameter generation rule.

[0107] In the above implementation process, the target clustering parameter generation rule can be determined from the clustering parameter rules based on the range of values ​​of semantic density, so that the clustering parameters can be accurately determined according to the target clustering parameter generation rule.

[0108] In one embodiment, deduplication of all candidate texts is performed based on clustering parameters to obtain multiple deduplicated candidate texts, which may include the following steps:

[0109] Step 1: Determine the cluster radius based on the clustering parameters.

[0110] Step 2: Based on the cluster radius, use the density algorithm of cluster analysis to cluster all candidate texts to obtain multiple deduplicated candidate texts.

[0111] For example, the clustering parameter can be determined as the cluster radius, and a density algorithm for cluster analysis, such as the DBSCAN algorithm, can be used to cluster all candidate texts according to the cluster radius. As an example, the minimum number of samples for the DBSCAN algorithm can be 5, i.e., minPts=5. It should be noted that this application only uses minPts=5 as an example for illustration. In actual application, minPts can also be 6 or 7. This number can be adaptively set according to the actual scenario, and is not limited here.

[0112] Specifically, a similarity matrix of all candidate texts can be constructed, and the DBSCAN algorithm can be used for clustering. The top K texts in each cluster are retained, and the retained texts are determined as candidate texts after deduplication, thus realizing the deduplication of multiple candidate texts.

[0113] As another example, when semantic density When the value is less than 0.2, the k-means clustering algorithm (K-means) can be used to cluster all candidate texts, with a K value of 5, thereby obtaining multiple deduplicated candidate texts, which can effectively avoid semantic density. The problem of inaccurate clustering results caused by excessively small cluster sizes.

[0114] In the above implementation process, the density algorithm of cluster analysis is used to cluster all candidate texts according to the cluster radius, which effectively removes duplicates from multiple candidate texts. Furthermore, the cluster radius is adaptively adjusted according to the information to be retrieved and all candidate texts, which can effectively improve the accuracy of deduplication.

[0115] In one embodiment, determining the search results corresponding to the information to be retrieved based on all deduplicated candidate texts may include the following steps:

[0116] Step 1: Select any one of the multiple deduplicated candidate texts as the target candidate text after deduplication.

[0117] Step 2: Determine the time decay parameter of the target candidate text after deduplication.

[0118] Step 3: Based on the time decay parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved, determine the comprehensive evaluation value of the deduplicated target candidate text.

[0119] Step 4: Based on the comprehensive evaluation value of each deduplicated candidate text, sort the multiple deduplicated candidate texts to obtain the search results corresponding to the information to be searched.

[0120] For example, any one of the multiple deduplicated candidate texts is determined as the target candidate text after deduplication.

[0121] Furthermore, a timeliness decay parameter for the deduplicated target candidate text can be determined based on its timeliness. Then, based on the timeliness decay parameter and the similarity between the deduplicated target candidate text and the information to be retrieved, a comprehensive evaluation value for the deduplicated target candidate text is determined. As an example, the comprehensive evaluation value for the deduplicated candidate text can be determined by the following expression:

[0122] (4)

[0123] Wherein, Score(i) is the comprehensive evaluation value of the i-th deduplicated candidate text; q is the information to be retrieved; di is the i-th deduplicated candidate text; sim(q, di) is the similarity between the i-th deduplicated candidate text and the information to be retrieved; ωt(i) is the time decay parameter of the i-th deduplicated candidate text.

[0124] Furthermore, this method can be used to determine the comprehensive evaluation value of all deduplicated candidate texts, and sort them in descending order of their comprehensive evaluation values. The sorted results are then used as the search results corresponding to the information to be retrieved.

[0125] In the above implementation process, based on the timeliness parameter of the deduplicated candidate text and the similarity between the deduplicated candidate text and the information to be retrieved, the comprehensive evaluation value of the deduplicated target candidate text is determined. This can effectively evaluate the importance of each deduplicated target candidate text and obtain the comprehensive evaluation value of each deduplicated candidate text. This facilitates the sorting of multiple deduplicated candidate texts based on the comprehensive evaluation value of each deduplicated candidate text, thereby obtaining the retrieval results corresponding to the information to be retrieved. This effectively improves the accuracy of sorting multiple deduplicated candidate texts and thus improves the accuracy of the retrieval results.

[0126] In one embodiment, determining the time decay parameter of the deduplicated target candidate text may include the following steps:

[0127] Step 1: Determine the timeliness parameter of the deduplicated target candidate text based on the query time of the information to be retrieved and the generation time of the deduplicated target candidate text.

[0128] Step 2: Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameters, determine the timeliness decay parameters of the deduplicated target candidate text.

[0129] For example, the timeliness parameter Δt of the deduplicated target candidate text can be determined based on the query time of the information to be retrieved and the generation time of the deduplicated target candidate text. Specifically, the timeliness parameter Δt of the deduplicated target candidate text can be the number of days between the query time of the information to be retrieved and the generation time of the deduplicated target candidate text.

[0130] Different text types correspond to different time decay periods; therefore, the time decay parameters for deduplicated candidate texts also differ for different text types. The time decay parameters for deduplicated target candidate texts can be determined based on the text type and corresponding time decay parameters. As an example, the time decay parameter ωt(i) for deduplicated candidate texts of different text types can be determined using the following expression:

[0131] (5)

[0132] Where ωt(i) is the time decay parameter of the i-th deduplicated candidate text; is the timeliness parameter for the i-th deduplicated candidate text.

[0133] The calculation rule for the time-decrease parameter of the deduplicated target candidate text can be determined from the above expression based on the text type of the deduplicated target candidate text, and the time-decrease parameter of the deduplicated target candidate text can be determined based on the determined calculation rule and the corresponding time-decrease parameter.

[0134] In the above implementation process, the time decay parameters of different deduplicated candidate texts are calculated according to different text types. This can accurately assess the impact of different text types on the search results and effectively avoid the problem of mismatch between the time decay results and the actual decay period of the text caused by setting fixed time decay coefficients for different types of text. This improves the accuracy of text time decay processing and thus improves the accuracy of search result ranking.

[0135] In one embodiment, determining the timeliness decay parameter of the deduplicated target candidate text based on the text type and corresponding timeliness parameter of the deduplicated target candidate text may include the following steps:

[0136] Step 1: Based on the text type and corresponding timeliness parameters of the deduplicated target candidate text, determine the initial timeliness decay parameter of the deduplicated target candidate text.

[0137] Step 2: Determine the language confidence level based on the language type of the deduplicated target candidate text.

[0138] Step 3: Adjust the initial time-delay parameter based on language confidence to obtain the time-delay parameter.

[0139] For example, in real-world scenarios, candidate texts can be in multiple languages, such as Chinese, Japanese, and English. When the model filters different languages, the accuracy of language translation has a certain impact on the time-depletion parameter. In order to reduce the impact of different languages ​​on the time-depletion parameter, the time-depletion parameter can be adaptively adjusted according to the language.

[0140] Specifically, when the model performs language recognition on text, it usually translates the text into Chinese. Therefore, the time decay parameter of the Chinese text can be determined by the above expression (5), and this time decay parameter is determined as the initial time decay parameter ωt'. That is, the initial time decay parameter ωt'(i) when translating the deduplicated target candidate text into Chinese is determined by the above expression (5).

[0141] Furthermore, the language confidence of the deduplicated target candidate text is determined based on its actual language type, and the initial time-decrease parameter is adjusted according to this language confidence to obtain the actual time-decrease parameter of the deduplicated target candidate text. As an example, the actual time-decrease parameter ωt(i) of the deduplicated candidate text can be determined by the following expression:

[0142] (6)

[0143] Wherein, ωt'(i) is the initial time decay parameter when the i-th deduplicated candidate text is translated into Chinese; Let be the language confidence score for the i-th deduplicated candidate text. As an example, when the actual language of the deduplicated candidate text is Chinese, It can be set to 0; when the actual language of the deduplicated candidate text is English, It can be set to 0.1 when the actual language of the deduplicated candidate text is Japanese. It can be set to 0.2. It should be noted that this application only uses the above values ​​as an example of language confidence for different languages. In practical applications, the language confidence values ​​for different languages ​​can be adaptively set according to different language types, and no restrictions are imposed here. Therefore, the time decay parameter of the deduplicated candidate text for different language types can be determined by the above expression (6).

[0144] In the above implementation process, the corresponding language confidence is determined according to the language type of the deduplicated candidate text, and the initial time-attenuation parameter is adjusted by the language confidence. This makes the adjusted time-attenuation parameter applicable to deduplicated candidate texts of different language types, thus improving the accuracy of the time-attenuation parameter determination for different language types.

[0145] In one embodiment, determining the comprehensive evaluation value of the deduplicated target candidate text based on the time decay parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved may include the following steps:

[0146] Step 1: Determine the popularity value of the target candidate text after deduplication.

[0147] Step 2: Based on the time-decrease parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value, determine the comprehensive evaluation value of the deduplicated target candidate text.

[0148] For example, when determining the overall evaluation value of the deduplicated candidate text, the popularity value of the corresponding deduplicated candidate text can also be used as an evaluation factor. Specifically, the popularity value of the deduplicated target candidate text can be determined based on the number of clicks it receives. As an example, the popularity value Sclick of the deduplicated target candidate text can be determined using the following expression:

[0149] (7)

[0150] Where Sclick(i) is the popularity value of the i-th deduplicated candidate text, and Nclick(i) is the number of times the i-th deduplicated candidate text is clicked in the knowledge base.

[0151] Furthermore, based on the timeliness decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value, the comprehensive evaluation value of the deduplicated target candidate text is determined. As an example, the comprehensive evaluation value of the deduplicated candidate text can be determined by the following expression:

[0152] (8)

[0153] In the above implementation process, the popularity value of the deduplicated candidate text is taken into consideration as a factor in the comprehensive evaluation value. This can effectively prioritize the display of recall results that are currently popular and match the information to be retrieved in the search results. The intelligent push of search results is realized based on the popularity value of the deduplicated candidate text.

[0154] In one embodiment, determining the comprehensive evaluation value of the deduplicated target candidate text based on the timeliness decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value may include the following steps:

[0155] Step 1: Determine the recall path to which the deduplicated target candidate text belongs.

[0156] Step 2: Determine the path parameters of the deduplicated target candidate text based on the path weights corresponding to the recall paths.

[0157] Step 3: Based on the time decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, the popularity value, and the path parameters, determine the comprehensive evaluation value of the deduplicated target candidate text.

[0158] For example, when determining the comprehensive evaluation value of the deduplicated candidate text, the recall path of the deduplicated candidate text can also be used as a reference factor, thereby realizing the intelligent fusion of multi-path recall strategies. Specifically, the recall path to which the deduplicated target candidate text belongs can be determined, and the path weight corresponding to the recall path can be determined as the path parameter of the deduplicated target candidate text.

[0159] Furthermore, based on the time-decrease parameter of the deduplicated target candidate text, the similarity and popularity value between the deduplicated target candidate text and the information to be retrieved, and the path parameters, the comprehensive evaluation value of the deduplicated target candidate text is determined. As an example, the comprehensive evaluation value of the deduplicated candidate text can be determined by the following expression:

[0160] (9)

[0161] Where L(i) is the path parameter of the i-th deduplicated candidate text.

[0162] In the above implementation process, the corresponding path parameters are determined according to the path weight of the recall path to which the deduplicated candidate text belongs, and the path parameters are used as parameter factors to determine the comprehensive evaluation value of the deduplicated candidate text. This can effectively realize the dynamic fusion of multi-path recall strategies, avoid the problem of strategy mismatch caused by static binding of recall strategies, and effectively improve the accuracy of the comprehensive evaluation value determination.

[0163] In one embodiment, before determining the path parameters of the deduplicated target candidate text based on the path weight corresponding to the recall path, the following steps may be included:

[0164] Step 1: Determine the number of time-related keywords, the number of technical terminology keywords, and the total number of words in the information to be retrieved.

[0165] Step 2: Determine the time sensitivity factor based on the number of time-related keywords and the total number of words.

[0166] Step 3: Determine the technical depth factor based on the number of technical term keywords and the total number of words.

[0167] Step 4: Based on the time sensitivity factor and the technical depth factor, determine the path weight corresponding to each recall path in the multi-path recall strategy.

[0168] For example, the number of time-related keywords, the number of technical term keywords, and the total number of words in the information to be retrieved can be determined separately.

[0169] Further, the timeliness sensitivity factor is determined based on the number of time-related keywords and the total number of keywords. As an example, the timeliness sensitivity factor... It can be determined by the following expression:

[0170] =Number of time-related keywords / Total number of keywords (10)

[0171] The technical depth factor is determined based on the number of technical terminology keywords and the total number of words. As an example, the technical depth factor... It can be determined by the following expression:

[0172] =Number of technical terminology keywords / Total number of words (11)

[0173] Furthermore, the path weight corresponding to each recall path in the multi-path recall strategy can be determined based on the timeliness sensitivity factor and the technical depth factor. The multi-path recall strategy includes the recall strategy of semantic vector retrieval, the recall strategy of keyword retrieval, and the recall strategy of hypothetical question recall.

[0174] Specifically, the path weights corresponding to the recall paths in semantic vector retrieval can be determined based on the technology depth factor. The path weight corresponding to the recall path of keyword retrieval is determined based on the time sensitivity factor. And based on the path weights corresponding to the recall paths retrieved by semantic vectors. and the path weight corresponding to the recall path of keyword retrieval. Determine the path weights corresponding to the recall paths for hypothetical question recall. .

[0175] As an example, the path weights corresponding to the recall paths in semantic vector retrieval. It can be determined by the following expression:

[0176] (12)

[0177] Path weight corresponding to the recall path of keyword retrieval It can be determined by the following expression:

[0178] (13)

[0179] Path weights corresponding to the recall paths in hypothetical question recall It can be determined by the following expression:

[0180] (14)

[0181] That is, L(i) in the above expression (9) is , or .

[0182] In the above implementation process, the technical depth factor and the time sensitivity factor are determined based on the number of time keywords, the number of professional term keywords and the total number of words in the information to be retrieved. This allows for the accurate determination of the path weight corresponding to each recall path based on the technical depth factor and the time sensitivity factor, effectively realizing the dynamic allocation of path weights and facilitating the subsequent determination of the corresponding path parameters based on the path weights.

[0183] Figure 3 This is a flowchart illustrating an application example of an information retrieval method provided in this application embodiment, such as... Figure 3 As shown, the flowchart includes the following steps:

[0184] Step 301: Trigger a multi-path recall strategy based on the information to be retrieved.

[0185] Specifically, when the total number of words in the information to be retrieved is greater than or equal to 3 and includes at least one technical term keyword, a multi-path recall strategy is triggered. Recall paths via semantic vector retrieval, keyword retrieval, and hypothetical question retrieval are respectively searched in the enterprise knowledge base, yielding 50 candidate texts related to the information to be retrieved for each path, resulting in a total of 150 candidate texts. These 150 candidate texts may contain duplicates.

[0186] Step 302: Determine semantic density.

[0187] Specifically, the semantic density is determined based on the average similarity between all candidate texts and the information to be retrieved. All candidate texts and the information to be retrieved are transformed into corresponding vectors, and the similarity between the vector corresponding to each candidate text and the vector corresponding to the information to be retrieved is calculated using a cosine similarity algorithm. The average of all similarities is then used to determine the semantic density. .

[0188] Step 303: Determine the clustering parameters based on semantic density, and use the density algorithm of cluster analysis to cluster all candidate texts according to the clustering parameters to obtain deduplicated candidate texts.

[0189] Specifically, the clustering parameter ε can be determined using the above expression (3), and the DBSCAN algorithm can be used to cluster all candidate texts by using the clustering parameter ε as the clustering radius and retaining the first 3 documents of each cluster. If 32 effective clusters are included in the 150 candidate texts, the first 3 texts in each cluster will be used as deduplicated candidate texts, thus obtaining 96 deduplicated candidate texts.

[0190] Step 304: Determine the overall evaluation value of each deduplicated candidate text.

[0191] Specifically, the comprehensive evaluation value of each deduplicated candidate text can be determined based on the timeliness decay parameter of each deduplicated candidate text, the similarity between the deduplicated candidate text and the information to be retrieved, the popularity value of each deduplicated candidate text, and the path weight corresponding to each deduplicated candidate text.

[0192] For example, the comprehensive evaluation value of each deduplicated candidate text can be determined by the above expression (9).

[0193] Step 305: Sort the deduplicated candidate texts according to the comprehensive evaluation value of each deduplicated candidate text to obtain the search results corresponding to the information to be searched.

[0194] Specifically, all deduplicated candidate texts are sorted from highest to lowest based on their comprehensive evaluation scores. The sorted results are the search results corresponding to the information to be retrieved.

[0195] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0196] Based on the same inventive concept, this application also provides an information retrieval device for implementing the information retrieval method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more information retrieval device embodiments provided below can be found in the limitations of the information retrieval method described above, and will not be repeated here.

[0197] In one exemplary embodiment, Figure 4 This is a schematic diagram of the structure of an information retrieval device provided in an embodiment of this application, such as... Figure 4 As shown, the device may include:

[0198] Information acquisition module 401 is used to acquire the user's search information;

[0199] The retrieval module 402 is used to retrieve multiple candidate texts by using a multi-path recall strategy to search the retrieval information database based on the information to be retrieved.

[0200] The semantic density determination module 403 is used to determine the semantic density based on the similarity between the information to be retrieved and all candidate texts;

[0201] Clustering parameter determination module 404 is used to determine clustering parameters based on semantic density;

[0202] The deduplication module 405 is used to deduplicatize all candidate texts based on clustering parameters to obtain multiple deduplicated candidate texts.

[0203] The retrieval result determination module 406 is used to determine the retrieval results corresponding to the information to be retrieved based on all deduplicated candidate texts.

[0204] In one embodiment, the semantic density determination module 403 is specifically used for:

[0205] Determine the similarity between the information to be retrieved and each candidate text;

[0206] The average of all similarities is used to determine the semantic density.

[0207] In one embodiment, the semantic density determination module 403 is specifically used for:

[0208] The cosine similarity between the information to be retrieved and each candidate text is determined as the initial similarity of the corresponding candidate text.

[0209] The initial similarity of each candidate text is adjusted based on its language type to obtain the similarity of each candidate text.

[0210] In one embodiment, the clustering parameter determination module 404 is specifically used for:

[0211] The target clustering parameter generation rule is determined from the clustering parameter rules based on the range of values ​​of semantic density. The clustering parameter rules include the clustering parameter generation rules corresponding to different ranges of semantic density.

[0212] Clustering parameters are generated based on the target clustering parameter generation rules.

[0213] In one embodiment, the deduplication module 405 is specifically used for:

[0214] Determine the cluster radius based on clustering parameters;

[0215] A density algorithm based on clustering radius is used to cluster all candidate texts, resulting in multiple deduplicated candidate texts.

[0216] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0217] Select any one of the multiple deduplicated candidate texts as the target candidate text after deduplication;

[0218] Determine the time decay parameter for the target candidate text after deduplication;

[0219] Based on the time decay parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0220] Based on the comprehensive evaluation value of each deduplicated candidate text, the multiple deduplicated candidate texts are sorted to obtain the retrieval results corresponding to the information to be retrieved.

[0221] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0222] Based on the query time of the information to be retrieved and the generation time of the deduplicated target candidate text, the timeliness parameter of the deduplicated target candidate text is determined.

[0223] Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameters, the timeliness decay parameters of the deduplicated target candidate text are determined.

[0224] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0225] Based on the text type and corresponding timeliness parameters of the deduplicated target candidate text, the initial timeliness decay parameter of the deduplicated target candidate text is determined.

[0226] Language confidence is determined based on the language category of the deduplicated target candidate text;

[0227] The initial aging decay parameter is adjusted based on language confidence to obtain the aging decay parameter.

[0228] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0229] Determine the popularity value of the target candidate text after deduplication;

[0230] Based on the timeliness decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0231] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0232] Determine the recall path to which the deduplicated target candidate text belongs;

[0233] The path parameters of the deduplicated target candidate text are determined based on the path weights corresponding to the recall paths.

[0234] Based on the time decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, the popularity value, and the path parameters, the comprehensive evaluation value of the deduplicated target candidate text is determined.

[0235] In one embodiment, the retrieval result determination module 406 is specifically used for:

[0236] Determine the number of time-related keywords, the number of technical terminology keywords, and the total number of words in the information to be retrieved;

[0237] Time sensitivity factors are determined based on the number of time-related keywords and the total number of words.

[0238] The technology depth factor is determined based on the number of technical terminology keywords and the total number of words.

[0239] Based on time sensitivity factors and technical depth factors, the path weight corresponding to each recall path in the multi-path recall strategy is determined.

[0240] Each module in the aforementioned information retrieval device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0241] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. Figure 5This is a schematic diagram of the internal structure of a computer device according to an embodiment of this application. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of this computer device provides computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of this computer device stores retrieval information and candidate text data. The I / O interfaces of this computer device are used for exchanging information between the processor and external devices. The communication interface of this computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an information retrieval method.

[0242] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0243] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the information retrieval method described in any of the above embodiments.

[0244] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the information retrieval method described in any of the above embodiments.

[0245] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0246] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0247] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0248] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An information retrieval method, characterized in that, The method includes: Obtain the user's search information; Based on the information to be retrieved, a multi-path recall strategy is used to search the information database separately to obtain multiple candidate texts. The semantic density is determined based on the similarity between the information to be retrieved and all the candidate texts. Clustering parameters are determined based on the semantic density; Based on the clustering parameters, all candidate texts are deduplicated to obtain multiple deduplicated candidate texts. The search results corresponding to the information to be searched are determined based on all the deduplicated candidate texts.

2. The method according to claim 1, characterized in that, The step of determining semantic density based on the similarity between the information to be retrieved and all the candidate texts includes: Determine the similarity between the information to be retrieved and each of the candidate texts; The average of all similarities is used to determine the semantic density.

3. The method according to claim 2, characterized in that, Determining the similarity between the information to be retrieved and each candidate text includes: The cosine similarity between the information to be retrieved and each candidate text is determined as the initial similarity of the corresponding candidate text. The initial similarity of each candidate text is adjusted based on its language type to obtain the similarity of each candidate text.

4. The method according to claim 1, characterized in that, The determination of clustering parameters based on the semantic density includes: Based on the range of values ​​of the semantic density, the target clustering parameter generation rule is determined from the clustering parameter rules, which include the clustering parameter generation rules corresponding to different ranges of semantic density. The clustering parameters are generated based on the target clustering parameter generation rules.

5. The method according to claim 1, characterized in that, The process of deduplicating all candidate texts based on the clustering parameters yields multiple deduplicated candidate texts, including: The cluster radius is determined based on the clustering parameters; Based on the cluster radius, a density algorithm for cluster analysis is used to cluster all the candidate texts to obtain the multiple deduplicated candidate texts.

6. The method according to claim 1, characterized in that, Based on all the deduplicated candidate texts, the search results corresponding to the information to be searched are determined, including: Any one of the multiple deduplicated candidate texts is determined as the target candidate text after deduplication; Determine the time decay parameter of the deduplicated target candidate text; Based on the time decay parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved, the comprehensive evaluation value of the deduplicated target candidate text is determined. Based on the comprehensive evaluation value of each deduplicated candidate text, the multiple deduplicated candidate texts are sorted to obtain the retrieval results corresponding to the information to be retrieved.

7. The method according to claim 6, characterized in that, The determination of the time-decrease parameter of the deduplicated target candidate text includes: Based on the query time of the information to be retrieved and the generation time of the deduplicated target candidate text, the timeliness parameter of the deduplicated target candidate text is determined; Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameter, the timeliness decay parameter of the deduplicated target candidate text is determined.

8. The method according to claim 7, characterized in that, The step of determining the timeliness decay parameter of the deduplicated target candidate text based on the text type and corresponding timeliness parameter of the deduplicated target candidate text includes: Based on the text type of the deduplicated target candidate text and the corresponding timeliness parameter, the initial timeliness decay parameter of the deduplicated target candidate text is determined. The language confidence level is determined based on the language type of the deduplicated target candidate text; The initial time-decrease parameter is adjusted based on the language confidence level to obtain the time-decrease parameter.

9. The method according to claim 6, characterized in that, The determination of the comprehensive evaluation value of the deduplicated target candidate text based on the time-decrease parameter of the deduplicated target candidate text and the similarity between the deduplicated target candidate text and the information to be retrieved includes: Determine the popularity value of the deduplicated target candidate text; Based on the time-effect decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value, the comprehensive evaluation value of the deduplicated target candidate text is determined.

10. The method according to claim 9, characterized in that, The determination of the comprehensive evaluation value of the deduplicated target candidate text based on the time-effect decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, and the popularity value includes: Determine the recall path to which the deduplicated target candidate text belongs; The path parameters of the deduplicated target candidate text are determined based on the path weight corresponding to the recall path. Based on the time decay parameter of the deduplicated target candidate text, the similarity between the deduplicated target candidate text and the information to be retrieved, the popularity value, and the path parameters, the comprehensive evaluation value of the deduplicated target candidate text is determined.

11. The method according to claim 10, characterized in that, Before determining the path parameters of the deduplicated target candidate text based on the path weight corresponding to the recall path, the method further includes: Determine the number of time-related keywords, the number of technical terminology keywords, and the total number of words in the information to be retrieved; The timeliness sensitivity factor is determined based on the number of time-related keywords and the total number of words. The technology depth factor is determined based on the number of technical terminology keywords and the total number of words. Based on the time sensitivity factor and the technology depth factor, the path weight corresponding to each recall path in the multi-path recall strategy is determined.

12. An information retrieval device, characterized in that, The device includes: The information acquisition module is used to acquire the user's search information; The retrieval module is used to search the retrieval information database separately based on the information to be retrieved, using a multi-path recall strategy to obtain multiple candidate texts; A semantic density determination module is used to determine semantic density based on the similarity between the information to be retrieved and all the candidate texts; A clustering parameter determination module is used to determine clustering parameters based on the semantic density; The deduplication module is used to perform deduplication processing on all the candidate texts based on the clustering parameters to obtain multiple deduplicated candidate texts. The retrieval result determination module is used to determine the retrieval result corresponding to the information to be retrieved based on all the deduplicated candidate texts.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images