Retrieval method and device, electronic equipment and storage medium

Abstract

The invention relates to a retrieval method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a plurality of first retrieval results in response to a retrieval request; the retrieval request comprises a first-class request or a second-class request, the first-class request is used for requesting to retrieve data under the same data class, and the second-class request is used for requesting to retrieve data under multiple data classes; based on the retrieval request and the data category to which each first retrieval result belongs, sorting the plurality of first retrieval results to obtain a sorting sequence of the plurality of first retrieval results; the first retrieval results under different data categories are arranged according to the data category requested by the retrieval request, and the first retrieval results under each data category are arranged according to the first correlation with the retrieval request; and selecting a target retrieval result from the plurality of first retrieval results based on the arrangement sequence. Based on the data category sorting of the retrieval results, the relationship between different retrieval results is considered, and the retrieval accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information retrieval technologies, and in particular, to a retrieval method, apparatus, electronic device, and storage medium. Background Art

[0002] In the field of information retrieval technologies, re-ranking refers to using a model with higher computational complexity but better performance to re-arrange the coarse-grained retrieval results to improve the retrieval accuracy.

[0003] In related technologies, according to a user's retrieval request, coarse-grained retrieval results are retrieved, and then the retrieval request is respectively concatenated with each retrieval result and input into a cross-encoder-based re-ranking model to obtain the correlation between each retrieval result and the retrieval request, so as to rank the coarse-grained retrieval results based on the correlation to determine the fine-grained retrieval results. This method treats each retrieval result as independent data during ranking and does not consider the relationship between different retrieval results, resulting in poor retrieval accuracy. Summary of the Invention

[0004] To overcome the problems in related technologies, the present disclosure provides a retrieval method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a retrieval method is provided, and the method includes:

[0006] In response to a retrieval request, obtain a plurality of first retrieval results; wherein, the retrieval request includes a first type of request or a second type of request, the first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories;

[0007] Based on the retrieval request and the data category to which each first retrieval result belongs, rank the plurality of first retrieval results to obtain an arrangement order of the plurality of first retrieval results; wherein, the first retrieval results under different data categories are arranged according to the data categories requested by the retrieval request, and the first retrieval results under each data category are arranged according to a first correlation with the retrieval request;

[0008] Based on the arrangement order, select a target retrieval result from the plurality of first retrieval results.

[0009] In some embodiments, the obtaining a plurality of first retrieval results in response to a retrieval request includes:

[0010] In response to the retrieval request, retrieve a plurality of second retrieval results from a retrieval database;

[0011] Determine a second correlation between the retrieval request and each second retrieval result;

[0012] Select the plurality of first retrieval results from the plurality of second retrieval results based on the second relevance corresponding to each of the second retrieval results.

[0013] In some embodiments, the selecting the plurality of first retrieval results from the plurality of second retrieval results based on the second relevance corresponding to each of the second retrieval results includes:

[0014] Select the plurality of first retrieval results from the plurality of second retrieval results based on the second relevance corresponding to each of the second retrieval results and the data set to which each of the second retrieval results belongs;

[0015] Wherein, the data set includes a first data set and a second data set, the data in the first data set is data for training a target model, the target model includes the model used in the retrieval process, and the data in the second data set is data in the retrieval database other than the data in the first data set.

[0016] In some embodiments, the selecting the plurality of first retrieval results from the plurality of second retrieval results based on the second relevance corresponding to each of the second retrieval results and the data set to which each of the second retrieval results belongs includes:

[0017] For each of the second retrieval results,

[0018] When the second retrieval result belongs to the first data set and the second relevance corresponding to the second retrieval result is greater than a first threshold, determine the second retrieval result as the first retrieval result; or,

[0019] When the second retrieval result belongs to the second data set and the second relevance corresponding to the second retrieval result is greater than a second threshold, determine the second retrieval result as the first retrieval result;

[0020] Wherein, the second threshold is less than the first threshold.

[0021] In some embodiments, the sorting the plurality of first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs to obtain the arrangement order of the plurality of first retrieval results includes:

[0022] When the retrieval request is the first type of request, select at least one third retrieval result belonging to the target data category from the plurality of first retrieval results, and determine the other first retrieval results in the plurality of first retrieval results except the third retrieval result as the fourth retrieval results, where the target data category is the data category requested by the retrieval request;

[0023] Sort based on the at least one third retrieval result and the at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the third retrieval results are arranged before the fourth retrieval results.

[0024] In some embodiments, before sorting based on the at least one third retrieval result and the at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results, the method further includes:

[0025] Obtain data belonging to the second data set from the retrieval database; wherein, the data sets to which the data in the retrieval database belong include a first data set and the second data set, the data in the first data set are data for training a target model, the target model includes the model used in the retrieval process, and the data in the second data set are the data in the retrieval database other than the data in the first data set;

[0026] Determine the data in the second data set that belongs to the target data category and is not included in the multiple first retrieval results as the third retrieval results.

[0027] In some embodiments, sorting based on the at least one third retrieval result and the at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results includes:

[0028] Sort the at least one third retrieval result to obtain a first arrangement order, and sort the at least one fourth retrieval result to obtain a second arrangement order;

[0029] Concatenate the first arrangement order and the second arrangement order to obtain the arrangement order of the multiple first retrieval results.

[0030] In some embodiments, sorting the multiple first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs to obtain the arrangement order of the multiple first retrieval results includes:

[0031] When the retrieval request is the second type of request, determine at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, each first retrieval result in the connected set belongs to the same data category as at least one other first retrieval result in the same connected set, or the semantic similarity between each first retrieval result in the connected set and at least one other first retrieval result in the same connected set is greater than a third threshold;

[0032] Select at least one fifth retrieval result from the first retrieval results in each connected set, and determine the other first retrieval results among the multiple first retrieval results except the fifth retrieval result as the sixth retrieval result; the first relevance corresponding to the selected fifth retrieval result is greater than the first relevance corresponding to the unselected first retrieval results in the connected set.

[0033] Sort based on the multiple fifth retrieval results and at least one sixth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the fifth retrieval result is arranged before the sixth retrieval result.

[0034] In some embodiments, the determining at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs includes:

[0035] Construct a connection relationship graph based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, the connection relationship graph includes multiple nodes and multiple connection lines, the nodes are used to represent the first retrieval results, and the connection line between two nodes is used to represent that the two first retrieval results corresponding to the two nodes belong to the same data category, or the semantic similarity between the two first retrieval results corresponding to the two nodes is greater than the third threshold.

[0036] Add the first retrieval results corresponding to the nodes directly or indirectly connected by the connection lines in the connection relationship graph to the same connected set.

[0037] In some embodiments, the sorting based on the multiple fifth retrieval results and at least one sixth retrieval result to obtain the arrangement order of the multiple first retrieval results includes:

[0038] Sort the multiple fifth retrieval results to obtain a third arrangement order, and sort the at least one sixth retrieval result to obtain a fourth arrangement order;

[0039] Concatenate the third arrangement order and the fourth arrangement order to obtain the arrangement order of the multiple first retrieval results.

[0040] According to the second aspect of the embodiments of the present disclosure, there is provided a retrieval device, the device includes:

[0041] A first result acquisition module, configured to acquire a plurality of first retrieval results in response to a retrieval request; wherein, the retrieval request includes a first type of request or a second type of request, the first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories.

[0042] A result sorting module, configured to sort the multiple first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs, to obtain the arrangement order of the multiple first retrieval results; wherein, the first retrieval results under different data categories are arranged according to the data category requested by the retrieval request, and the first retrieval results under each data category are arranged according to the first relevance to the retrieval request;

[0043] A target result acquisition module, configured to select a target retrieval result from the multiple first retrieval results based on the arrangement order.

[0044] In some embodiments, the first result acquisition module is configured to:

[0045] In response to the retrieval request, retrieve multiple second retrieval results from a retrieval database;

[0046] Determine the second relevance between the retrieval request and each second retrieval result;

[0047] Based on the second relevance corresponding to each second retrieval result, select the multiple first retrieval results from the multiple second retrieval results.

[0048] In some embodiments, the first result acquisition module is configured to:

[0049] Based on the second relevance corresponding to each second retrieval result and the data set to which each second retrieval result belongs, select the multiple first retrieval results from the multiple second retrieval results;

[0050] Wherein, the data set includes a first data set and a second data set, the data in the first data set is data for training a target model, the target model includes the model used in the retrieval process, and the data in the second data set is the data in the retrieval database other than the data in the first data set.

[0051] In some embodiments, the first result acquisition module is configured to:

[0052] For each second retrieval result,

[0053] When the second retrieval result belongs to the first data set and the second relevance corresponding to the second retrieval result is greater than a first threshold, determine the second retrieval result as the first retrieval result; or,

[0054] When the second retrieval result belongs to the second data set and the second relevance corresponding to the second retrieval result is greater than a second threshold, determine the second retrieval result as the first retrieval result;

[0055] Wherein, the second threshold is less than the first threshold.

[0056] In some embodiments, the result sorting module is configured to:

[0057] When the retrieval request is the first type of request, select at least one third retrieval result belonging to the target data category from the multiple first retrieval results, and determine the other first retrieval results except the third retrieval result among the multiple first retrieval results as the fourth retrieval results, where the target data category is the data category requested by the retrieval request;

[0058] Sort based on the at least one third retrieval result and at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the third retrieval result is arranged before the fourth retrieval result.

[0059] In some embodiments, the result sorting module is further configured to:

[0060] Obtain the data belonging to the second data set from the retrieval database; wherein, the data sets to which the data in the retrieval database belongs include the first data set and the second data set, the data in the first data set is the data for training the target model, the target model includes the model used in the retrieval process, and the data in the second data set is the data in the retrieval database except the data in the first data set;

[0061] Determine the data belonging to the target data category in the second data set and not included in the multiple first retrieval results as the third retrieval results.

[0062] In some embodiments, the result sorting module is configured to:

[0063] Sort the at least one third retrieval result to obtain a first arrangement order, and sort the at least one fourth retrieval result to obtain a second arrangement order;

[0064] Concatenate the first arrangement order and the second arrangement order to obtain the arrangement order of the multiple first retrieval results.

[0065] In some embodiments, the result sorting module is configured to:

[0066] When the retrieval request is the second type of request, at least one connected set is determined based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, each first retrieval result in the connected set belongs to the same data category as at least one other first retrieval result in the same connected set, or the semantic similarity between each first retrieval result and at least one other first retrieval result in the same connected set is greater than a third threshold;

[0067] At least one fifth retrieval result is selected from the first retrieval results in each connected set, and the other first retrieval results among the multiple first retrieval results except the fifth retrieval result are determined as sixth retrieval results; the first relevance corresponding to the selected fifth retrieval result is greater than the first relevance corresponding to the unselected first retrieval results in the connected set;

[0068] Based on the multiple fifth retrieval results and at least one sixth retrieval result, sorting is performed to obtain the arrangement order of the multiple first retrieval results; wherein, the fifth retrieval results are arranged before the sixth retrieval results.

[0069] In some embodiments, the result sorting module is configured to:

[0070] Based on the multiple first retrieval results and the data category to which each first retrieval result belongs, a connection relationship graph is constructed; wherein, the connection relationship graph includes multiple nodes and multiple connection lines, the nodes are used to represent the first retrieval results, and the connection line between two nodes is used to represent that the two first retrieval results corresponding to the two nodes belong to the same data category, or the semantic similarity between the two first retrieval results corresponding to the two nodes is greater than the third threshold;

[0071] The first retrieval results corresponding to the nodes directly or indirectly connected by the connection lines in the connection relationship graph are added to the same connected set.

[0072] In some embodiments, the result sorting module is configured to:

[0073] Sort the multiple fifth retrieval results to obtain a third arrangement order, and sort the at least one sixth retrieval result to obtain a fourth arrangement order;

[0074] Concatenate the third arrangement order and the fourth arrangement order to obtain the arrangement order of the multiple first retrieval results.

[0075] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0076] A processor;

[0077] A memory for storing processor-executable instructions;

[0078] Wherein, the processor is configured to execute the method described in the first aspect of the embodiments of the present disclosure.

[0079] According to the fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, which enables an electronic device to execute the method described in the first aspect of the embodiments of the present disclosure when the instructions in the storage medium are executed by a processor of the electronic device.

[0080] Adopting the above method of the present disclosure has the following beneficial effects:

[0081] The method provided by the embodiments of the present disclosure first obtains a plurality of first retrieval results, and then sorts the plurality of first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs. Since the data categories requested by different retrieval requests are different, when sorting, the data category to which each first retrieval result belongs is considered, that is, the relationship between different retrieval results is considered, and each retrieval result is not regarded as independent data, thereby improving the accuracy of retrieval.

[0082] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0084] Figure 1 is a schematic diagram of an application scenario shown according to an exemplary embodiment;

[0085] Figure 2 is a flowchart of a retrieval method shown according to an exemplary embodiment;

[0086] Figure 3 is a flowchart of a retrieval method shown according to an exemplary embodiment;

[0087] Figure 4 is a schematic diagram of a relevance determination model shown according to an exemplary embodiment;

[0088] Figure 5 is a flowchart of a retrieval method shown according to an exemplary embodiment;

[0089] Figure 6 is a flowchart of a retrieval method shown according to an exemplary embodiment;

[0090] Figure 7 It is a block diagram of a retrieval device shown according to an exemplary embodiment;

[0091] Figure 8 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0092] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0093] In the field of information retrieval technology, in the related art, various retrieval models can be adopted for retrieval. For example, the retrieval model based on BM25 and the retrieval model based on the twin - tower architecture, etc. The retrieval model based on BM25 retrieves according to literal similarity and does not consider semantic similarity. While the retrieval model based on the twin - tower architecture considers semantic similarity, it is only a coarse - grained retrieval and lacks fine - grained interaction. And the re - ranking model based on the cross - encoder can re - rank the coarse - grained retrieval results to obtain fine - grained retrieval results. However, since each retrieval result is regarded as independent data during ranking and the relationship between different retrieval results is not considered, the accuracy of retrieval is poor.

[0094] The embodiments of the present disclosure provide a new retrieval method for the above - mentioned problems. First, multiple first retrieval results are obtained, and then, based on the retrieval request and the data category to which each first retrieval result belongs, the multiple first retrieval results are ranked. Since different retrieval requests request different data categories for retrieval, when ranking, the data category to which each first retrieval result belongs is considered, that is, the relationship between different retrieval results is considered, and each retrieval result is not regarded as independent data, thereby improving the accuracy of retrieval.

[0095] The method provided by the embodiments of the present disclosure can be applied to a variety of retrieval scenarios. For example, it can be applied to tool retrieval in the field of machine learning technology, literature retrieval in the field of network query, patent retrieval and other scenarios.

[0096] Taking the tool retrieval scenario as an example, tool learning is to enhance the capabilities of large language models using external tools, that is, adding the API (Application Programming Interface) documentation of the tools to the input of the large language model. Since the number of provided API documents is limited by the context length of the large language model, a major challenge in tool learning is how to support a large number of tools. To address this challenge, a tool retrieval-based method is proposed to retrieve the appropriate tool API documents from the tool library according to the retrieval request and provide them to the large language model.

[0097] See Figure 1 the schematic diagram of the application scenario shown. The user interacts with the dialogue robot server deployed with the large language model through the network. The server retrieves in the tool library according to the user's retrieval request and returns a coarse-grained retrieval result. Then, the method provided in the embodiments of the present disclosure for re-ranking the coarse-grained retrieval result is used to obtain a fine-grained retrieval result. The large language model then calls the appropriate tool API document in the tool library according to the fine-grained retrieval result, so as to provide the appropriate tool API document for the large language model. Moreover, the result of the call can be summarized and returned to the user.

[0098] The method provided in the embodiments of the present disclosure is executed by an electronic device, which can be devices such as mobile phones, tablet computers, laptop computers, and various types of servers.

[0099] Figure 2 is a flowchart of a retrieval method shown according to an exemplary embodiment, which is executed by an electronic device. See Figure 2 and this method includes the following steps:

[0100] Step S201, in response to the retrieval request, obtain multiple first retrieval results; wherein, the retrieval request includes a first type of request or a second type of request. The first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories.

[0101] Among them, the retrieval request is used to request data retrieval, and the data to be retrieved can be various types of data such as tool API documents, papers, patent documents, pictures, and voices. The retrieval request includes a first type of request or a second type of request. The first type of request is used to request to retrieve data under the same data category, that is, when the first type of request is received, it indicates that the user hopes to retrieve data under the same data category. The second type of request is used to request to retrieve data under multiple data categories, that is, when the second type of request is received, it indicates that the user hopes to retrieve data under different data categories.

[0102] The first search results are data that meet the requirements of the search request. Multiple first search results are coarse-grained search results. For example, when the search request is used to request a search for a tool with a certain function, the multiple first search results retrieved are API documents of tools with that function.

[0103] Step S202: Sort the multiple first search results based on the search request and the data category to which each first search result belongs, to obtain the arrangement order of the multiple first search results; wherein, the first search results under different data categories are arranged according to the data category requested by the search request, and the first search results under each data category are arranged according to the first relevance with the search request.

[0104] Multiple first search results are coarse-grained search results. In order to obtain fine-grained search results, it is necessary to sort the multiple first search results, so as to obtain fine-grained search results according to the arrangement order obtained after sorting.

[0105] For the sorting of multiple first search results, in the related art, the relevance between the search request and the search results is directly obtained, and then sorted according to the relevance. This method does not consider the relationship between different search results, resulting in poor search accuracy. In the embodiments of the present disclosure, the multiple first search results are sorted based on the search request and the data category to which each first search result belongs, and moreover, the first search results under different data categories are arranged according to the data category requested by the search request, and the first search results under each data category are arranged according to the first relevance with the search request. That is to say, considering the data category requested by the search request, when sorting, it is sorted not only according to the first relevance between each first search result and the search request, but also according to the data category to which each search result belongs. Among them, the greater the first relevance, the greater the possibility that the corresponding first search result is the data that the current user hopes to retrieve, and the smaller the first relevance, the smaller the possibility that the corresponding first search result is the data that the current user hopes to retrieve.

[0106] For the first type of request, which is used to request a search for data under the same data category, then when sorting the multiple first search results, the first search results under the data category requested by the first type of request should be arranged before the first search results under other data categories, and the first search results under each data category are arranged in descending order according to the corresponding first relevance.

[0107] For the second type of requests, which are used to request the retrieval of data under multiple data categories, when sorting multiple first retrieval results, the first retrieval results under the multiple data categories requested by the first type of requests need to be ranked in the front to avoid having only the first retrieval results under a certain data category in the front positions. Moreover, the first retrieval results under each data category are sorted in descending order according to the corresponding first relevance.

[0108] Of course, the above-mentioned implementation manner is sorted in the order from the data that the user most hopes to retrieve to the data that the user least hopes to retrieve (i.e., sorted in ascending order). In another embodiment, it can also be sorted in the order from the data that the user least hopes to retrieve to the data that the user most hopes to retrieve (i.e., sorted in descending order).

[0109] Step S203, select the target retrieval results from the multiple first retrieval results based on the sorting order.

[0110] Among them, the target retrieval results are fine-grained retrieval results, and according to the retrieval needs, the target retrieval results can be one or more. When the sorting order is ascending order, at least one first retrieval result starting from the first first retrieval result is used as the target retrieval result. When the sorting order is descending order, at least one first retrieval result starting from the last first retrieval result is used as the target retrieval result.

[0111] The method provided by the embodiments of the present disclosure first obtains multiple first retrieval results, and then sorts the multiple first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs. Since the data categories requested by different retrieval requests are different, when sorting, the data category to which each first retrieval result belongs is considered, that is, the relationship between different retrieval results is considered, and each retrieval result is not regarded as independent data, thereby improving the accuracy of retrieval.

[0112] In the embodiments of the present disclosure, during the retrieval process, a model will be used. For example, a model for determining the first relevance is used. Before using the model, it is necessary to first train the model with training data. For the model, when processing the training data with the model, more accurate processing results can be obtained. If the data processed by the model is not the data used during the training process, the processing results corresponding to the unused data are not as accurate as the processing results corresponding to the training data. That is to say, there will be a difference in the processing accuracy of the model for the data used for training and the data not used for training.

[0113] In the embodiments of the present disclosure, considering this situation, when retrieving the coarse-grained retrieval results (i.e., the first retrieval results), different processing will be performed based on whether the data in the retrieval database is the data used for training. The following is through Figure 3The illustrated embodiment describes the process of obtaining coarse-grained retrieval results.

[0114] Figure 3 is a flowchart of a retrieval method shown according to an exemplary embodiment, which is executed by an electronic device. Refer to Figure 3 The method includes the following steps:

[0115] Step S301, in response to a retrieval request, retrieve multiple second retrieval results from a retrieval database.

[0116] Among them, the retrieval database stores data of multiple data categories.

[0117] In some embodiments, the electronic device receives a retrieval request, which is sent to the electronic device by another electronic device. In this case, the electronic device performs a retrieval locally. After the electronic device retrieves the retrieval results, it needs to return them to the other electronic device. Alternatively, the retrieval request is a request triggered by the user in the electronic device.

[0118] In some embodiments, the retrieval request carries retrieval information, which is used to characterize the data that the user hopes to retrieve. Retrieving multiple second retrieval results from the retrieval database includes: determining a third relevance between the retrieval request and each data in the retrieval database, and based on the third relevance corresponding to each data, determining multiple second retrieval results. Optionally, select a preset number of data with the largest third relevance as the second retrieval results, or use the data with a corresponding third relevance greater than a preset value as the second retrieval results. Among them, the third relevance corresponding to the data is the third relevance between the data and the retrieval request.

[0119] Optionally, extract the retrieval information features of the retrieval information, extract the data features of each data in the retrieval database, and calculate the third relevance between the retrieval request and each data based on the retrieval information features and the data features. Or, call a relevance determination model to calculate the third relevance between the retrieval request and each data. The embodiments of the present disclosure do not limit this.

[0120] It should be noted that the implementation manner of calculating the third relevance and the implementation manner of calculating the first relevance may be the same or different. For example, the same relevance determination model can be used to determine the first relevance and the third relevance, or different relevance determination models can be used to determine the first relevance and the third relevance. For example, for the determination of the first relevance, refer to Figure 4 the schematic diagram of the relevance determination model shown in Figure 4The correlation determination model shown is a cross-encoder model based on the Transformer architecture. After concatenating the retrieval request and the first retrieval result, they are input into this correlation determination model, and the correlation between the retrieval request and the first retrieval result is output. For the determination of the third correlation, other correlation determination models can be used.

[0121] Of course, other implementation manners can also be adopted to retrieve multiple second retrieval results from the retrieval database, and the present disclosure embodiment does not limit the retrieval manner of the second retrieval results.

[0122] Step S302: Determine the second correlation between the retrieval request and each second retrieval result.

[0123] In some embodiments, if the second retrieval result is determined based on the third correlation between the retrieval request and the data in the retrieval database, then directly use the third correlation corresponding to the second retrieval result as the second correlation. If the second retrieval result is determined by other means, then calculate the second correlation between the retrieval request and the second retrieval result, and the calculation method of the second correlation is the same as that of the above-mentioned third correlation.

[0124] Step S303: Select multiple first retrieval results from the multiple second retrieval results based on the second correlation corresponding to each second retrieval result and the data set to which each second retrieval result belongs.

[0125] Among them, the data set includes a first data set and a second data set. The data in the first data set is the data for training the target model. The target model includes the models used in the retrieval process. The data in the second data set is the data in the retrieval database other than the data in the first data set, that is, the data in the second data set is not used to train the target model. For example, the target model can be a correlation determination model, a similarity determination model, etc.

[0126] In some embodiments, for each second retrieval result, when the second retrieval result belongs to the first data set and the second correlation corresponding to the second retrieval result is greater than the first threshold, determine the second retrieval result as the first retrieval result; or when the second retrieval result belongs to the second data set and the second correlation corresponding to the second retrieval result is greater than the second threshold, determine the second retrieval result as the first retrieval result. Among them, the first threshold and the second threshold are preset values, and the second threshold is less than the first threshold.

[0127] In some embodiments, based on the second relevance corresponding to each second retrieval result, the multiple second retrieval results are sorted in descending order of the second relevance to obtain the ranking of each second retrieval result. Then, for each second retrieval result, when the second retrieval result belongs to the first data set and the ranking of the second retrieval result is less than the fourth threshold, the second retrieval result is determined as the first retrieval result; or, when the second retrieval result belongs to the second data set and the ranking of the second retrieval result is less than the fifth threshold, the second retrieval result is determined as the first retrieval result. Wherein, the fourth threshold and the fifth threshold are pre-set data, and the fifth threshold is greater than the fourth threshold.

[0128] Since the target model uses the data in the first data set during the training process and does not use the data in the second data set, when the second retrieval result belongs to the first data set, the second relevance corresponding to the second retrieval result determined by the relevance determination model is more accurate, and the second relevance corresponding to the second retrieval result may be relatively high; when the second retrieval result belongs to the second data set, the second relevance corresponding to the second retrieval result determined by the relevance determination model is relatively less accurate, and the second relevance corresponding to the second retrieval result may be relatively low. In view of this situation, in order to avoid selecting too much data in the first data set when selecting the first retrieval result and erroneously losing the data that meets the requirements in the second data set, when selecting the first retrieval result, the second threshold is made less than the first threshold, or the fifth threshold is made greater than the fourth threshold.

[0129] It should be noted that the embodiments of the present disclosure are to obtain more accurate first retrieval results. Therefore, the first retrieval results are selected based on the second relevance and the data set to which the second retrieval result belongs. In another embodiment, if the difference in the processing accuracy of the data used for training and the data not used for training by the model is ignored, the first retrieval results can be directly selected from the multiple second retrieval results based on the second relevance corresponding to each second retrieval result, that is, according to the magnitude of the second relevance, the second retrieval results with relatively larger second relevance are selected as the first retrieval results.

[0130] In the embodiments of the present disclosure, the first retrieval results are selected from the multiple second retrieval results based on the second relevance corresponding to each second retrieval result and the data set to which each second retrieval result belongs, taking into account the situation where the retrieval database changes dynamically, that is, considering that new data is added on the basis of the data used for training the target model, and the new data is not used for training the target model, so that the selected first retrieval results are more accurate.

[0131] In the embodiments of the present disclosure, the retrieval request includes a first type of request or a second type of request. The first type of request is used to request the retrieval of data under the same data category, and the second type of request is used to request the retrieval of data under multiple data categories.

[0132] In some embodiments, a classifier is used to classify the retrieval request to determine the category of the current retrieval request, that is, to determine whether the current retrieval request is a first type of request or a second type of request. Among them, the classifier classifies the retrieval request based on the data categories included in the retrieval database (i.e., the structure of the retrieval database).

[0133] The following uses Figure 5 the embodiments shown to illustrate the case where the retrieval request is a first type of request, and uses Figure 6 the embodiments shown to illustrate the case where the retrieval request is a second type of request.

[0134] Figure 5 is a flowchart of a retrieval method shown according to an exemplary embodiment, which is executed by an electronic device. Refer to Figure 5 , and the method includes the following steps:

[0135] Step S501, in response to the retrieval request, obtain multiple first retrieval results.

[0136] For the implementation manner of step S501, refer to the above Figure 3 shown embodiments, which will not be elaborated here.

[0137] It should be noted that in some embodiments, after obtaining multiple first retrieval results, the multiple first retrieval results can be initially sorted to obtain the initial arrangement order of the multiple first retrieval results. Optionally, obtain the first relevance between each first retrieval result and the retrieval request, and then sort the multiple first retrieval results based on the magnitudes of the first relevance corresponding to the multiple first retrieval results to obtain the initial arrangement order.

[0138] Step S502, when the retrieval request is a first type of request, select at least one third retrieval result belonging to the target data category from the multiple first retrieval results, and determine the other first retrieval results among the multiple first retrieval results as the fourth retrieval results.

[0139] Among them, the target data category is the data category requested by the retrieval request.

[0140] Since the data requested by the retrieval request is the data of the target data category, the multiple first retrieval results are divided into the third retrieval results belonging to the target data category and the fourth retrieval results belonging to other data categories.

[0141] In some embodiments, when the preliminary sorting order of multiple first retrieval results has been obtained, selecting at least one third retrieval result belonging to the target data category from the multiple first retrieval results includes: determining, as the third retrieval result, the first retrieval result among the multiple first retrieval results whose corresponding first relevance is greater than the sixth threshold. If there is no first retrieval result among the multiple first retrieval results whose corresponding first relevance is greater than the sixth threshold, then determining, as the third retrieval result, the first retrieval result with the greatest corresponding first relevance.

[0142] Optionally, for each first retrieval result that has not been determined as a third retrieval result, if the data category to which it belongs is the same as the data category of any of the already determined third retrieval results, then determining it as a third retrieval result.

[0143] In some embodiments, when the preliminary sorting order of multiple first retrieval results has not been obtained, first determine the corresponding first relevance of each first retrieval result, and then use the above-described implementation to determine the third retrieval result.

[0144] In the embodiments of the present disclosure, when the data requested to be retrieved by the retrieval request is data of the target data category, the first retrieval result with a relatively large corresponding first relevance is likely to belong to the target data category. Therefore, determining, as the third retrieval result, the first retrieval result whose corresponding first relevance is greater than the sixth threshold.

[0145] Optionally, add each third retrieval result to the first set, and add each fourth retrieval result to the second set.

[0146] Step S503: Obtain the data belonging to the second data set from the retrieval database, and determine, as the third retrieval result, the data belonging to the target data category in the second data set and not included in the multiple first retrieval results.

[0147] In the embodiments of the present disclosure, for the newly added data in the retrieval database, that is, for the data in the second data set, in order to make full use of the structural information of the retrieval database, supplement the third retrieval result determined in step S502 based on the data belonging to the target data category in the second data set. Therefore, determining, as the third retrieval result, the data belonging to the target data category in the second data set and not included in the multiple first retrieval results.

[0148] Optionally, add the determined third retrieval result to the first set.

[0149] Step S504: Sort based on at least one third retrieval result and at least one fourth retrieval result to obtain the sorting order of the multiple first retrieval results; wherein, the third retrieval result is arranged before the fourth retrieval result.

[0150] In some embodiments, at least one third search result is sorted to obtain a first sorting order, and at least one fourth search result is sorted to obtain a second sorting order; the first sorting order and the second sorting order are concatenated to obtain a sorting order of multiple first search results. Herein, concatenating means appending the second sorting order after the first sorting order.

[0151] Optionally, in the case where a preliminary sorting order of multiple first search results has been obtained, the second sorting order is directly determined according to the sorting of each fourth search result in the preliminary sorting order.

[0152] Optionally, in the case where a preliminary sorting order of multiple first search results has not been obtained, the first relevance corresponding to each third search result and the first relevance corresponding to each fourth search result are obtained, and then at least one third search result and at least one fourth search result are sorted respectively based on the first relevance.

[0153] Optionally, the second set is concatenated after the first set, and the third search results in the first set are arranged in descending order of the corresponding first relevance, and the fourth search results in the second set are arranged in descending order of the corresponding first relevance.

[0154] It should be noted that in the embodiments of the present disclosure, taking the sorting order as the ascending order as an example, in another embodiment, when the sorting order is the descending order, the third search results are arranged after the fourth search results, and multiple third search results are arranged in ascending order of the corresponding first relevance, and multiple fourth search results are arranged in ascending order of the corresponding first relevance.

[0155] Step S505: Select a target search result from multiple first search results based on the sorting order.

[0156] At least one first search result starting from the first first search result is used as the target search result.

[0157] In the embodiments of the present disclosure, for the case where the search request is a first type of request, multiple first search results are first obtained, and then the third search results and the fourth search results are respectively determined based on the target data category requested by the search request and the data category to which each first search result belongs. Then at least one third search result and at least one fourth search result are sorted respectively. Therefore, when sorting, the data category to which each first search result belongs is considered, that is, the relationship between different search results is considered, and each search result is not regarded as independent data, thereby improving the accuracy of the search.

[0158] Figure 6It is a flowchart of a retrieval method shown according to an exemplary embodiment, which is executed by an electronic device. Refer to Figure 6 , the method includes the following steps:

[0159] Step S601, in response to a retrieval request, obtain multiple first retrieval results.

[0160] For the implementation manner of step S601, refer to the above Figure 3 illustrated embodiment, which will not be elaborated here.

[0161] It should be noted that in some embodiments, after obtaining multiple first retrieval results, the multiple first retrieval results can be initially sorted to obtain a preliminary arrangement order of the multiple first retrieval results. Optionally, obtain the first relevance between each first retrieval result and the retrieval request, and then sort the multiple first retrieval results based on the magnitudes of the first relevance corresponding to the multiple first retrieval results to obtain a preliminary arrangement order.

[0162] Step S602, when the retrieval request is a second type of request, determine at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, each first retrieval result in the connected set belongs to the same data category as at least one other first retrieval result in the same connected set, or the semantic similarity between each first retrieval result and at least one other first retrieval result in the same connected set is greater than a third threshold.

[0163] In the embodiments of the present disclosure, when the retrieval request is a second type of request, the user hopes that the retrieved retrieval results are data of multiple data categories and the semantics of the data are diverse. Therefore, it is necessary to consider the data category to which each first retrieval result belongs and the semantic similarity between different first retrieval results. Among them, the greater the semantic similarity, the more similar the two first retrieval results are, and the smaller the semantic similarity, the less similar the two first retrieval results are.

[0164] In some embodiments, based on the multiple first retrieval results and the data category to which each first retrieval result belongs, construct a connection relationship graph; wherein, the connection relationship graph includes multiple nodes and multiple connection lines, the nodes are used to represent the first retrieval results, and the connection line between two nodes is used to represent that the two first retrieval results corresponding to the two nodes belong to the same data category, or the semantic similarity between the two first retrieval results corresponding to the two nodes is greater than a third threshold; add the first retrieval results corresponding to the nodes directly or indirectly connected by the connection lines in the connection relationship graph to the same connected set. Among them, the connection relationship graph is used to describe the relationship between multiple first retrieval results in terms of data category and semantics.

[0165] Optionally, based on a retrieval model with a dual-tower structure, calculate the semantic similarity between two first retrieval results.

[0166] Step S603: Select at least one fifth search result from the first search results in each connected set, and determine the other first search results in the multiple first search results except the fifth search results as the sixth search results.

[0167] Among them, the first relevance corresponding to the selected fifth search result is greater than the first relevance corresponding to the unselected first search results in the connected set. In the embodiments of the present disclosure, several first search results with the largest first similarity corresponding to each connected set are used as the fifth search results.

[0168] In some embodiments, in the case where the preliminary arrangement order of the multiple first search results has been obtained, according to the first relevance corresponding to each first search result, select n fifth search results from each connected set, and the first relevance corresponding to the n fifth search results is greater than the first relevance corresponding to the other first search results in the connected set. Wherein, n is an integer greater than 0.

[0169] Optionally, add the fifth search results to the third set and add the sixth search results to the fourth set.

[0170] Step S604: Sort based on the multiple fifth search results and at least one sixth search result to obtain the arrangement order of the multiple first search results; among them, the fifth search results are arranged before the sixth search results.

[0171] In some embodiments, sort the multiple fifth search results to obtain the third arrangement order, and sort at least one sixth search result to obtain the fourth arrangement order; splice the third arrangement order and the fourth arrangement order to obtain the arrangement order of the multiple first search results. Wherein, splicing means splicing the fourth arrangement order after the third arrangement order.

[0172] Optionally, in the case where the preliminary arrangement order of the multiple first search results has been obtained, directly determine the third arrangement order according to the sorting of each fifth search result in the preliminary arrangement order, and determine the fourth arrangement order according to the sorting of each sixth search result in the preliminary arrangement order.

[0173] Optionally, in the case where the preliminary arrangement order of the multiple first search results has not been obtained, obtain the first relevance corresponding to each fifth search result and the first relevance corresponding to each sixth search result, and then sort the multiple fifth search results and the multiple sixth search results respectively based on the first relevance.

[0174] Optionally, splice the fourth set after the third set, and arrange the fifth retrieval results in the third set in descending order according to the corresponding first relevance, and arrange the sixth retrieval results in the fourth set in descending order according to the corresponding first relevance.

[0175] It should be noted that in the embodiment of the present disclosure, the sorting order is taken as an example. In another embodiment, when the sorting order is reverse sorting, the fifth retrieval result is arranged after the sixth retrieval result, and multiple fifth retrieval results are arranged in ascending order according to the corresponding first relevance, and multiple sixth retrieval results are arranged in ascending order according to the corresponding first relevance.

[0176] Step S605, select a target retrieval result from multiple first retrieval results based on the sorting order.

[0177] Use at least one first retrieval result starting from the first first retrieval result as the target retrieval result.

[0178] In the embodiment of the present disclosure, for the case where the retrieval request is a second type of request, first obtain multiple first retrieval results, and then based on the multiple data categories requested by the retrieval request and the data category to which each first retrieval result belongs, determine the fifth retrieval result and the sixth retrieval result respectively by dividing the connected sets, and then sort the multiple fifth retrieval results and the multiple sixth retrieval results respectively. Therefore, when sorting, consider the data category to which each first retrieval result belongs, that is, consider the relationship between different retrieval results, and do not regard each retrieval result as independent data, thereby improving the accuracy of retrieval.

[0179] The retrieval method provided by the embodiment of the present disclosure has better effects compared with the retrieval methods in the related art. For example, taking the tool retrieval in the field of machine learning technology as an example, see Table 1 below:

[0180] Table 1

[0181]

[0182] Table 1 shows a comparison of the test results of different technical solutions on 6 sub - test sets of ToolBench. The ToolBench dataset is a test set for evaluating the tool learning ability in the scenario of user - chatbot interaction. There are some newly added tools in the tool library and a certain hierarchical structure (data categories). The retrieval requests corresponding to I1 - Inst, I2 - Inst, and I3 - Inst are related to the tools existing in the training data, while the retrieval requests corresponding to I1 - Tool, I1 - Cat, and I2 - Cat are related to the newly added tools. In addition, the retrieval requests corresponding to I1 - Inst, I1 - Tool, and I1 - Cat belong to the first - type requests, which require calling one or more APIs in the same tool to solve the retrieval requests; the retrieval requests corresponding to I2 - Inst, I3 - Inst, and I2 - Cat belong to the second - type requests, which require calling different APIs belonging to multiple tools to solve the retrieval requests.

[0183] The data in Table 1 are the recall rates of the top 5 target retrieval results obtained by retrieval. For the existing technical solutions, the numbers in parentheses indicate the number of candidates (the first retrieval result) provided to the re - ranking model.

[0184] It can be seen from Table 1 that, on average, the effect is better when using the existing re - ranking technical solutions compared to the case without re - ranking, indicating that re - ranking can effectively improve the accuracy of retrieval results. On the test set related to the existing tools, when there are fewer candidates, the re - ranking effect is better, mainly because the vast majority (91.9%) of the correct retrieval results appear in the top 10 results. Therefore, giving more candidates to the re - ranking model will introduce more noise and is not conducive to improving the retrieval accuracy. On the test set related to the newly added tools, when there are more candidates, the re - ranking effect is better, mainly because the proportion of correct retrieval results in the top 10 results is relatively small (70.2%). Therefore, giving more candidates to the re - ranking model can increase the chance of finding the correct result.

[0185] The adaptive truncation method in the technical solution of the present disclosure can provide fewer candidates when the retrieval request is related to the existing tools, and more candidates when the retrieval request is related to the newly added tools. This enables better results to be obtained regardless of whether the user request is related to the existing or newly added tools. By using re - ranking related to the structure (data categories), the technical solution of the present disclosure makes the retrieval results of the first - type requests more concentrated and the retrieval results of the second - type requests more dispersed, further improving the accuracy of the retrieval results. On average, compared with the existing technical solutions (10), existing technical solutions (30), and existing technical solutions (50), the recall rates of the top 5 target retrieval results of the technical solution of the present disclosure are increased by 5.0, 3.9, and 4.8 percentage points respectively.

[0186] For another example, as shown in Table 2 below:

[0187] Table 2

[0188] Technical solution Pass rate (%) Winning rate (%) Without using reordering 57.2 49.8 Existing technical solution (10) 60.2 53.7 Existing technical solution (30) 59.8 53.7 Existing technical solution (50) 59.8 55.2 Technical solution of the present disclosure 61.5 57.0

[0189] Table 2 shows the impact of the technical solution of the present disclosure on the ability of the large-scale language model call tool to solve retrieval requests. Compared with when re-ranking is not used, both the passing rate and the winning rate increase when using the prior art solution, indicating that using re-ranking during tool retrieval can improve the ability of the large-scale language model call tool to solve user requests. Compared with the prior art solution, the technical solution of the present disclosure can further improve the passing rate and the winning rate, further demonstrating that the re-ranking method proposed in the technical solution of the present disclosure can make full use of the characteristics of the tool library and more effectively improve the ability of the large model call tool to solve retrieval requests.

[0190] Figure 7 is a block diagram of a retrieval device shown according to an exemplary embodiment, configured in an electronic device. Refer to Figure 7 , the device includes:

[0191] A first result acquisition module 701, configured to acquire a plurality of first retrieval results in response to a retrieval request; wherein, the retrieval request includes a first type of request or a second type of request, the first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories;

[0192] A result sorting module 702, configured to sort a plurality of first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs, to obtain an arrangement order of the plurality of first retrieval results; wherein, the first retrieval results under different data categories are arranged according to the data category requested by the retrieval request, and the first retrieval results under each data category are arranged according to the first relevance with the retrieval request;

[0193] A target result acquisition module 703, configured to select a target retrieval result from the plurality of first retrieval results based on the arrangement order.

[0194] In some embodiments, the first result acquisition module 701 is configured to:

[0195] In response to a retrieval request, retrieve a plurality of second retrieval results from a retrieval database;

[0196] Determine a second relevance between the retrieval request and each second retrieval result;

[0197] Based on the second relevance corresponding to each second retrieval result, select a plurality of first retrieval results from the plurality of second retrieval results.

[0198] In some embodiments, the first result acquisition module 701 is configured to:

[0199] Select a plurality of first retrieval results from a plurality of second retrieval results based on the second relevance corresponding to each second retrieval result and the data set to which each second retrieval result belongs;

[0200] Wherein, the data set includes a first data set and a second data set. The data in the first data set is the data used to train the target model, and the target model includes the model used in the retrieval process. The data in the second data set is the data in the retrieval database other than the data in the first data set.

[0201] In some embodiments, the first result acquisition module 701 is configured to:

[0202] For each second retrieval result,

[0203] When the second retrieval result belongs to the first data set and the second relevance corresponding to the second retrieval result is greater than the first threshold, determine the second retrieval result as the first retrieval result; or,

[0204] When the second retrieval result belongs to the second data set and the second relevance corresponding to the second retrieval result is greater than the second threshold, determine the second retrieval result as the first retrieval result;

[0205] Wherein, the second threshold is less than the first threshold.

[0206] In some embodiments, the result sorting module 702 is configured to:

[0207] When the retrieval request is a first type of request, select at least one third retrieval result belonging to the target data category from a plurality of first retrieval results, and determine the other first retrieval results in the plurality of first retrieval results except the third retrieval results as fourth retrieval results. The target data category is the data category requested by the retrieval request;

[0208] Sort based on at least one third retrieval result and at least one fourth retrieval result to obtain the arrangement order of the plurality of first retrieval results; wherein, the third retrieval results are arranged before the fourth retrieval results.

[0209] In some embodiments, the result sorting module 702 is further configured to:

[0210] Obtain data belonging to the second data set from the retrieval database; wherein, the data sets to which the data in the retrieval database belong include a first data set and a second data set, the data in the first data set are data for training the target model, the target model includes the model used in the retrieval process, and the data in the second data set are the data in the retrieval database other than the data in the first data set;

[0211] Determine the data belonging to the target data category in the second data set and not included in the multiple first retrieval results as the third retrieval result.

[0212] In some embodiments, the result sorting module 702 is configured to:

[0213] Sort at least one third retrieval result to obtain a first arrangement order, and sort at least one fourth retrieval result to obtain a second arrangement order;

[0214] Concatenate the first arrangement order and the second arrangement order to obtain the arrangement order of the multiple first retrieval results.

[0215] In some embodiments, the result sorting module 702 is configured to:

[0216] When the retrieval request is a second type of request, determine at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, each first retrieval result in the connected set belongs to the same data category as at least one other first retrieval result in the same connected set, or the semantic similarity between each first retrieval result and at least one other first retrieval result in the same connected set is greater than a third threshold;

[0217] Select at least one fifth retrieval result from the first retrieval results in each connected set respectively, and determine the other first retrieval results in the multiple first retrieval results except the fifth retrieval results as the sixth retrieval results; the first relevance corresponding to the selected fifth retrieval results is greater than the first relevance corresponding to the first retrieval results not selected in the connected set;

[0218] Sort based on the multiple fifth retrieval results and at least one sixth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the fifth retrieval results are arranged before the sixth retrieval results.

[0219] In some embodiments, the result sorting module 702 is configured to:

[0220] Construct a connectivity graph based on multiple first retrieval results and the data categories to which each first retrieval result belongs; wherein, the connectivity graph includes multiple nodes and multiple connection lines, the nodes are used to represent the first retrieval results, and the connection line between two nodes is used to represent that the two first retrieval results corresponding to the two nodes belong to the same data category, or the semantic similarity between the two first retrieval results corresponding to the two nodes is greater than a third threshold;

[0221] Add the first retrieval results corresponding to the nodes directly or indirectly connected by the connection lines in the connectivity graph to the same connected set.

[0222] In some embodiments, the result sorting module 702 is configured to:

[0223] Sort multiple fifth retrieval results to obtain a third sorting order, and sort at least one sixth retrieval result to obtain a fourth sorting order;

[0224] Concatenate the third sorting order and the fourth sorting order to obtain the sorting order of multiple first retrieval results.

[0225] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0226] An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute the retrieval method in the above embodiments.

[0227] Figure 8 It is a block diagram of an electronic device 800 shown according to an exemplary embodiment.

[0228] Refer to Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0229] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0230] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0231] The power supply component 806 provides power to various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0232] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0233] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0234] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a start button, and a lock button.

[0235] The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects for the electronic device 800. For example, the sensor assembly 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0236] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0237] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0238] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0239] Embodiments of the present disclosure also provide a non-transitory computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the retrieval method in the above embodiments.

[0240] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only to be considered as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0241] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A retrieval method, characterized in that, The method includes: In response to a retrieval request, obtaining a plurality of first retrieval results; wherein, the retrieval request includes a first type of request or a second type of request, the first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories; Based on the retrieval request and the data category to which each first retrieval result belongs, sorting the plurality of first retrieval results to obtain an arrangement order of the plurality of first retrieval results; wherein, the first retrieval results under different data categories are arranged according to the data categories requested by the retrieval request, and the first retrieval results under each data category are arranged according to a first relevance to the retrieval request; Based on the arrangement order, selecting a target retrieval result from the plurality of first retrieval results.

2. The method according to claim 1, characterized in that, The step of, in response to a retrieval request, obtaining a plurality of first retrieval results includes: In response to the retrieval request, retrieving a plurality of second retrieval results from a retrieval database; Determining a second relevance between the retrieval request and each second retrieval result; Based on the second relevance corresponding to each second retrieval result, selecting the plurality of first retrieval results from the plurality of second retrieval results.

3. The method according to claim 2, characterized in that, The step of, based on the second relevance corresponding to each second retrieval result, selecting the plurality of first retrieval results from the plurality of second retrieval results includes: Based on the second relevance corresponding to each second retrieval result and the data set to which each second retrieval result belongs, selecting the plurality of first retrieval results from the plurality of second retrieval results; wherein, the data set includes a first data set and a second data set, the data in the first data set is data for training a target model, the target model includes a model used in the retrieval process, and the data in the second data set is data in the retrieval database other than the data in the first data set.

4. The method according to claim 3, wherein The step of, based on the second relevance corresponding to each second retrieval result and the data set to which each second retrieval result belongs, selecting the plurality of first retrieval results from the plurality of second retrieval results includes: For each second retrieval result, when the second retrieval result belongs to the first data set and the second relevance corresponding to the second retrieval result is greater than a first threshold, determining the second retrieval result as the first retrieval result; or, when the second retrieval result belongs to the second data set and the second relevance corresponding to the second retrieval result is greater than a second threshold, determining the second retrieval result as the first retrieval result; wherein, the second threshold is less than the first threshold.

5. The method according to claim 1, characterized in that, The step of, based on the retrieval request and the data category to which each first retrieval result belongs, sorting the plurality of first retrieval results to obtain an arrangement order of the plurality of first retrieval results includes: When the retrieval request is the first type of request, select at least one third retrieval result belonging to the target data category from the multiple first retrieval results, and determine the other first retrieval results among the multiple first retrieval results except the third retrieval result as the fourth retrieval result, where the target data category is the data category requested by the retrieval request; Sort based on the at least one third retrieval result and at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the third retrieval result is arranged before the fourth retrieval result.

6. The method according to claim 5, wherein Before the step of sorting based on the at least one third retrieval result and at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results, the method further includes: Obtain data belonging to the second data set from the retrieval database; wherein, the data sets to which the data in the retrieval database belongs include the first data set and the second data set, the data in the first data set is the data for training the target model, the target model includes the model used in the retrieval process, and the data in the second data set is the data in the retrieval database except the data in the first data set; Determine the data belonging to the target data category in the second data set and not included in the multiple first retrieval results as the third retrieval result.

7. The method according to claim 5, characterized in that, The step of sorting based on the at least one third retrieval result and at least one fourth retrieval result to obtain the arrangement order of the multiple first retrieval results includes: Sort the at least one third retrieval result to obtain a first arrangement order, and sort the at least one fourth retrieval result to obtain a second arrangement order; Concatenate the first arrangement order and the second arrangement order to obtain the arrangement order of the multiple first retrieval results.

8. The method according to claim 1, wherein The step of sorting the multiple first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs to obtain the arrangement order of the multiple first retrieval results includes: When the retrieval request is the second type of request, determine at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, each first retrieval result in the connected set belongs to the same data category as at least one other first retrieval result in the same connected set, or the semantic similarity between each first retrieval result and at least one other first retrieval result in the same connected set is greater than a third threshold; Select at least one fifth retrieval result from the first retrieval results in each connected set respectively, and determine the other first retrieval results among the multiple first retrieval results except the fifth retrieval result as the sixth retrieval result; the first relevance corresponding to the selected fifth retrieval result is greater than the first relevance corresponding to the first retrieval results not selected in the connected set; Sort based on the multiple fifth retrieval results and at least one sixth retrieval result to obtain the arrangement order of the multiple first retrieval results; wherein, the fifth retrieval result is arranged before the sixth retrieval result.

9. The method according to claim 8, wherein Determining at least one connected set based on the multiple first retrieval results and the data category to which each first retrieval result belongs, includes: Constructing a connection relationship graph based on the multiple first retrieval results and the data category to which each first retrieval result belongs; wherein, the connection relationship graph includes multiple nodes and multiple connection lines, the nodes are used to represent the first retrieval results, and the connection line between two nodes is used to represent that the two first retrieval results corresponding to the two nodes belong to the same data category, or the semantic similarity between the two first retrieval results corresponding to the two nodes is greater than the third threshold; Adding the first retrieval results corresponding to the nodes directly or indirectly connected by the connection lines in the connection relationship graph to the same connected set.

10. The method according to claim 8, characterized in that, Sorting the multiple fifth retrieval results and at least one sixth retrieval result to obtain the arrangement order of the multiple first retrieval results, includes: Sorting the multiple fifth retrieval results to obtain a third arrangement order, and sorting the at least one sixth retrieval result to obtain a fourth arrangement order; Concatenating the third arrangement order and the fourth arrangement order to obtain the arrangement order of the multiple first retrieval results.

11. A retrieval device, characterized in that, The apparatus includes: A first result acquisition module, configured to acquire multiple first retrieval results in response to a retrieval request; wherein, the retrieval request includes a first type of request or a second type of request, the first type of request is used to request to retrieve data under the same data category, and the second type of request is used to request to retrieve data under multiple data categories; A result sorting module, configured to sort the multiple first retrieval results based on the retrieval request and the data category to which each first retrieval result belongs, to obtain the arrangement order of the multiple first retrieval results; wherein, the first retrieval results under different data categories are arranged according to the data categories requested by the retrieval request, and the first retrieval results under each data category are arranged according to the first relevance with the retrieval request; A target result acquisition module, configured to select a target retrieval result from the multiple first retrieval results based on the arrangement order.

12. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the method according to any one of claims 1-10.

13. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the method according to any one of claims 1-10.

Images