Retrieval model determination method based on use scene and related device

Abstract

The invention discloses a retrieval model determination method based on a use scene and a related device. The method comprises the following steps: firstly, determining an applicable retrieval knowledge base by identifying a specific use scene, then constructing a question set containing a plurality of retrieval questions, and adding a result label for each question. Inputting the questions into a retrieval model for retrieval to obtain a series of model question and answer results, and comparing differences between the model question and answer results and result labels to obtain prediction deviations. And then the prediction deviation is utilized to adjust the retrieval model, so that the retrieval model can better adapt to the current use scene, and the accuracy and the performance of the model are further improved. According to the method, the retrieval model is adjusted based on the prediction deviation, so that the retrieval model is continuously optimized to adapt to various use scenes, the model performance is improved, high accuracy is kept to face new and old scenes, meanwhile, manual selection dependence is reduced through an automatic systematization method, time cost and resource consumption are reduced, and the accuracy and reliability of model selection are improved.

Description

Technical Field

[0001] The present application relates to the field of information retrieval technology, and in particular to a method for determining a retrieval model based on a usage scenario and a related device. Background Art

[0002] With the rise of big models, the integration and retrieval methods of external knowledge bases have been widely studied and applied to update the knowledge bases of these big models and enhance their professional question-answering capabilities. However, in practice, with the increase in usage scenarios and the number of users, it was found that the big models required for different usage scenarios were significantly different despite the similarity of the overall retrieval framework. In this case, for a specific problem, it is highly dependent on the developer to select a big model that is suitable for the scenario to which the problem belongs to ensure that it can provide the best performance. However, relying on the choice of developers requires a lot of time and effort to evaluate the performance of different big models, which is not only time-consuming but also prone to misjudgment. At the same time, developers have limited knowledge and experience and cannot fully consider all potential usage scenarios and needs, resulting in poor performance in certain specific scenarios.

[0003] Therefore, how to reduce dependence on developers while improving the adaptability and accuracy of large models in various usage scenarios is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0004] Based on the above problems, the present application provides a retrieval model determination method and related devices based on usage scenarios, which can reduce dependence on developers while improving the adaptability and accuracy of large models in various usage scenarios.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] A method for determining a retrieval model based on a usage scenario, the method comprising:

[0007] Identifying a usage scenario, and determining a search knowledge base based on the usage scenario;

[0008] Constructing a search question set applicable to the search knowledge base and having a result label; the search question set includes multiple search questions; the result label is used to mark the actual question-answering result of each search question in the search question set;

[0009] Input the retrieval question set into the retrieval model for retrieval to obtain a model question and answer result set;

[0010] Obtaining a prediction deviation based on a difference between the model question-answer result set and the result label;

[0011] The retrieval model is adjusted based on the prediction bias.

[0012] In a possible implementation, the retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence;

[0013] Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

[0014] In a possible implementation, adjusting the retrieval model based on the prediction deviation includes:

[0015] If the prediction deviation indicates that the literal retrieval recall rate is lower than the first target recall rate, adjusting the variable parameters of the literal retrieval module based on the literal retrieval recall rate;

[0016] If the prediction deviation indicates that the semantic retrieval recall rate is lower than the second target recall rate, adjusting the variable parameters of the semantic retrieval module based on the semantic retrieval recall rate;

[0017] If the prediction deviation indicates that the refined ranking accuracy is lower than the target accuracy, the model parameters of the deep refined ranking model are adjusted based on the refined ranking accuracy.

[0018] In a possible implementation, determining a search knowledge base based on the usage scenario includes:

[0019] Determining a target knowledge base based on the usage scenario;

[0020] Parsing the documents in the target knowledge base to obtain the document title and document content;

[0021] Performing knowledge management on the document title and the document content to obtain keywords, abstracts, and question-answer pairs of the target knowledge base;

[0022] The keywords, the abstracts, the question-answer pairs, the document titles and the document contents of the target knowledge base are organized into the retrieval knowledge base.

[0023] In a possible implementation, the knowledge governance includes: format problem governance, paragraph-level information governance, and chapter-level information governance.

[0024] A retrieval model determination device based on usage scenarios, the device comprising:

[0025] A scene identification unit is used to identify the usage scene;

[0026] A first knowledge base determination unit, configured to determine a search knowledge base based on the usage scenario;

[0027] A question set construction unit is used to construct a search question set applicable to the search knowledge base and having a result label; the search question set includes multiple search questions; the result label is used to mark the actual question and answer result of each search question in the search question set;

[0028] A retrieval unit, used to input the retrieval question set into the retrieval model for retrieval, and obtain a model question and answer result set;

[0029] A prediction unit, configured to obtain a prediction deviation based on a difference between the model question and answer result set and the result label;

[0030] A model adjustment unit is used to adjust the retrieval model based on the prediction deviation.

[0031] In a possible implementation, the retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence;

[0032] Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

[0033] In a possible implementation, the model adjustment unit specifically includes:

[0034] a first adjustment unit, configured to adjust a variable parameter of the literal retrieval module based on the literal retrieval recall rate if the prediction deviation indicates that the literal retrieval recall rate is lower than a first target recall rate;

[0035] a second adjustment unit, configured to adjust a variable parameter of the semantic retrieval module based on the semantic retrieval recall rate if the prediction deviation indicates that the semantic retrieval recall rate is lower than a second target recall rate;

[0036] The third adjustment unit is used to adjust the model parameters of the deep fine ranking model based on the fine ranking accuracy if the prediction deviation indicates that the fine ranking accuracy is lower than the target accuracy.

[0037] A device for determining a retrieval model based on a usage scenario comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for determining a retrieval model based on a usage scenario as described above is implemented.

[0038] A computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute the above-mentioned method for determining a retrieval model based on a usage scenario.

[0039] Compared with the prior art, this application has the following beneficial effects:

[0040] The present application provides a method for determining a retrieval model based on a usage scenario and a related device. Specifically, when executing the method for determining a retrieval model based on a usage scenario provided by an embodiment of the present application, the usage scenario can be identified first, and the corresponding retrieval knowledge base can be determined based on the usage scenario. Next, a retrieval question set suitable for the selected retrieval knowledge base is constructed, the set contains multiple specific retrieval questions, and each retrieval question is equipped with a result label for marking the actual question and answer results of each retrieval question. Then, the constructed retrieval question set is input into the retrieval model for retrieval, so as to obtain the question and answer result set of the model. By analyzing the difference between the model question and answer result set and the actual result label, the prediction deviation is calculated. Finally, the retrieval model is adjusted and optimized according to the prediction deviation to improve its performance and accuracy in a specific usage scenario. The present application adjusts the retrieval model based on the prediction deviation, which can continuously optimize the model to meet the needs of different usage scenarios. This method can not only improve the overall performance of the retrieval model, but also respond quickly when encountering new or complex usage scenarios, and maintain high accuracy. At the same time, the dependence on human selection is reduced through automated and systematic methods, which reduces time costs and human resource consumption, and improves the accuracy and reliability of model selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A method flow chart of a method for determining a retrieval model based on a usage scenario provided in an embodiment of the present application;

[0043] Figure 2A method flow chart of a method for determining a search knowledge base provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a retrieval model determination device based on usage scenarios provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0046] With the rise of big models, the integration and retrieval methods of external knowledge bases have been widely studied and applied, aiming to update the knowledge bases of these big models and enhance their professional question-answering capabilities. However, in practice, with the increase in usage scenarios and the number of users, it was found that the big models required for different usage scenarios were significantly different despite the similarity of the overall retrieval framework. In this case, for a specific problem, it is highly dependent on developers to select a big model that is suitable for the scenario to which the problem belongs to ensure that it can provide the best performance. However, there are many problems with relying on the choice of developers: first, it requires a lot of time and effort to evaluate the performance of different big models, which is not only time-consuming, but also prone to misjudgment; second, developers have limited knowledge and experience and cannot fully consider all potential usage scenarios and needs, resulting in poor performance in certain specific scenarios.

[0047] In order to solve this problem, a method and related device for determining a retrieval model based on a usage scenario are provided in an embodiment of the present application. First, the usage scenario is identified, and the corresponding retrieval knowledge base is determined based on the usage scenario. Next, a retrieval question set suitable for the retrieval knowledge base and having a result label is constructed, which contains multiple retrieval questions, and the result label of each question is used to mark its actual question and answer results. Then, the retrieval question set is input into the retrieval model for retrieval to obtain the question and answer result set of the model. By comparing the model question and answer result set with the actual question and answer results of the result label, the prediction deviation is calculated. Finally, the retrieval model is adjusted based on the prediction deviation to optimize its performance and accuracy. The present application adjusts the retrieval model based on the prediction deviation, which enables the model to be continuously optimized to adapt to the needs of different usage scenarios, not only improving the overall performance, but also responding quickly and maintaining high accuracy when encountering new or complex usage scenarios, while reducing the reliance on human selection through automated and systematic methods, reducing time costs and human resource consumption, and improving the accuracy and reliability of model selection.

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] See also Figure 1 , which is a flow chart of a method for determining a retrieval model based on a usage scenario provided in an embodiment of the present application, such as Figure 1 As shown, the method for determining a retrieval model based on a usage scenario may include steps S101-S105:

[0050] S101: Identify usage scenarios, and determine a search knowledge base based on the usage scenarios.

[0051] Identifying the usage scenario and determining the retrieval knowledge base based on the usage scenario means that when processing information retrieval or question-answering system tasks, you first need to clarify the current application scenario. The usage scenario can be a specific field or industry, such as medicine, law, technology, etc. By identifying this specific usage scenario, you can more accurately determine which retrieval knowledge base should be used. The knowledge base in each field contains the expertise and information in that field, and can provide more accurate and professional search results. Therefore, identifying the usage scenario helps to select the most appropriate knowledge base, thereby improving the accuracy and practicality of the retrieval.

[0052] For example, if the usage scenario is diagnostic assistance in the medical field, then you should choose a knowledge base that contains medical knowledge, such as a disease database, clinical guidelines, etc. If it is legal consultation, you should choose a legal knowledge base that contains information such as legal provisions and precedents. In this way, you can provide more accurate answers and suggestions for different scenarios, improving user experience and system practicality.

[0053] See also Figure 2 , Figure 2 A method flow chart of a method for determining a search knowledge base provided in an embodiment of the present application. Accordingly, in step S101, determining the search knowledge base based on the usage scenario can be implemented through steps A1-A4:

[0054] A1: Determine a target knowledge base based on the usage scenario.

[0055] First, identify the specific usage scenario (such as medicine, law, technology, etc.), and then select the knowledge base that best meets the needs of the scenario. Each field has its own specific knowledge base, which contains corresponding expertise and information.

[0056] A2: Parse the documents in the target knowledge base to obtain the document title and document content.

[0057] Once the target knowledge base is selected, the next step is to parse the documents in it. The parsing process involves extracting the title and content of the document for further processing and analysis.

[0058] Document title: The document title is a concise summary of the document content, usually using one or two sentences or a few words to describe the main theme or core content of the document.

[0059] For example, an article about artificial intelligence might be titled “Application of Deep Learning in Image Recognition.”

[0060] Document content: Document content refers to all text information contained in the document, including paragraphs, sentences, picture captions, etc. It describes the subject of the document and related information in detail.

[0061] For example, the content of an article about artificial intelligence may include a detailed introduction to deep learning algorithms, specific application scenarios, analysis of experimental results, etc.

[0062] In a possible implementation, the corresponding parsing tool can be used to parse the documents in the target knowledge base. The present application supports various file types such as Portable Document Format (PDF), Word, Excel, pictures, text files (TXT), codes, etc.

[0063] A3: Perform knowledge management on the document title and the document content to obtain keywords, summaries, and question-answer pairs of the target knowledge base.

[0064] The parsed document titles and contents are then used for knowledge management, keywords are extracted, summaries are generated, and question-answer pairs are constructed. These steps are designed to transform the original document content into a form that is easier to retrieve and understand.

[0065] Keywords:

[0066] Keywords are words or phrases extracted from a document that can represent the subject or main content of the document.

[0067] They can help users quickly understand the core content of the document and serve as indexes when searching.

[0068] For example, for an article about artificial intelligence, keywords may include "deep learning", "image recognition", "neural network", etc.

[0069] Abstract: An abstract is a high-level summary and conclusion of the document's contents. It is usually a short paragraph or a few sentences that concisely describes the main information and conclusions of the document.

[0070] The abstract helps readers quickly understand the core content of the document and avoid reading the full text.

[0071] For example, the abstract of an article about artificial intelligence might mention the main question the article investigated, the methods used, and the conclusions reached.

[0072] Question-answer pair: A question-answer pair is a relationship consisting of a question and its corresponding answer, which is used to build an interactive knowledge framework.

[0073] These question-answer pairs can be generated based on the document content, aiming to provide key information in the form of questions and answers, making it easier for users to obtain the required knowledge in the form of questions.

[0074] For example, for an article about artificial intelligence, possible question-answer pairs might be: “Q: What is deep learning? A: Deep learning is a machine learning technique that uses multi-layer neural network models to learn the characteristics of data.”

[0075] In one possible implementation, the knowledge governance includes: formatting issue governance (such as line breaks, tabs, etc.), paragraph-level information governance (such as paragraph division, summary generation, keyword extraction, and question-answer pair generation, etc.) and chapter-level information governance (such as summary generation, keyword extraction, and question-answer pair generation, etc.).

[0076] A4: The keywords, the abstracts, the question-answer pairs, the document titles and the document contents of the target knowledge base are organized into the retrieval knowledge base.

[0077] Finally, the managed keywords, abstracts, question-answer pairs, document titles, and document contents are integrated into a structured retrieval knowledge base for subsequent retrieval and query.

[0078] S102: Constructing a search question set that is applicable to the search knowledge base and has result labels; the search question set includes multiple search questions.

[0079] Constructing a set of retrieval questions with result labels for a specific retrieval knowledge base means creating a dataset containing multiple search queries and associating labels of actual question and answer results for each query. A series of related questions designed based on the selected knowledge base should cover important knowledge points and common query scenarios in the knowledge base. Each question in the retrieval question set should be representative and able to test the coverage and accuracy of the knowledge base. The actual question and answer results collected for each retrieval question can be obtained through professional knowledge personnel or authoritative channels to ensure their accuracy and reliability. These actual question and answer results are attached to each retrieval question as result labels to form a set of retrieval questions with result labels. The designed retrieval questions and their corresponding result labels are organized into a structured set of retrieval questions. Each entry in this set contains a question and a label that annotates the actual question and answer results.

[0080] S103: Input the search question set into the search model for search, and obtain a model question and answer result set.

[0081] Inputting a set of retrieval questions into a retrieval model for retrieval to obtain a set of model question and answer results means inputting a designed set of questions containing multiple retrieval questions and their result labels into a pre-trained retrieval model to obtain the model's answer to each question.

[0082] In a possible implementation, the retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence;

[0083] Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

[0084] The retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence;

[0085] Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

[0086] S104: Obtain a prediction deviation based on the difference between the model question and answer result set and the result label.

[0087] The prediction bias is obtained based on the difference between the model question and answer result set and the result label. This means that the prediction accuracy of the model is evaluated by comparing the answer results generated by the retrieval model (model question and answer result set) with the pre-labeled actual question and answer results (result labels), and the difference between the two is calculated, that is, the prediction bias. This process helps to identify where the model performs poorly, so as to further optimize and improve the model.

[0088] S105: Adjust the retrieval model based on the prediction deviation.

[0089] Adjusting the retrieval model based on prediction bias means systematically identifying and improving the deficiencies of the model by comparing the answer results generated by the retrieval model with the actual question-answer results that are pre-labeled (i.e., prediction bias). This process helps improve the performance and accuracy of the model in specific application scenarios.

[0090] In a possible implementation, adjusting the retrieval model based on the prediction deviation includes:

[0091] If the prediction deviation indicates that the literal retrieval recall rate is lower than the first target recall rate, adjusting the variable parameters of the literal retrieval module based on the literal retrieval recall rate;

[0092] If the prediction deviation indicates that the semantic retrieval recall rate is lower than the second target recall rate, adjusting the variable parameters of the semantic retrieval module based on the semantic retrieval recall rate;

[0093] If the prediction deviation indicates that the refined ranking accuracy is lower than the target accuracy, the model parameters of the deep refined ranking model are adjusted based on the refined ranking accuracy.

[0094] That is to say, we analyze the specific types and degrees of prediction bias, focusing on three indicators: literal retrieval recall rate, semantic retrieval recall rate, and precision sorting accuracy rate. The specific analysis steps are as follows:

[0095] (1) Literal retrieval recall rate: Check whether the model has achieved the predetermined target recall rate (first target recall rate) in literal retrieval based on keyword matching.

[0096] If the literal retrieval recall rate is lower than the first target recall rate, it is necessary to adjust the variable parameters of the literal retrieval module, such as adjusting the threshold, optimizing the matching algorithm, etc.

[0097] (2) Semantic retrieval recall rate: Check whether the model has achieved the predetermined target recall rate (second target recall rate) in terms of retrieval based on semantic understanding.

[0098] If the semantic retrieval recall rate is lower than the second target recall rate, it is necessary to adjust the variable parameters of the semantic retrieval module, such as improving the semantic representation method, adjusting the attention mechanism, etc.

[0099] (3) Fine sorting accuracy: Check whether the model has achieved the target accuracy when deeply sorting the search results.

[0100] If the refined ranking accuracy is lower than the target accuracy, you need to adjust the model parameters of the deep refined ranking model, such as the learning rate, network layer structure, etc.

[0101] After determining the cause of the deviation, appropriate adjustment measures can be taken based on the analysis results of the forecast deviation.

[0102] Specific adjustment measures may include:

[0103] If the prediction deviation indicates that the literal retrieval recall rate is lower than the first target recall rate, it is necessary to adjust the variable parameters of the literal retrieval module based on the literal retrieval recall rate. This means that when adjusting the literal retrieval module, it is necessary to pay attention to aspects such as the literal retrieval algorithm selection parameters, the literal retrieval paragraph granularity parameters, and the literal recall range parameters. By adjusting these variable parameters, the effect of literal retrieval can be improved, the recall rate can be increased, and more relevant documents can be successfully retrieved. For example, optimizing the parameter settings of the literal retrieval algorithm, adjusting the paragraph granularity to balance the recall rate and accuracy, and expanding or narrowing the recall range can help increase the coverage and relevance of the retrieval results.

[0104] If the prediction deviation shows that the semantic retrieval recall rate is lower than the second target recall rate, it is necessary to adjust the variable parameters of the semantic retrieval module based on the semantic retrieval recall rate. In this case, the variable parameters of interest include the semantic representation model selection parameter, the semantic recall index type selection parameter, the semantic representation text length parameter, and the semantic recall range parameter. By adjusting these parameters, such as improving the selection of the semantic representation model, optimizing the recall index type to strengthen semantic matching, and adjusting the length and range of text processing, the model's understanding and matching capabilities of semantic information can be improved, thereby improving the recall rate and ensuring that more relevant documents are successfully retrieved.

[0105] In addition, if the prediction deviation indicates that the precision ranking accuracy is lower than the target accuracy, the model parameters of the deep precision ranking model need to be adjusted based on the precision ranking accuracy. At this time, it is necessary to pay attention to the model parameters of the deep precision ranking model, such as feature selection, loss function optimization, model structure adjustment, and sample balance. By adjusting these model parameters, the ranking quality and correctness of search results can be improved, ensuring that users get relevant document rankings that better meet their needs. These adjustment measures help optimize the performance of the entire semantic retrieval system, improve recall and accuracy, and thus improve user experience and search results.

[0106] Based on the contents of S101-S105, it can be known that a specific usage scenario is first identified to determine the retrieval knowledge base applicable to the scenario. Subsequently, a set of retrieval questions with result labels is constructed, and these result labels are used to mark the actual question and answer results for the retrieval questions in each retrieval question set. Next, this set of retrieval questions is input into the retrieval model for retrieval to obtain a set of question and answer results generated by the model. Then, the prediction deviation is calculated by comparing the difference between the model question and answer result set and the result label. Finally, the retrieval model is adjusted based on this prediction deviation to optimize the performance and accuracy of the model in a specific scenario. The present application is based on a retrieval model adjusted based on prediction deviation, and aims to achieve continuous optimization of the model so that it can flexibly adapt to the needs of different usage scenarios, thereby improving overall performance, quickly responding to new or complex situational requirements, maintaining high accuracy, and reducing time costs and human resource consumption through automated and systematic methods, while improving the accuracy and reliability of model selection.

[0107] See also Figure 3 , Figure 3 A schematic diagram of a structure of a retrieval model determination device based on a usage scenario provided in an embodiment of the present application. Figure 3 As shown, the retrieval model determination device based on the usage scenario includes:

[0108] A scene identification unit 301 is used to identify a usage scene;

[0109] A first knowledge base determining unit 302, configured to determine a search knowledge base based on the usage scenario;

[0110] A question set building unit 303 is used to build a search question set applicable to the search knowledge base and having a result label; the search question set includes multiple search questions; the result label is used to mark the actual question-answering result of each search question in the search question set;

[0111] A retrieval unit 304 is used to input the retrieval question set into the retrieval model for retrieval to obtain a model question and answer result set;

[0112] A prediction unit 305, configured to obtain a prediction deviation based on a difference between the model question and answer result set and the result label;

[0113] The model adjustment unit 306 is used to adjust the retrieval model based on the prediction deviation.

[0114] In a possible implementation, the retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence;

[0115] Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

[0116] In a possible implementation, the model adjustment unit 306 specifically includes:

[0117] a first adjustment unit, configured to adjust a variable parameter of the literal retrieval module based on the literal retrieval recall rate if the prediction deviation indicates that the literal retrieval recall rate is lower than a first target recall rate;

[0118] a second adjustment unit, configured to adjust a variable parameter of the semantic retrieval module based on the semantic retrieval recall rate if the prediction deviation indicates that the semantic retrieval recall rate is lower than a second target recall rate;

[0119] The third adjustment unit is used to adjust the model parameters of the deep fine ranking model based on the fine ranking accuracy if the prediction deviation indicates that the fine ranking accuracy is lower than the target accuracy.

[0120] In a possible implementation, the first knowledge base determining unit 302 specifically includes:

[0121] A second knowledge base determination unit, configured to determine a target knowledge base based on the usage scenario;

[0122] A document parsing unit, used to parse the documents in the target knowledge base to obtain the document title and document content;

[0123] A knowledge management unit, used to perform knowledge management on the document title and the document content to obtain keywords, abstracts, and question-answer pairs of the target knowledge base;

[0124] The arranging unit is used to arrange the keywords, the abstracts, the question-answer pairs, the document titles and the document contents of the target knowledge base into the retrieval knowledge base.

[0125] In a possible implementation, the knowledge governance includes: format problem governance, paragraph-level information governance, and chapter-level information governance.

[0126] In addition, an embodiment of the present application also provides a retrieval model determination device based on usage scenarios, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, it implements the retrieval model determination method based on usage scenarios as described above.

[0127] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the retrieval model determination method based on the usage scenario as described above.

[0128] The embodiment of the present application provides a retrieval model determination device based on a usage scenario, first using the recognition scenario unit 301 to identify the usage scenario, then using the first knowledge base determination unit 302 to determine the retrieval knowledge base based on the usage scenario, and using the construction question set unit 303 to construct a retrieval question set that is suitable for the retrieval knowledge base and has a result label. The retrieval unit 304 inputs the retrieval question set into the retrieval model for retrieval to obtain a model question and answer result set. Then, the prediction unit 305 is used to obtain the prediction deviation based on the difference between the model question and answer result set and the result label, so that the model adjustment unit 306 can adjust the retrieval model based on the prediction deviation. The present application adjusts the retrieval model based on the prediction deviation, which can continuously optimize the model to meet the needs of different usage scenarios, not only improving the overall performance of the retrieval model, but also responding quickly and maintaining high accuracy when encountering new or complex usage scenarios. At the same time, it reduces the reliance on human selection through automated and systematic methods, reduces time costs and human resource consumption, and improves the accuracy and reliability of model selection.

[0129] The above is a detailed introduction to a retrieval model determination method based on usage scenarios and related devices provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0130] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0131] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for determining a retrieval model based on a usage scenario, characterized in that: The method comprises: Identifying a usage scenario, and determining a search knowledge base based on the usage scenario; Constructing a search question set applicable to the search knowledge base and having a result label; the search question set includes multiple search questions; the result label is used to mark the actual question-answering result of each search question in the search question set; Input the retrieval question set into the retrieval model for retrieval to obtain a model question and answer result set; Obtaining a prediction deviation based on a difference between the model question-answer result set and the result label; The retrieval model is adjusted based on the prediction bias.

2. The method according to claim 1, characterized in that The retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence; Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

3. The method according to claim 2, characterized in that The adjusting the retrieval model based on the prediction deviation comprises: If the prediction deviation indicates that the literal retrieval recall rate is lower than the first target recall rate, adjusting the variable parameters of the literal retrieval module based on the literal retrieval recall rate; If the prediction deviation indicates that the semantic retrieval recall rate is lower than the second target recall rate, adjusting the variable parameters of the semantic retrieval module based on the semantic retrieval recall rate; If the prediction deviation indicates that the refined ranking accuracy is lower than the target accuracy, the model parameters of the deep refined ranking model are adjusted based on the refined ranking accuracy.

4. The method according to claim 1, characterized in that: The determining of the search knowledge base based on the usage scenario includes: Determining a target knowledge base based on the usage scenario; Parsing the documents in the target knowledge base to obtain the document title and document content; Performing knowledge management on the document title and the document content to obtain keywords, abstracts, and question-answer pairs of the target knowledge base; The keywords, the abstracts, the question-answer pairs, the document titles and the document contents of the target knowledge base are organized into the retrieval knowledge base.

5. The method according to claim 4, characterized in that The knowledge governance includes: format problem governance, paragraph-level information governance and chapter-level information governance.

6. A retrieval model determination device based on usage scenario, characterized in that: The device comprises: A scene identification unit is used to identify the usage scene; A first knowledge base determination unit, configured to determine a search knowledge base based on the usage scenario; A question set construction unit is used to construct a search question set applicable to the search knowledge base and having a result label; the search question set includes multiple search questions; the result label is used to mark the actual question and answer result of each search question in the search question set; A retrieval unit, used to input the retrieval question set into the retrieval model for retrieval, and obtain a model question and answer result set; A prediction unit, configured to obtain a prediction deviation based on a difference between the model question and answer result set and the result label; A model adjustment unit is used to adjust the retrieval model based on the prediction deviation.

7. The device according to claim 6, characterized in that The retrieval model includes a literal retrieval module, a semantic retrieval module and a deep sorting model connected in sequence; Among them, the variable parameters of the literal retrieval module include literal retrieval algorithm selection parameters, literal retrieval paragraph granularity parameters, and literal recall range parameters; the variable parameters of the semantic retrieval module include semantic representation model selection parameters, semantic recall index type selection parameters, semantic representation text length parameters, and semantic recall range parameters; the deep precise ranking model is constructed based on the Porter model and combined with the ranking review model of the ranking learning framework.

8. The device according to claim 7, characterized in that The model adjustment unit specifically includes: a first adjustment unit, configured to adjust a variable parameter of the literal retrieval module based on the literal retrieval recall rate if the prediction deviation indicates that the literal retrieval recall rate is lower than a first target recall rate; a second adjustment unit, configured to adjust a variable parameter of the semantic retrieval module based on the semantic retrieval recall rate if the prediction deviation indicates that the semantic retrieval recall rate is lower than a second target recall rate; The third adjustment unit is used to adjust the model parameters of the deep fine ranking model based on the fine ranking accuracy if the prediction deviation indicates that the fine ranking accuracy is lower than the target accuracy.

9. A retrieval model determination device based on usage scenarios, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining a retrieval model based on a usage scenario as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method for determining a retrieval model based on a usage scenario as described in any one of claims 1 to 5.

Images