Model determination method and device, equipment and medium

Abstract

The embodiment of the invention relates to a model determination method and device, equipment and a medium. The method comprises the steps of obtaining a plurality of model component combinations; wherein each model component combination comprises at least one retrieval generation component, and different model component combinations comprise different retrieval generation components; respectively combining and integrating each model component into the basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations are in one-to-one correspondence with the to-be-tested retrieval generation models; evaluating the plurality of to-be-tested retrieval generation models to obtain a plurality of model evaluation results; and according to the plurality of model evaluation results, determining a target retrieval generation model matched with the current problem in the plurality of to-be-tested retrieval generation models. Therefore, the automatic determination of the retrieval generation model is realized based on wide evaluation, and the efficiency of determining the retrieval generation model is improved while the retrieval generation model is relatively adaptive to the current problem.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to a model determination method, device, equipment and medium. Background Art

[0002] With the development of natural language processing technology, retrieval-augmented generation (RAG) technology has been widely used to improve the performance of language models. In related technologies, there are multiple components built based on retrieval-augmented generation technology. It is necessary to manually select components based on human experience, and combine the selected components to obtain a model, and then answer questions through the model. However, the model determination in this method relies on human experience, and it is difficult to efficiently determine a model suitable for the current problem. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a model determination method, device, equipment and medium.

[0004] The present disclosure provides a model determination method, including:

[0005] Acquire multiple model component combinations; wherein each of the model component combinations includes at least one retrieval generation component, and different model component combinations include different retrieval generation components;

[0006] Integrate each of the model component combinations into a basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations correspond one to one to the retrieval generation models to be tested;

[0007] Evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results;

[0008] According to the multiple model evaluation results, a target retrieval generation model matching the current question among the multiple retrieval generation models to be tested is determined.

[0009] The present disclosure also provides a model determination device, including:

[0010] An acquisition module, used for acquiring a plurality of model component combinations; wherein each of the model component combinations includes at least one retrieval generation component, and different model component combinations include different retrieval generation components;

[0011] An integration module, used to integrate each of the model component combinations into a basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations correspond one to one to the retrieval generation models to be tested;

[0012] An evaluation module, used for evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results;

[0013] A matching module is used to determine a target retrieval generation model that matches the current question among the multiple retrieval generation models to be tested based on the evaluation results of the multiple models.

[0014] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the model determination method provided by the embodiment of the present disclosure.

[0015] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the model determination method provided by the embodiment of the present disclosure.

[0016] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: the model determination solution provided in the embodiment of the present disclosure, the method includes: obtaining multiple model component combinations; wherein each model component combination includes at least one retrieval generation component, and different model component combinations include different retrieval generation components; respectively integrating each model component combination into the basic retrieval generation model to obtain multiple retrieval generation models to be tested; wherein the model component combination corresponds to the retrieval generation model to be tested one by one; evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results; according to the multiple model evaluation results, determining the target retrieval generation model that matches the current problem among the multiple retrieval generation models to be tested. By adopting the above technical solution, a relatively comprehensive retrieval generation component matching situation is covered through the model component combination, a retrieval generation model to be tested with a relatively comprehensive component matching situation is generated according to the model component combination, and the target retrieval generation model adapted to the current problem is determined according to the evaluation result of the retrieval generation model to be tested, and the automatic determination of the retrieval generation model is realized based on the extensive evaluation, while making the retrieval generation model more adapted to the current problem, the efficiency of determining the retrieval generation model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of a model determination method provided in an embodiment of the present disclosure;

[0020] Figure 2 A flowchart of another model determination method provided in an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of a model determination method provided in an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of the structure of a model determination device provided in an embodiment of the present disclosure;

[0023] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0026] With the development of natural language processing technology, retrieval enhancement generation technology has been widely used to improve the performance of language models. In related technologies, there are multiple components built based on retrieval enhancement generation technology. It is necessary to manually select components based on human experience, and combine the selected components to obtain a model, and then answer questions through the model.

[0027] However, the model finally determined in this method relies on manual experience, the selection and combination of components is time-consuming and complicated, and the combined generated model needs to be manually evaluated, so it is difficult to efficiently determine the model that is suitable for the current problem. In addition, as the amount of data increases, the data covers more and more fields, and the currently determined model may not be applicable to the field, and it is difficult to determine the model that is suitable for the field. There is an urgent need for a technical solution for automatically optimizing and evaluating the combination of retrieval enhancement generation components.

[0028] In order to solve the above problems, the embodiments of the present disclosure provide a model determination method, which is introduced below in conjunction with specific embodiments.

[0029] Figure 1The present invention provides a flow chart of a model determination method provided in an embodiment of the present invention. The model determination method can be applied to a model determination device. The model determination device can be implemented by software and / or hardware, and the model determination device can generally be integrated in an electronic device. Figure 1 As shown, the model determination method includes:

[0030] Step 101, obtaining a plurality of model component combinations; wherein each model component combination includes at least one retrieval generation component, and different model component combinations include different retrieval generation components.

[0031] Among them, the retrieval generation component can be a component built based on the retrieval enhancement generation technology. This embodiment does not limit the type of the retrieval generation component. The retrieval generation component can be understood as a node in the processing flow (pipeline) based on which the model is constructed. The retrieval enhancement generation technology can first retrieve the data content related to the input question from the preset database, and then use the data content to guide the answer to the input question and generate the answer. Thereby, the quality and accuracy of the prediction are improved by searching the preset database. The combination of multiple model components can be a set obtained by combining multiple predetermined alternative retrieval generation components. Optionally, the combination of model components can be a full combination determined based on multiple alternative retrieval generation components. The combination of model components can be a set built based on the retrieval generation component.

[0032] In the embodiment of the present disclosure, a plurality of alternative retrieval generation components are pre-set, and the model matching device can select N retrieval generation components from the plurality of alternative retrieval generation components multiple times to form corresponding multiple model component combinations, wherein N can be set according to user needs, etc., and this embodiment does not impose any restrictions. For example, N can be 2, 3, etc.

[0033] In some embodiments of the present disclosure, the retrieval generation component includes at least one of a query expansion component, a paragraph enhancement component, a paragraph rearrangement component, a prompt word creation component, and an information source component.

[0034] Among them, the query expansion component is used to determine the background content according to the pending problem, and rewrite the pending problem according to the background content. Specifically, the pending problem can be a problem of the user input model determination device, and the background content can be the content related to the pending problem in the preset database. The preset database can be a pre-configured knowledge base. In this embodiment, a pending problem can be decomposed into multiple decomposition problems through a large language model (LargeLanguage Model, LLM) through the query expansion component, and the decomposition problem can be understood as a sub-problem of the pending problem. New pending problems are regenerated according to the multiple decomposition problems and the background content to achieve the rewriting of the pending problem. Optionally, the rewriting can be to generate a hypothetical paragraph based on the background content, and semantic fusion is performed according to the hypothetical paragraph and the decomposition problem. Thereby, by rewriting the pending problem, the similarity between the new pending problem and the database in the preset database is improved, and the accuracy of subsequent query processing is improved.

[0035] The paragraph enhancement component is used to retrieve and process the preset database according to the problem to be processed to obtain background content. In this embodiment, the preset database can be queried through the paragraph enhancement component to determine the background knowledge related to the problem to be processed in the preset database. In order to perform query expansion, paragraph rearrangement, prompt word generation and other processing based on the background knowledge, the paragraph enhancement component can be used to supplement or replace the paragraph enhancement function in other components.

[0036] The paragraph rearrangement component is used to reorder the data content determined based on the preset database. Specifically, the data content can be the content determined by querying the preset database, and the data content can include background content, prediction results corresponding to the problem to be processed, etc. In this embodiment, the data content queried from the preset database can be input into the paragraph rearrangement model through the paragraph rearrangement component, and the data content is reordered through the paragraph rearrangement model to adjust the relevance ranking of the data content and the problem to be processed, thereby improving the accuracy of the final answer to the question.

[0037] The prompt word creation component is used to generate prompt words according to the background content. Specifically, the prompt words can be used to generate specific content for the prompt model. In this embodiment, the prompt word creation component can combine the background content queried according to the problem to be processed with the pre-set prompt word template to obtain the prompt words input to the model. Therefore, the context information related to the problem to be processed is provided to the model based on the prompt words.

[0038] The information source component is used to access a preset database of a corresponding type. Specifically, the present embodiment does not limit the type of the preset database. For example, the preset database may include an external database and an internal database. The external database may include an Internet data area, and the internal database may include a database within the company. In the present embodiment, different information source components may correspond to different types of preset databases. Different types of preset databases may be applied by accessing different information source components.

[0039] Step 102, respectively integrate each model component combination into the basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combination corresponds to the retrieval generation model to be tested one by one.

[0040] The basic search generation model can be used as a basic model for adding components, and this embodiment does not limit the type of the basic search generation model. The search generation model to be tested can be a search generation model for which performance testing is to be performed.

[0041] In the embodiment of the present disclosure, for each model component combination, the model determination device may integrate the retrieval generation component included in the model component combination into the basic retrieval generation model to obtain the retrieval generation model to be tested corresponding to the model component combination.

[0042] In some embodiments of the present disclosure, each model component is integrated into a basic retrieval generation model to obtain multiple retrieval generation models to be tested, including:

[0043] For each model component combination, a processing flow is arranged according to the retrieval generation component and the basic retrieval generation model in the model component combination to obtain the retrieval generation model to be tested corresponding to the model component combination.

[0044] In this embodiment, a positional relationship between a retrieval generation component and the position of the component in the processing flow is pre-set. For each model component combination, the model determination device can match the retrieval generation component with the positional relationship to determine the component position corresponding to the retrieval generation component. Further, based on the basic retrieval generation model, each retrieval generation component is added to the corresponding component position to obtain the processing flow corresponding to the model component combination. The processing flow as a whole can be understood as the retrieval generation model to be tested corresponding to the model component combination.

[0045] In the above scheme, the retrieval generation component is integrated into the basic retrieval generation model in a process orchestration manner to obtain the retrieval generation model to be tested, thereby achieving efficient determination of the retrieval generation model to be tested.

[0046] Step 103, evaluating multiple retrieval generation models to be tested to obtain multiple model evaluation results.

[0047] The model evaluation result can be used to characterize the performance of the retrieval generation model to be tested in answering questions. A model evaluation result can include multiple evaluation results of the retrieval generation model to be tested in different problem domains. This embodiment does not limit the type of the model evaluation result. For example, the model evaluation result can include accuracy, precision, recall, etc.

[0048] In the embodiment of the present disclosure, after determining a plurality of retrieval generation models to be tested, the model determination device may perform evaluation processing on the plurality of retrieval generation models to obtain model evaluation results corresponding one-to-one to the retrieval generation models.

[0049] In some embodiments of the present disclosure, multiple retrieval generation models to be tested are evaluated to obtain multiple model evaluation results, including: obtaining multiple test question groups; wherein a test question group includes multiple test questions belonging to the same problem domain, and the test question group corresponds to the problem domain one-to-one; multiple retrieval generation models to be tested are evaluated through each test question group to obtain model evaluation results of each retrieval generation model to be tested in each problem domain.

[0050] The test question group may be a set of test questions. The test questions may be used to perform performance testing on the model, and the test questions may have corresponding standard answers. The problem domain may be the application domain to which the test questions belong. This embodiment does not limit the problem domain. For example, the application domain may include company processes, cutting-edge technologies, etc.

[0051] In this embodiment, a plurality of test questions are pre-set, and the test questions may have corresponding domain labels. The model determination device may classify the plurality of test questions according to the domain labels, and classify the test questions with the same domain labels into the same test question group, so as to obtain a test question group corresponding to the problem domain one by one. For each retrieval generation model to be tested, the retrieval generation model to be tested is evaluated by using a plurality of test question groups, and a plurality of evaluation results of the retrieval generation model to be tested determined based on the plurality of test question groups are obtained, that is, a plurality of evaluation results of the retrieval generation model to be tested in a plurality of problem domains are obtained, and the plurality of evaluation results are used as the model evaluation results of the retrieval generation model to be tested.

[0052] Specifically, for each test question, the test question can be input into the retrieval generation model to be tested, the predicted answer output by the retrieval generation model to be tested is obtained, and the text similarity between the predicted answer and the standard answer corresponding to the test question is determined. For each test question group, the model evaluation result of the retrieval generation model to be tested in the test question group is determined according to the text similarity corresponding to each test question in the test question group.

[0053] In the above scheme, the retrieval generation model to be tested is evaluated by a set of test questions corresponding one to one with the problem domain, thereby realizing the evaluation of the answering ability of the retrieval generation model to be tested in different problem domains, and creating a basis for the subsequent dynamic selection of models for problems in different problem domains.

[0054] Step 104: Determine a target retrieval generation model that matches the current question among the multiple retrieval generation models to be tested based on the multiple model evaluation results.

[0055] The current question may be a question to be answered, and this embodiment does not limit the problem domain of the current question. The target retrieval generation model may be a retrieval generation model for processing the current question, and the domain corresponding to the target retrieval generation model may be the same as or contain the domain corresponding to the current question.

[0056] In the disclosed embodiment, the model determination device can determine the target retrieval generation model for processing the current problem according to the evaluation results of each retrieval generation model to be tested in each problem domain in the model evaluation results, and the current domain corresponding to the current problem. Afterwards, the model determination device can input the current problem into the target retrieval generation model to obtain the current prediction result corresponding to the current problem output by the target retrieval generation model.

[0057] Figure 2 A flow chart of another model determination method provided in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, in some embodiments of the present disclosure, according to multiple model evaluation results, determining a target retrieval generation model that matches the current question among multiple retrieval generation models to be tested includes:

[0058] Step 201 : for each problem domain, the retrieval generation model to be tested with the best model evaluation result in the problem domain is determined as the domain retrieval generation model corresponding to the problem domain.

[0059] The domain retrieval generation model may be a retrieval generation model for processing problems in a specific problem domain, and a domain retrieval generation model may correspond to one or more problem domains.

[0060] In this embodiment, for each problem domain, the model determination device can obtain multiple evaluation results under the problem domain from multiple model evaluation results, and determine the retrieval generation model to be tested corresponding to the optimal value among the multiple evaluation results as the domain retrieval generation model corresponding to the problem domain.

[0061] Step 202, determining a target retrieval generation model according to a domain retrieval generation model corresponding to a problem domain matching the current domain; wherein the current domain is the problem domain to which the current problem belongs.

[0062] In this embodiment, the model determination device can perform text analysis on the current question to determine the current field to which the current field belongs. And perform text matching on the current field and the problem field to determine the problem field with the highest matching degree, and determine the field retrieval generation model corresponding to the problem field as the target retrieval generation model, or perform fine-tuning processing based on the field retrieval generation model corresponding to the problem field to determine the target retrieval generation model. Thus, the target retrieval generation model is flexibly configured according to the field to which the current question belongs, so that the final determined model has a high adaptability to the current question. Based on the model determination method as a whole, problems in different fields can be processed, which enhances the field extensibility of the model processing method. Since the correspondence between the model and the problem field is predetermined, the complexity of determining the model when answering questions across fields is reduced.

[0063] In some embodiments of the present disclosure, determining a target retrieval generation model according to a domain retrieval generation model corresponding to a problem domain matching the current domain includes:

[0064] The domain retrieval generation model corresponding to the problem domain matching the current domain is determined as the intermediate retrieval generation model; the intermediate retrieval generation model is fine-tuned to obtain a fine-tuned retrieval generation model; if the model evaluation result of the fine-tuned retrieval generation model is better than the model evaluation result of the intermediate retrieval generation model, the fine-tuned retrieval generation model is determined as the target retrieval generation model.

[0065] The intermediate retrieval generation model may be a model determined in the intermediate process of determining the target retrieval generation model. The intermediate retrieval generation model may be a domain retrieval generation model with the highest matching degree between the problem domain and the current domain. The fine-tuning retrieval generation model may be a model obtained by performing one or more of the following processes on the intermediate retrieval generation model: deleting, adding, or replacing components.

[0066] In this embodiment, the model determination device can perform text analysis on the current problem to determine the current field to which the current field belongs. And perform text matching on the current field and the problem field to determine the problem field with the best matching degree, and determine the field retrieval generation model corresponding to the problem field as the intermediate retrieval generation model. Further, a retrieval generation component is randomly determined from multiple retrieval generation components. If the retrieval generation component is included in the intermediate retrieval generation model, the retrieval generation component in the intermediate retrieval generation model is deleted or replaced; if the retrieval generation component is not included in the intermediate retrieval generation model, the retrieval generation component is added to the intermediate retrieval generation model. After the above-mentioned fine-tuning processing is performed on the intermediate retrieval generation model respectively, multiple fine-tuned retrieval generation models are obtained.

[0067] Furthermore, the model determination device can use the evaluation data corresponding to the problem domain matching the current domain to evaluate the fine-tuned retrieval generation model, wherein the evaluation data can be the entire test problem set under the problem domain, partial evaluation data, or other data for model evaluation in the problem domain. Then, the model evaluation result corresponding to the fine-tuned retrieval generation model under the problem domain is obtained. It is determined whether the model evaluation result is better than the evaluation result under the problem domain in the model evaluation result of the intermediate retrieval generation model. If so, the fine-tuned retrieval generation model is determined as the target retrieval generation model, otherwise, the intermediate retrieval generation model is determined as the target retrieval generation model.

[0068] In the above scheme, the determined intermediate retrieval generation model is fine-tuned and the fine-tuned model is evaluated. When the model component combination is not a complete combination of retrieval generation components, by fine-tuning the model, it is possible to judge whether there is a model with better performance based on the currently determined model, thereby further improving the performance of the final determined model.

[0069] The model determination method provided by the embodiment of the present disclosure includes: obtaining multiple model component combinations; wherein each model component combination includes at least one retrieval generation component, and different model component combinations include different retrieval generation components; respectively integrating each model component combination into the basic retrieval generation model to obtain multiple retrieval generation models to be tested; wherein the model component combination corresponds to the retrieval generation model to be tested one by one; evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results; and determining the target retrieval generation model that matches the current problem among the multiple retrieval generation models to be tested based on the multiple model evaluation results. By adopting the above technical scheme, a relatively comprehensive retrieval generation component matching situation is covered through the model component combination, a retrieval generation model to be tested with a relatively comprehensive component matching situation is generated based on the model component combination, and the target retrieval generation model adapted to the current problem is determined based on the evaluation result of the retrieval generation model to be tested, and the automatic determination of the retrieval generation model is realized based on the extensive evaluation, which improves the efficiency of determining the retrieval generation model while making the retrieval generation model more adapted to the current problem.

[0070] Next, the model determination method in the embodiment of the present disclosure is further described by using a specific example. Figure 3 A schematic diagram of a model determination method provided by an embodiment of the present disclosure is shown as follows: Figure 3 As shown, in this example, the model determination device can modularize the retrieval generation components and dynamically select the retrieval generation component that is most suitable for processing the current problem to construct a better processing flow. Specifically, the model determination method may include:

[0071] First, the original document is processed to generate multiple test question groups. Each test question has a standard answer. By determining the text similarity between the predicted answer generated by the model and the standard answer, the test result of the test question on the model can be determined.

[0072] Further, through the evaluation of the test problem group, a target retrieval generation model is generated based on the retrieval generation component. Specifically, multiple model component combinations are determined, and the retrieval generation component in each module component combination can be selected and determined from multiple alternative retrieval generation components. For example, there are 4 alternative retrieval generation components, and the model component combination can be selected from 4. The component position of each retrieval generation component in the processing flow is pre-set, and then multiple retrieval generation models to be tested are determined according to multiple model component combinations. The multiple retrieval generation models to be tested are experimentally evaluated. The domain retrieval generation model corresponding to each problem domain is determined according to the model evaluation results. For example, for company processes, the domain retrieval generation model can be model 1; for cutting-edge knowledge, the domain retrieval generation model can be model 2. The domain retrieval generation model corresponding to the problem domain matching the current domain of the current problem is determined as an intermediate retrieval generation model, and the intermediate retrieval generation model is fine-tuned and optimized to further improve the performance of the model. For example, the extended query component of the intermediate retrieval generation model can be added or replaced. Further, the target retrieval generation model is deployed online.

[0073] In the model determination method provided by the embodiment of the present disclosure, the retrieval generation component is combined with the basic retrieval generation model in a modular manner, and the modular approach automatically realizes the simple and fast determination of the model component combination suitable for the current problem. The model-based determination strategy improves the efficiency of determining the retrieval generation component, and there are corresponding domain retrieval generation models in different problem fields, so that the method can handle problems in different fields, thereby improving the scalability of the method and being suitable for a wider range of application scenarios.

[0074] Figure 4 This is a schematic diagram of the structure of a model determination device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware, and the device can also be integrated in an electronic device. Figure 4 As shown, the model determination device includes:

[0075] The acquisition module 401 is used to acquire multiple model component combinations; wherein each of the model component combinations includes at least one retrieval generation component, and different model component combinations include different retrieval generation components;

[0076] An integration module 402 is used to integrate each of the model component combinations into a basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations correspond to the retrieval generation models to be tested one by one;

[0077] An evaluation module 403 is used to evaluate the multiple retrieval generation models to be tested to obtain multiple model evaluation results;

[0078] The matching module 404 is used to determine a target retrieval generation model that matches the current question among the multiple retrieval generation models to be tested according to the multiple model evaluation results.

[0079] Optionally, the retrieval generation component includes at least one of a query expansion component, a paragraph enhancement component, a paragraph rearrangement component, a prompt word creation component, and an information source component.

[0080] Optionally, the query expansion component is used to determine the background content according to the problem to be processed, and to rewrite the problem to be processed according to the background content; the paragraph enhancement component is used to retrieve the preset database according to the problem to be processed to obtain the background content; the paragraph rearrangement component is used to reorder the data content determined based on the preset database; the prompt word creation component is used to generate prompt words according to the background content; the information source component is used to access the preset database of the corresponding type.

[0081] Optionally, the integration module 402 is used to:

[0082] For each of the model component combinations, a processing flow is arranged according to the retrieval generation component and the basic retrieval generation model in the model component combination to obtain the retrieval generation model to be tested corresponding to the model component combination.

[0083] Optionally, the evaluation module 403 is used to:

[0084] Acquire multiple test question groups; wherein one of the test question groups includes multiple test questions belonging to the same problem domain, and the test question group corresponds to the problem domain one by one;

[0085] The plurality of retrieval generation models to be tested are evaluated through each of the test problem groups to obtain a model evaluation result of each of the retrieval generation models to be tested in each of the problem areas.

[0086] Optionally, the matching module 404 includes:

[0087] A determination submodule is used to determine, for each of the problem domains, the retrieval generation model to be tested with the best model evaluation result in the problem domain as the domain retrieval generation model corresponding to the problem domain;

[0088] The matching submodule is used to determine the target retrieval generation model according to the domain retrieval generation model corresponding to the problem domain matching the current domain; wherein the current domain is the problem domain to which the current problem belongs.

[0089] Optionally, the matching submodule is used to:

[0090] Determine the domain retrieval generation model corresponding to the problem domain matching the current domain as the intermediate retrieval generation model;

[0091] The intermediate retrieval generation model is fine-tuned to obtain a fine-tuned retrieval generation model; if a model evaluation result of the fine-tuned retrieval generation model is better than a model evaluation result of the intermediate retrieval generation model, the fine-tuned retrieval generation model is determined as the target retrieval generation model.

[0092] It should be noted that Figure 4 The model determination device 400 shown can execute each step in the above-mentioned model determination method embodiment, and realize each process and effect in the above-mentioned model determination method embodiment, which will not be described in detail here.

[0093] Figure 5 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the electronic device 500 includes one or more processors 501 and a memory 502 .

[0094] The processor 501 may be a central processing unit (CPU) or other forms of processing units having a model determination capability and / or an instruction execution capability, and may control other components in the electronic device 500 to perform desired functions.

[0095] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may run the program instructions to implement the model determination method of the embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0096] In one example, the electronic device 500 may further include: an input device 503 and an output device 504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0097] In addition, the input device 503 may also include, for example, a keyboard, a mouse, and the like.

[0098] The output device 504 can output various information to the outside, including the determined distance information, direction information, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0099] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device 500 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 500 may also include any other appropriate components.

[0100] In addition to the above-mentioned method and device, the embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the processor executes the model determination method provided by the embodiment of the present disclosure.

[0101] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0102] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the model determination method provided by the embodiment of the present disclosure.

[0103] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0104] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0105] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model determination method, characterized in that: include: Acquire multiple model component combinations; wherein each of the model component combinations includes at least one retrieval generation component, and different model component combinations include different retrieval generation components; Integrate each of the model component combinations into a basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations correspond one to one to the retrieval generation models to be tested; Evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results; According to the multiple model evaluation results, a target retrieval generation model matching the current question among the multiple retrieval generation models to be tested is determined.

2. The method according to claim 1, characterized in that The retrieval generation component includes at least one of a query expansion component, a paragraph enhancement component, a paragraph rearrangement component, a prompt word creation component, and an information source component.

3. The method according to claim 2, characterized in that The query expansion component is used to determine the background content according to the problem to be processed, and rewrite the problem to be processed according to the background content; the paragraph enhancement component is used to retrieve the preset database according to the problem to be processed to obtain the background content; The paragraph rearrangement component is used to reorder the data content determined based on the preset database; the prompt word creation component is used to generate prompt words according to the background content; The information source component is used to access the preset database of the corresponding type.

4. The method according to claim 1, characterized in that: The model components are respectively combined and integrated into the basic retrieval generation model to obtain multiple retrieval generation models to be tested, including: For each of the model component combinations, a processing flow is arranged according to the retrieval generation component and the basic retrieval generation model in the model component combination to obtain the retrieval generation model to be tested corresponding to the model component combination.

5. The method according to claim 1, characterized in that: The step of evaluating the plurality of retrieval generation models to be tested to obtain a plurality of model evaluation results includes: Acquire multiple test question groups; wherein one of the test question groups includes multiple test questions belonging to the same problem domain, and the test question group corresponds to the problem domain one by one; The plurality of retrieval generation models to be tested are evaluated through each of the test problem groups to obtain a model evaluation result of each of the retrieval generation models to be tested in each of the problem areas.

6. The method according to claim 5, characterized in that Determining a target retrieval generation model that matches the current question among the multiple retrieval generation models to be tested according to the multiple model evaluation results includes: For each of the problem domains, the retrieval generation model to be tested with the best model evaluation result in the problem domain is determined as the domain retrieval generation model corresponding to the problem domain; The target retrieval generation model is determined according to a domain retrieval generation model corresponding to a problem domain matching the current domain; wherein the current domain is the problem domain to which the current problem belongs.

7. The method according to claim 6, characterized in that The step of determining the target retrieval generation model according to the domain retrieval generation model corresponding to the problem domain matching the current domain includes: Determine the domain retrieval generation model corresponding to the problem domain matching the current domain as the intermediate retrieval generation model; The intermediate retrieval generation model is fine-tuned to obtain a fine-tuned retrieval generation model; if a model evaluation result of the fine-tuned retrieval generation model is better than a model evaluation result of the intermediate retrieval generation model, the fine-tuned retrieval generation model is determined as the target retrieval generation model.

8. A model determination device, characterized in that: include: An acquisition module, used for acquiring a plurality of model component combinations; wherein each of the model component combinations includes at least one retrieval generation component, and different model component combinations include different retrieval generation components; An integration module, used to integrate each of the model component combinations into a basic retrieval generation model to obtain a plurality of retrieval generation models to be tested; wherein the model component combinations correspond one to one to the retrieval generation models to be tested; An evaluation module, used for evaluating the multiple retrieval generation models to be tested to obtain multiple model evaluation results; A matching module is used to determine a target retrieval generation model that matches the current question among the multiple retrieval generation models to be tested based on the evaluation results of the multiple models.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the model determination method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the model determination method described in any one of claims 1 to 7.

Images