Method and device for carrying out model selection by utilizing subgraph execution engine

Obtain model selection strategies and service dependencies through the subgraph execution engine, and dynamically select large models or small models, solving the problem of rational use of large models and small models. It is suitable for various inference task scenarios, reducing resource consumption and improving output flexibility .

CN118940858AActive Publication Date: 2024-11-12BEIJING YOUZHUJU NETWORK TECH CO LTD

Patent Information

Application Number
CN202411018103.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-12
Estimated Expiration
2044-07-26

AI Technical Summary

Technical Problem

How to rationally use large models and small models to perform inference tasks requires not only reducing resource consumption, but also ensuring the accuracy of inference results. The existing technology is difficult to apply to various application scenarios.

Method used

The model selection strategy for the task to be reasoned is obtained through the sub-graph execution engine, the target service is determined based on the dependencies between services, and the target model (large model or small model) is selected based on the model selection strategy and service value to realize dynamic selection of the model.

Benefits of technology

It realizes dynamic selection of models according to task requirements, rationally utilizes large models and small models, and is suitable for various scenarios, reducing resource consumption of inference tasks, and improving the flexibility of output format.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940858A_ABST
    Figure CN118940858A_ABST
Patent Text Reader

Abstract

The invention discloses a method for carrying out model selection by utilizing a subgraph execution engine, which comprises the following steps: obtaining a model selection strategy corresponding to a to-be-reasoned task, the model selection strategy is used for selecting a model for executing the to-be-reasoned task, and the model selection strategy comprises at least one field and a judgment condition corresponding to the at least one field; and according to a dependency relationship between services pre-stored in the sub-graph execution engine, determining a target service needing to be executed for obtaining the at least one field. The target service is executed to derive a value of the at least one field. According to the value of the at least one field and a model selection strategy, a target model for executing the to-be-reasoned task is determined, and the target model is a large model or a small model. Therefore, according to the scheme, whether the large model or the small model is adopted to execute the to-be-reasoned task can be determined based on the model selection strategy, and the scheme has no strict requirement on the output format of the small model, so that the scheme can select the corresponding models for the to-be-reasoned tasks corresponding to various scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for performing model selection using a subgraph execution engine. Background Art

[0002] With the development of large models, more and more reasoning tasks can be implemented using large models, where the large model can be, for example, a large language model (LLM) or a multimodal large language model (MLLM).

[0003] However, large language models and multimodal large language models consume more resources for reasoning, while small models consume relatively fewer resources for task reasoning. Therefore, it is particularly important to use large models and small models to perform reasoning tasks.

[0004] How to reasonably use large models and small models to perform reasoning tasks is a problem that has yet to be solved. Summary of the invention

[0005] In order to solve or at least partially solve the above technical problems, the present application provides a method and device for model selection using a subgraph execution engine.

[0006] In a first aspect, the present application provides a method for performing model selection using a subgraph execution engine, the method comprising:

[0007] Obtaining a model selection strategy corresponding to the task to be inferred, wherein the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field;

[0008] Determining a target service required to be executed to obtain the at least one field according to dependencies between services pre-stored in the subgraph execution engine;

[0009] executing the target service to obtain a value of the at least one field;

[0010] A target model for executing the task to be inferred is determined according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

[0011] Optionally, the dependency relationship between services pre-stored in the subgraph execution engine includes:

[0012] At least one directed acyclic graph, each directed acyclic graph in the at least one directed acyclic graph is used to indicate a group of services with dependencies, wherein a node in the directed acyclic graph corresponds to a service, the direction of the edge between the nodes indicates the upstream and downstream relationship between the nodes, and the edge between the nodes also indicates the fields transmitted between the nodes.

[0013] Optionally, determining the target service required to be executed to obtain the at least one field according to the dependency relationship between the services pre-stored in the subgraph execution engine includes:

[0014] For each field of the at least one field, determine a first service that generates the field, and search for a target directed acyclic graph including the first service;

[0015] Determining the target service includes the first service in the target directed acyclic graph and a second service upstream of the first service.

[0016] Optionally, obtaining a model selection strategy corresponding to the task to be inferred includes:

[0017] The model selection strategy input by the user in the strategy configuration page provided by the strategy configuration platform is obtained.

[0018] Optionally, determining a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy includes:

[0019] Calling a strategy engine to judge the model selection strategy using the value of the at least one field, determining whether a judgment condition corresponding to the at least one field is satisfied, and obtaining a judgment result;

[0020] According to the judgment result, a target model for executing the task to be inferred is determined.

[0021] Optionally, the large model includes: a large language model or a multimodal large language model, and the small model includes other reasoning models except the large language model and the multimodal large language model.

[0022] In a second aspect, the present application provides a device for performing model selection using a subgraph execution engine, the device comprising:

[0023] an acquisition unit, configured to acquire a model selection strategy corresponding to the task to be inferred, wherein the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field;

[0024] A first determining unit, configured to determine a target service required to be executed to obtain the at least one field according to dependencies between services pre-stored in the subgraph execution engine;

[0025] an execution unit, configured to execute the target service to obtain a value of the at least one field;

[0026] The second determining unit is used to determine a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

[0027] Optionally, the dependency relationship between services pre-stored in the subgraph execution engine includes:

[0028] At least one directed acyclic graph, each directed acyclic graph in the at least one directed acyclic graph is used to indicate a group of services with dependencies, wherein a node in the directed acyclic graph corresponds to a service, the direction of the edge between the nodes indicates the upstream and downstream relationship between the nodes, and the edge between the nodes also indicates the fields transmitted between the nodes.

[0029] Optionally, the first determining unit is configured to:

[0030] For each field of the at least one field, determine a first service that generates the field, and search for a target directed acyclic graph including the first service;

[0031] Determining the target service includes the first service in the target directed acyclic graph and a second service upstream of the first service.

[0032] Optionally, the acquisition unit is used to:

[0033] The model selection strategy input by the user in the strategy configuration page provided by the strategy configuration platform is obtained.

[0034] Optionally, the second determining unit is configured to:

[0035] Calling a strategy engine to judge the model selection strategy using the value of the at least one field, determining whether a judgment condition corresponding to the at least one field is satisfied, and obtaining a judgment result;

[0036] According to the judgment result, a target model for executing the task to be inferred is determined.

[0037] Optionally, the large model includes: a large language model or a multimodal large language model, and the small model includes other reasoning models except the large language model and the multimodal large language model.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising a processor and a memory;

[0039] The processor is configured to execute instructions stored in the memory, so that the device performs the method as described in any one of the first aspects above.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0042] Compared with the prior art, the embodiments of the present application have the following advantages:

[0043] The present application provides a method for model selection using a subgraph execution engine, the method comprising: obtaining a model selection strategy corresponding to a task to be inferred, the model selection strategy being used to select a model for executing the task to be inferred, the model selection strategy comprising at least one field and a judgment condition corresponding to the at least one field. Further, according to the dependency relationship between services pre-stored in the subgraph execution engine, the target service required to be executed to obtain the at least one field is determined. After determining the target service, the target service can be further executed to obtain the value of the at least one field. And according to the value of the at least one field and the model selection strategy, the target model for executing the task to be inferred is determined, the target model being a large model or a small model. It can be seen from this that, using this solution, it is possible to determine whether to use a large model or a small model to execute the task to be inferred based on the model selection strategy, thereby reasonably using a large model or a small model to execute the task to be inferred. This solution has no strict requirements on the output format of the small model, therefore, this solution can select corresponding models for tasks to be inferred corresponding to various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a method for selecting a model using a subgraph execution engine provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a process of a method for selecting a model using a subgraph execution engine provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a process of another method for selecting a model using a subgraph execution engine provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of a device for performing model selection using a subgraph execution engine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The inventor of this application has found through research that, in order to reduce the resources consumed in executing reasoning tasks, it is hoped that small models can be used to execute reasoning tasks as much as possible. However, not all reasoning tasks can be executed using small models, because for some complex tasks, the reasoning results of small models may not be accurate. Therefore, in some scenarios:

[0051] The method of throwing stones for sampling can be used to determine whether the small model can be used to perform the reasoning task. Specifically, the small model can be used to generate a draft token for the reasoning task, and the large model can be used to determine whether the draft token can be adopted. If the large model determines that the draft token can be adopted, the reasoning result of the reasoning task is generated based on the draft token. If the large model determines that the draft token cannot be adopted, the large model is selected to perform the reasoning task.

[0052] However, this method requires that the small model outputs tokens that can be used to generate the inference results of the inference task, that is, the small model is required to be a generative model. In actual applications, the type of small models is not limited to generative models, so the stone throwing sampling method is not suitable for all application scenarios.

[0053] In view of this, the present application provides a method and device for model selection using a subgraph execution engine, which has no strict requirements on the output format of the small model and can select corresponding models for tasks to be inferred corresponding to various scenarios.

[0054] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0055] Exemplary Methods

[0056] See also Figure 1 , which is a flow chart of a method for selecting a model using a subgraph execution engine provided in an embodiment of the present application. The solution provided in an embodiment of the present application can be applied to a client or a server. As a specific example, the method can be applied to a subgraph execution engine deployed in a client or a server.

[0057] In this embodiment, the method may include the following steps: S101-S104.

[0058] S101: Obtain a model selection strategy corresponding to the task to be inferred, where the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field.

[0059] In an embodiment of the present application, the model selection strategy may be a strategy set by a user for the task to be inferred, and the model selection strategy is used to select a model for executing the task to be inferred. In other words, the model selection strategy is used to determine whether to use a large model or a small model to execute the task to be inferred.

[0060] In the embodiment of the present application, the large model can be a large language model or a multimodal large language model, and the small language model is a model other than the large language model and the multimodal large language model. The embodiment of the present application does not specifically limit the type of the small model, and the small model can be a classification model, a generation model, or other types of models, which are not listed here one by one.

[0061] In one example, the model selection strategy may be input by a user through a human-computer interaction interface.

[0062] In another example, the model selection strategy may be input by the user in a strategy configuration page provided by the strategy configuration platform. In one example, the strategy configuration page may include a strategy input area, and the user may input the model selection strategy in the strategy input area. The user may input the corresponding model selection strategy in the strategy configuration page according to his / her own needs.

[0063] In an embodiment of the present application, the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field. The judgment condition corresponding to the at least one field may include one or more sub-conditions, and a sub-condition may be associated with one or more fields of the aforementioned at least one field. The judgment condition corresponding to the at least one field may be, for example, a condition that needs to be satisfied when executing the task to be inferred using a large model, or a condition that needs to be satisfied when executing the task to be inferred using a small model, which is not specifically limited in the embodiment of the present application.

[0064] Regarding the fields included in the model selection strategy, they may be related to the task to be inferred or to the load of the large model (such as the calling frequency), and the embodiments of the present application do not make specific limitations.

[0065] S102: Determine a target service that needs to be executed to obtain the at least one field according to the dependency relationship between services pre-stored in the subgraph execution engine.

[0066] In the embodiment of the present application, the subgraph execution engine may pre-store the dependency relationship between services. Wherein, a service refers to a functional module that can implement a specific function. For example:

[0067] For the product library service, the service can output fields such as product title, product image, and product attributes based on the input product identifier (ID).

[0068] For the word segmentation service, it can segment the input sentence. The word segmentation service can segment the product title to obtain the corresponding word segmentation.

[0069] For the image recognition service, it can process the input image and identify the subject included in the image. The image recognition service can process the product image and identify the subject in the image.

[0070] For the keyword recognition service, it can perform keyword recognition on the input word segments and output the recognized keywords.

[0071] In the embodiment of the present application, the subgraph execution engine may store the identifiers of each service and the input fields and output fields of each service, which may be used to determine the dependency between services.

[0072] The so-called dependency relationship between services may include the association relationship between services. For a service, the services that have an association relationship with it may include its upstream services and / or its downstream services. The dependency relationship between services also includes the fields transferred between services. The fields transferred between services refer to the fields output by the upstream service to the downstream service directly associated with it. Regarding the dependency relationship between services, an example is given below: Service A outputs Field 1 to Service B, and Service B outputs Field 2 to Service C. In this scenario, Service A is the upstream service of Service B, and Service B is the upstream service of Service C. The field transferred from Service A to Service B is Field 1, and the field transferred from Service B to Service C is Field 2.

[0073] In a specific example, the dependency relationship between the aforementioned services can be embodied as at least one directed acyclic graph (DAG). Each DAG in the at least one DAG is used to indicate a group of services with dependencies, wherein each DAG may include multiple nodes, the edges between the multiple nodes are directed edges, and the direction of the directed edge is used to indicate the upstream and downstream relationship between the nodes. Specifically, the directed edge can be directed from the upstream node to the downstream node, the upstream node corresponds to the upstream service, and the downstream node corresponds to the downstream service. In addition, the directed edge can also indicate the fields transferred between nodes, wherein the fields transferred between nodes can be used as attributes of the directed edge. Regarding the DAG, it is now explained by taking the aforementioned service A outputting field 1 to service B, service B outputting field 2 to service C, and service C outputting field 3 as an example. In this scenario:

[0074] The DAG may include three nodes, one node for each service A, service B, and service C, wherein there is a directed edge between the node corresponding to service A and the node corresponding to service B, the directed edge points from the node corresponding to service A to the node corresponding to service B, and the directed edge also has attribute 1, which indicates that the field pointed to by service A is field 1. There is a directed edge between the node corresponding to service B and the node corresponding to service C, the directed edge points from the node corresponding to service B to the node corresponding to service C, and the directed edge also has attribute 2, which indicates that the field pointed to by service B is field 2.

[0075] In the embodiment of the present application, when S102 is specifically implemented, the method of determining the service required to be executed for obtaining each field in the at least one field is the same. Accordingly, the aforementioned target service may include the service required to be executed for obtaining each field in the at least one field.

[0076] For the convenience of description, any one of the at least one field is referred to as a "target field". Next, a method for determining a service to be executed to obtain the target field is described.

[0077] In one example, the target field can be used as an index to first search for a first service that outputs the target field. Further, based on the dependency relationship between the aforementioned services, at least one second service upstream of the first service is determined. The second service upstream of the first service mentioned here may include a service that directly passes the corresponding field to the first service (i.e., the direct upstream service of the first service). If the upstream service of the first service also has an upstream service, the second service also includes the upstream service of the upstream service of the first service. After determining the first service and the second service, it can be determined that the services required to obtain the target field include the first service and the second service. In one example, if the first service does not include an upstream service, the services required to obtain the target field may only include the first service.

[0078] In the scenario where the dependency between services is embodied as a DAG, S102, when implemented specifically, can first use the target field as an index to search for the first service that outputs the target field. Further, the target DAG including the first service is searched, for example, using the first service as an index, multiple pre-stored DAGs are queried to obtain the target DAG including the first service. In the present application, the target DAG includes the first service, which can be understood as the target service including the node corresponding to the first service. After determining the target DAG, the first service in the target DAG and the second service upstream of the first service can be determined as the services required to be executed to obtain the target field. Of course, the target DAG may also include only one node (i.e., the node corresponding to the first service). In this scenario, the services required to be executed to obtain the target field only include the first service.

[0079] S103: Execute the target service to obtain the value of the at least one field.

[0080] After determining the target service, the target service can be executed to obtain the value of the at least one field. Specifically, for each of the at least one field, the service required to obtain the field can be executed separately to obtain the value of the field. Taking the target field as an example, in a scenario where the service required to obtain the target field only includes the first service, the first service can be executed to obtain the value of the target field. In a scenario where the service required to obtain the target field includes the first service and the second service, the first service and the second service can be executed in the order of first executing the upstream service and then executing the downstream service to obtain the value of the target field.

[0081] In the scenario where the dependency between services is embodied as a DAG, S103 can be implemented, for example, by executing the target DAG to obtain the value of the target field. Executing the target DAG means executing the services in the target DAG in order according to the upstream and downstream relationships between the services indicated by the target DAG. Of course, when executing the target DAG, the first service can be executed instead of the downstream services of the first service, thereby saving computing resources.

[0082] S104: Determine a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

[0083] After obtaining the value of the at least one field, the model selection strategy can be judged based on the value of the at least one field to determine whether the judgment condition corresponding to the at least one field is satisfied, and a judgment result is obtained. In a specific example, the aforementioned policy configuration platform can be associated with a policy engine, and the subgraph execution engine can include a policy engine. Accordingly, in the specific implementation of S104, the policy engine can be called to judge the model selection strategy using the value of the at least one field to determine whether the judgment condition corresponding to the at least one field is satisfied, and a judgment result is obtained.

[0084] Furthermore, the target model for executing the task to be inferred can be determined based on the judgment result. For example, when the aforementioned judgment condition is the condition that needs to be met to execute the task to be inferred using a large model, if the judgment result indicates that the judgment condition corresponding to the at least one field is satisfied, then it can be determined that the target model for executing the task to be inferred is a large model; if the judgment result indicates that the judgment condition corresponding to the at least one field is not satisfied, then it can be determined that the target model for executing the task to be inferred is a small model. Similarly, when the aforementioned judgment condition is the condition that needs to be met to execute the task to be inferred using a small model, if the judgment result indicates that the judgment condition corresponding to the at least one field is satisfied, then it can be determined that the target model for executing the task to be inferred is a small model; if the judgment result indicates that the judgment condition corresponding to the at least one field is not satisfied, then it can be determined that the target model for executing the task to be inferred is a large model.

[0085] From the above description, it can be seen that using this solution, it is possible to determine whether to use a large model or a small model to perform the task to be reasoned based on the model selection strategy, so as to reasonably use the large model or the small model to perform the task to be reasoned. This solution has no strict requirements on the output format of the small model. Therefore, this solution can select corresponding models for the tasks to be reasoned corresponding to various scenarios, so that in various application scenarios, large models or small models can be reasonably used to perform the tasks to be reasoned.

[0086] The above is an introduction to the solution provided in the embodiment of the present application. Figure 2 and Figure 3 , and then introduce the solution provided by the embodiment of the present application. Among them, Figure 2 A schematic diagram of a method for selecting a model using a subgraph execution engine is shown. Figure 3 A process diagram of a method for selecting a model using a subgraph execution engine for the task of “predicting product categories” to be inferred.

[0087] like Figure 2 As shown, the subgraph execution engine includes a policy engine, which corresponds to a policy configuration platform. Users can configure model selection strategies on the policy configuration platform, such as Figure 2 As shown:

[0088] The model selection strategy involves field 1 and field 2. Figure 3 As shown, for the task to be inferred, "predict product category", field 1 is the product title, and field 2 is the image body. The subgraph execution engine obtains the model selection strategy through the strategy engine, and obtains the services required to obtain field 1 and field 2 based on at least one pre-saved DAG.

[0089] like Figure 2 As shown in the figure: To obtain field 1, you need to execute service X1 and service X. Service X1 is the upstream service of service X, and field 1 is the field output by service X. Execute services X1 and X in sequence to get the value of field 1. In the task to be inferred, "predict product category", service X1 is the product library service, and service X is the word segmentation service. For the product library service and word segmentation service, please refer to the relevant description in the previous article, and no repetition will be given here.

[0090] To obtain field 2, you need to execute service Y1 and service Y, where service Y1 is the upstream service of service Y, and field 2 is the field output by service Y. Execute service Y1 and service Y in sequence to get the value of field 2. In the task to be inferred, "predict product category", service Y1 is the product library service, and service Y is the image recognition service. For the image recognition service, please refer to the relevant description in the previous article, which will not be repeated here.

[0091] After obtaining the values ​​of field 1 and field 2, the strategy engine can be called to judge the model selection strategy based on the values ​​of the aforementioned field 1 and field 2 to obtain the judgment result. Figure 2 In the model selection strategy, field 1 satisfies condition A, field 2 satisfies condition B, and field 1 is first judged whether it satisfies condition A, and field 2 is judged whether it satisfies condition B only when field 1 does not satisfy condition A. However, Figure 2It is only shown to facilitate understanding of the present solution and does not constitute a limitation on the embodiments of the present application.

[0092] The embodiment of the present application does not specifically limit the field 1 to satisfying condition A. In one example, field 1 satisfies condition A, for example, the number of segmented words included in field 1 is greater than 10. The embodiment of the present application does not specifically limit the field 2 to satisfying condition B. In one example, field 2 satisfies condition B, for example, the number of entities included in field 2 is greater than 3.

[0093] After obtaining the judgment result, it is determined whether the model that executes the task to be inferred is a large model or a small model based on the judgment result.

[0094] It should be noted that for Figure 2 In the scheme shown, the user can adjust the model selection strategy through the policy configuration platform. For example, in one example, the user adjusts the model selection strategy through the policy platform and introduces a new field 3. In this scenario, the subgraph execution engine can determine the service required to execute to obtain field 3 in combination with the dependencies between the pre-saved services.

[0095] like Figure 2 As shown, to obtain field 3, you need to execute service Z1 and service Z. Service Z1 is the upstream service of service Z, and field 3 is the field output by service Z. By executing service Z1 and service Z in sequence, you can get the value of field 3 (corresponding to Figure 2 The dotted part in the figure) is provided as input to the strategy engine for judgment. In the task to be inferred for "predicting product categories", field 3 is the keyword, service Z1 is the product library service, and service Z is the keyword recognition service. Regarding the keyword recognition service, please refer to the relevant description in the previous text, and no repetitive description is made here. Correspondingly, the judgment condition corresponding to field 3, for example, can be that the keywords output by field 3 include "women's clothing". For example, when the keywords corresponding to field 3 include "women's clothing", the target model is determined to be a large model.

[0096] In this application, the newly added field 3 and the judgment condition corresponding to field 3 can take effect immediately without modifying the software code logic related to model selection, thereby ensuring the flexibility and convenience of model selection strategy configuration.

[0097] In summary, the solution provided by the embodiment of the present application has high flexibility and convenience for adjusting the fields and judgment conditions involved in the model selection strategy, and can easily and quickly reduce the frequency of using large models in reasoning tasks. In addition, this solution has no special requirements for the fields involved in the model selection strategy and the judgment conditions corresponding to the fields. Users can set them as needed, and it is highly scalable. For example, in addition to Figure 3In addition to the examples shown, the field involved in the model selection strategy can be the type of product. For type 1 (e.g., women's clothing), a large model is used, and for type 2 (e.g., non-women's clothing), a small model is used. For another example, the field involved in the model selection strategy can be the frequency of use of the large model. If the frequency of use is higher than a certain threshold, such as the query rate per second (QPS) is greater than 100, a small model is used, and if the QPS is less than or equal to 100, a small model is used, and so on. These are not listed here.

[0098] Exemplary Devices

[0099] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0100] See also Figure 4 , Figure 4 A schematic diagram of the structure of a device for performing model selection using a subgraph execution engine provided in an embodiment of the present application.

[0101] Figure 4 The device 400 for performing model selection using a subgraph execution engine as shown can be used to execute the method for performing model selection using a subgraph execution engine provided in the above method embodiment.

[0102] The device 400 may specifically include, for example: an acquiring unit 401 , a first determining unit 402 , an executing unit 403 , and a second determining unit 404 .

[0103] An acquisition unit 401 is used to acquire a model selection strategy corresponding to the task to be inferred, wherein the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field;

[0104] A first determining unit 402 is used to determine a target service required to be executed to obtain the at least one field according to the dependency relationship between services pre-stored in the subgraph execution engine;

[0105] An execution unit 403, configured to execute the target service to obtain a value of the at least one field;

[0106] The second determining unit 404 is used to determine a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

[0107] Optionally, the dependency relationship between services pre-stored in the subgraph execution engine includes:

[0108] At least one directed acyclic graph, each directed acyclic graph in the at least one directed acyclic graph is used to indicate a group of services with dependencies, wherein a node in the directed acyclic graph corresponds to a service, the direction of the edge between the nodes indicates the upstream and downstream relationship between the nodes, and the edge between the nodes also indicates the fields transmitted between the nodes.

[0109] Optionally, the first determining unit 402 is configured to:

[0110] For each field of the at least one field, determine a first service that generates the field, and search for a target directed acyclic graph including the first service;

[0111] Determining the target service includes the first service in the target directed acyclic graph and a second service upstream of the first service.

[0112] Optionally, the acquiring unit 401 is used to:

[0113] The model selection strategy input by the user in the strategy configuration page provided by the strategy configuration platform is obtained.

[0114] Optionally, the second determining unit 404 is configured to:

[0115] Calling a strategy engine to judge the model selection strategy using the value of the at least one field, determining whether a judgment condition corresponding to the at least one field is satisfied, and obtaining a judgment result;

[0116] According to the judgment result, a target model for executing the task to be inferred is determined.

[0117] Optionally, the large model includes: a large language model or a multimodal large language model, and the small model includes other reasoning models except the large language model and the multimodal large language model.

[0118] Since the device 400 is a device corresponding to the method for using a subgraph execution engine to perform model selection provided in the above method embodiment, the specific implementation of each unit of the device 400 is based on the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 400, reference can be made to the relevant description part of the above method embodiment, which will not be repeated here.

[0119] The embodiment of the present application also provides an electronic device, the electronic device comprising a processor and a memory;

[0120] The processor is used to execute instructions stored in the memory, so that the device executes the method for performing model selection using a subgraph execution engine provided in the above method embodiment.

[0121] An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute the method for performing model selection using a subgraph execution engine provided in the above method embodiment.

[0122] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the method for performing model selection using a subgraph execution engine provided in the above method embodiment.

[0123] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0124] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0125] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for model selection using a subgraph execution engine, characterized in that: The method comprises: Obtaining a model selection strategy corresponding to the task to be inferred, wherein the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field; Determining a target service required to be executed to obtain the at least one field according to dependencies between services pre-stored in the subgraph execution engine; executing the target service to obtain a value of the at least one field; A target model for executing the task to be inferred is determined according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

2. The method according to claim 1, characterized in that The dependency relationship between services pre-stored in the subgraph execution engine includes: At least one directed acyclic graph, each directed acyclic graph in the at least one directed acyclic graph is used to indicate a group of services with dependencies, wherein a node in the directed acyclic graph corresponds to a service, the direction of the edge between the nodes indicates the upstream and downstream relationship between the nodes, and the edge between the nodes also indicates the fields transmitted between the nodes.

3. The method according to claim 2, characterized in that The step of determining the target service required to be executed to obtain the at least one field according to the dependency relationship between the services pre-stored in the subgraph execution engine includes: For each field of the at least one field, determine a first service that generates the field, and search for a target directed acyclic graph including the first service; Determining the target service includes the first service in the target directed acyclic graph and a second service upstream of the first service.

4. The method according to claim 1, characterized in that The obtaining of the model selection strategy corresponding to the task to be inferred includes: The model selection strategy input by the user in the strategy configuration page provided by the strategy configuration platform is obtained.

5. The method according to claim 4, characterized in that The step of determining a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy includes: The policy engine is called to judge the model selection policy by using the value of the at least one field, to determine whether a judgment condition corresponding to the at least one field is satisfied, and to obtain a judgment result; According to the judgment result, a target model for executing the task to be inferred is determined.

6. The method according to any one of claims 1 to 5, characterized in that: The large model includes: a large language model or a multimodal large language model, and the small model includes other reasoning models except the large language model and the multimodal large language model.

7. A device for performing model selection using a subgraph execution engine, characterized in that: The device comprises: an acquisition unit, configured to acquire a model selection strategy corresponding to the task to be inferred, wherein the model selection strategy is used to select a model for executing the task to be inferred, and the model selection strategy includes at least one field and a judgment condition corresponding to the at least one field; A first determining unit, configured to determine a target service required to be executed to obtain the at least one field according to dependencies between services pre-stored in the subgraph execution engine; an execution unit, configured to execute the target service to obtain a value of the at least one field; The second determining unit is used to determine a target model for executing the task to be inferred according to the value of the at least one field and the model selection strategy, wherein the target model is a large model or a small model.

8. The device according to claim 7, characterized in that The dependency relationship between services pre-stored in the subgraph execution engine includes: At least one directed acyclic graph, each directed acyclic graph in the at least one directed acyclic graph is used to indicate a group of services with dependencies, wherein a node in the directed acyclic graph corresponds to a service, the direction of the edge between the nodes indicates the upstream and downstream relationship between the nodes, and the edge between the nodes also indicates the fields transmitted between the nodes.

9. An electronic device, characterized in that: The device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory so that the device executes the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct a device to execute the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Task adjustment method applied to task engine, related device and storage medium

    CN113377348A

  • Unmanned aerial vehicle cluster dynamic collaborative reasoning system and method based on large and small model switching

    CN116451576A

  • Class prediction method, device and equipment based on large language model

    CN117390497A

  • Process management method and device, computer equipment and storage medium

    CN117853059A

Cited By

  • Federal reasoning-based data flexible service method and device

    CN121351936A

  • Model routing method and device

    CN121503692A

  • Large language model reasoning method and device and storage medium

    CN121981275A