Enterprise large model selection method and system oriented to scene application

Abstract

The invention discloses a scene application-oriented enterprise large model selection method and system, and the method comprises the steps: obtaining a large model which can be selected by an enterprise according to a commercial permission agreement of the large model; according to existing hardware resources of an enterprise, screening from the large model list to obtain a primary selection model meeting hardware use requirements; according to the reasoning speed required by the use scene, candidate models meeting the parameter range are obtained through screening from the primary selection models; and performing capability evaluation on the candidate model by using the test data of the corresponding scene, and determining an optimal model according to an evaluation result. Through the technical scheme of the invention, the availability of the candidate large model under different hardware conditions is comprehensively evaluated, the resource configuration is optimized, the adaptation efficiency of the large model in actual business is improved, the business operation efficiency is improved, and the market competitiveness is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a method for selecting enterprise large models for scenario applications and a system for selecting enterprise large models for scenario applications. Background Art

[0002] In today's data-driven world, enterprise big models are rapidly becoming a key driver of innovation and competitiveness across industries. Enterprise big models refer to artificial intelligence models trained with large amounts of data and complex algorithms that can process and analyze a variety of complex tasks for specific enterprise operation scenarios. These models are usually based on deep learning and natural language processing technologies and have a high degree of accuracy and predictive power. Enterprise big models have a wide range of applications, including automated customer service systems, personalized recommendation engines, intelligent manufacturing systems, and financial risk management. Enterprise big models are not only at the forefront of current technological development, but also an important tool for enterprises to achieve digital transformation and intelligent operations.

[0003] In the process of building an enterprise big model, it is crucial to select a suitable open source big model as the base big model to further develop application scenarios. Enterprise big model selection refers to selecting the model that best suits the specific needs and strategic goals of the enterprise from the many available artificial intelligence models (for example: LLaMA, Qwen, Baichuan, etc.). This process requires comprehensive consideration of multiple factors, including the performance of the model, applicable scenarios, scalability, performance requirements, computing resources and costs. At the same time, enterprise big model selection is a complex and important process that requires enterprises to comprehensively evaluate various factors and select the big model that best meets their business needs and strategic goals. This can not only improve the operational efficiency and decision-making ability of the enterprise, but also bring continuous competitive advantages to the enterprise.

[0004] Currently, there is no complete implementation procedure and fully functional system implementation for scenario-oriented enterprise big model selection, which means that enterprises often face many challenges and uncertainties when selecting big models that suit their specific business needs. First, due to the lack of standardized selection processes and detailed implementation steps, enterprises often need to rely on their own technical teams and external consulting expertise when evaluating and comparing different models. This not only increases the complexity and time cost of the selection process, but may also lead to subjectivity and inconsistency in selection decisions.

[0005] Secondly, the current lack of a one-stop selection system or tool in the market makes it difficult for companies to systematically evaluate the performance and applicability of models. Some existing tools and platforms usually only cover certain aspects of the selection process, such as hardware adaptation or performance testing, and cannot provide full-process support for model selection. This functional incompleteness forces companies to integrate multiple tools and perform many steps manually during the selection process, further increasing the workload and complexity.

[0006] In addition, enterprises also need to face the ever-changing technology trends and market dynamics during the selection process. With the rapid development of artificial intelligence technology and the continuous expansion of application scenarios, new models and new algorithms emerge in an endless stream, which requires enterprises to continue to pay attention to the forefront of the industry and update selection standards and strategies in a timely manner. However, the lack of systematic and fully functional selection support makes it difficult for enterprises to quickly adapt to these changes, and they may miss the best selection opportunity or choose a model that is not suitable for their long-term development. Summary of the invention

[0007] In response to the above problems, the present invention provides a method and system for selecting enterprise big models for scenario applications. By comprehensively considering factors such as hardware resources and scenario performance, it systematically evaluates and selects the model that best suits its specific needs, thereby optimizing resource allocation, improving the adaptation efficiency of big models in actual business, improving business operation efficiency, and enhancing market competitiveness.

[0008] To achieve the above object, the present invention provides a method for selecting a large enterprise model for scenario applications, comprising:

[0009] Obtain large models for enterprises to choose from according to the commercial license agreement of large models;

[0010] According to the existing hardware resources of the enterprise, a preliminary model that meets the hardware usage requirements is selected from the large model list;

[0011] According to the inference speed required by the usage scenario, a candidate model satisfying the parameter range is screened from the preliminary selected models;

[0012] The candidate models are evaluated for their capabilities using test data from the corresponding scenarios, and the best model is determined based on the evaluation results.

[0013] In the above technical solution, preferably, the process of selecting a preliminary model that meets the hardware usage requirements from a large model list based on the existing hardware resources of the enterprise includes:

[0014] Determine the CPU model, GPU graphics card model and NPU graphics card model of the enterprise's existing hardware;

[0015] Determine the adaptable reasoning framework and training framework based on the hardware resource model owned by the enterprise;

[0016] In the large model list, the types of large models corresponding to the reasoning framework and the training framework are screened to obtain the initially selected models that can be adapted.

[0017] In the above technical solution, preferably, the candidate model satisfying the parameter range is screened from the preliminary selected models according to the inference speed required by the usage scenario, and the specific process includes:

[0018] The inference efficiency that can be supported by hardware testing based on the usage scenario;

[0019] According to the parameter index corresponding to the inference efficiency, a candidate model that can meet the parameter index is screened out from the preliminary selected models.

[0020] In the above technical solution, preferably, the specific process of evaluating the capability of the candidate model using the test data of the corresponding scenario includes:

[0021] Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task;

[0022] For each candidate model, the test data is used to test it, and the corresponding evaluation index is used to evaluate the candidate model;

[0023] According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

[0024] In the above technical solution, preferably, the specific process of determining the best model according to the evaluation results includes:

[0025] The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

[0026] The present invention also proposes a scenario-oriented enterprise big model selection system, which uses the scenario-oriented enterprise big model selection method disclosed in any one of the above technical solutions, including:

[0027] A model list acquisition module is used to acquire large models available for enterprise selection according to the commercial license agreement of the large models;

[0028] A model hardware preliminary selection module is used to select a preliminary model that meets the hardware usage requirements from a large model list according to the existing hardware resources of the enterprise;

[0029] A model scenario candidate module is used to select candidate models that meet the parameter range from the preliminary selected models according to the reasoning speed required by the usage scenario;

[0030] The model capability evaluation module is used to evaluate the capabilities of the candidate models using the test data of the corresponding scenario and determine the best model based on the evaluation results.

[0031] In the above technical solution, preferably, the model hardware preliminary selection module is specifically used for:

[0032] Determine the CPU model, GPU graphics card model and NPU graphics card model of the enterprise's existing hardware;

[0033] Determine the adaptable reasoning framework and training framework based on the hardware resource model owned by the enterprise;

[0034] In the large model list, the types of large models corresponding to the reasoning framework and the training framework are screened to obtain the initially selected models that can be adapted.

[0035] In the above technical solution, preferably, the model scene candidate module is specifically used to:

[0036] The inference efficiency that can be supported by hardware testing based on the usage scenario;

[0037] According to the parameter index corresponding to the inference efficiency, a candidate model that can meet the parameter index is screened out from the preliminary selected models.

[0038] In the above technical solution, preferably, the model capability evaluation module is specifically used to:

[0039] Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task;

[0040] For each candidate model, the test data is used to test it, and the corresponding evaluation index is used to evaluate the candidate model;

[0041] According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

[0042] In the above technical solution, preferably, the model capability evaluation module is also used to:

[0043] The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. Comprehensively consider the hardware availability of the model: This invention comprehensively considers the adaptability of the graphics card, the maximum carrying capacity of the environment, and the inference efficiency to effectively select large models. Through comprehensive hardware environment testing, the availability of candidate large models under different hardware conditions can be comprehensively evaluated, providing enterprises with specific large model selection suggestions and guidance.

[0046] 2. Improve enterprise-level big model evaluation standards: This paper establishes a scenario-oriented big model selection evaluation system, and uses a multi-task combination evaluation method to accurately evaluate the application capabilities of big models in various practical scenarios. This comprehensive evaluation standard helps enterprises better understand and measure the performance of candidate big models in specific scenarios, which helps to improve the selection quality and decision-making accuracy of big models.

[0047] 3. Streamline the selection process of enterprise large models: This invention introduces an end-to-end selection system to comprehensively evaluate the hardware adaptability and scenario application capabilities of large models, making the selection process of large models more streamlined and efficient. This streamlined method significantly improves the efficiency of selection and provides enterprises with a more excellent large model selection solution that meets actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a method for selecting a large enterprise model for scenario applications disclosed in an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the operation logic of a method for selecting a large enterprise model for scenario applications disclosed in an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a specific implementation process of a method for selecting a large enterprise model for scenario applications disclosed in an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the modules of a scenario-oriented enterprise large model selection system disclosed in an embodiment of the present invention.

[0052] In the figure, the corresponding relationship between each component and the reference numeral is as follows:

[0053] 1. Model list acquisition module, 2. Model hardware preliminary selection module, 3. Model scene candidate module, 4. Model capability evaluation module. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0056] like Figure 1 and Figure 2 As shown, a method for selecting a large enterprise model for scenario applications provided by the present invention includes:

[0057] Obtain large models for enterprises to choose from according to the commercial license agreement of large models;

[0058] According to the existing hardware resources of the enterprise, select the preliminary model that meets the hardware usage requirements from the large model list;

[0059] Based on the inference speed required by the usage scenario, select candidate models that meet the parameter range from the preliminary models;

[0060] Use the test data of the corresponding scenario to evaluate the capabilities of the candidate models, and determine the best model based on the evaluation results.

[0061] In this implementation, by comprehensively considering factors such as hardware resources and scenario performance, the model that best suits its specific needs is systematically evaluated and selected, and the availability of candidate big models under different hardware conditions is comprehensively evaluated, thereby optimizing resource allocation and improving the adaptation efficiency of big models in actual business. It provides enterprises with specific big model selection suggestions and guidance, improves business operation efficiency, and enhances market competitiveness.

[0062] Specifically, according to the commercial license agreement of the open source large model, a list of large models available for enterprise use is listed. For example, the LLaMA series is completely open for commercial use, while the Qwen series and InternLM series models need to submit an application to the official and obtain authorization before they can be used commercially. It should be noted that different series of models have different parameter ranges for commercial use. For example, the parameter range that can be applied for the Qwen series is 1.8B to 72B, while the InternLM series includes 7B, 20B and higher versions of the base and Chat version.

[0063] In the implementation process, assuming that for an enterprise large model selection instance for a conversational robot scenario, through commercial authorization application, the candidate large models obtained are shown in the following table, including the 7B and 20B versions of the InternLM series and the 1.8B, 7B, 14B, and 72B versions of the Qwen series. The details are shown in Table 1.

[0064] Table 1 List of large models

[0065]

[0066] In the above implementation, preferably, according to the existing hardware resources of the enterprise, a preliminary model that meets the hardware usage requirements is screened from the large model list, and the specific process includes:

[0067] Determine the CPU model, GPU model, and NPU model of the enterprise's existing hardware, and use this to determine its memory capacity;

[0068] Determine the adaptable inference and training frameworks based on the hardware resource models owned by the enterprise;

[0069] In the big model list, select the big model types corresponding to the inference framework and training framework to obtain the preliminary models that can be adapted.

[0070] In this implementation, the hardware adaptation of large models on different graphics cards and CPU models is considered. For example, for GPU graphics cards released by NVIDIA (such as A40, A100 and V100, etc.), commonly used distributed reasoning frameworks include vLLM, Deepspeed and Ray; while for domestic NPU graphics cards (such as H100, 910a, 910b, etc.), mindformers is used as a typical distributed reasoning framework. In addition, for the case of using CPU for reasoning, the common distributed reasoning framework is llama.cpp. In this implementation, the corresponding reasoning framework is selected according to the specific graphics card model to ensure that the deployment of the large model is compatible with the graphics card adaptation. In particular, when large model parameter training is required, the corresponding training framework should also be considered, such as the training framework based on Huggingface and ColossoalAI, and the supported algorithms include fine-tuning, pre-training and reinforcement learning (such as PPO and DPO), as well as supported parameter optimization modes such as LoRA and full parameter fine-tuning.

[0071] In the implementation process, it is assumed that the graphics card model in the hardware test environment is NVIDIA A40. Based on the open source large model distributed reasoning framework vLLM, the models InternLM and Qwen series in Table 1 above can be adapted, so the preliminary model is still the full series of InternLM and Qwen models.

[0072] In the above implementation, preferably, according to the inference speed required by the usage scenario, a candidate model that meets the parameter range is screened from the preliminary selection model, and the specific process includes:

[0073] The inference efficiency that can be supported by hardware testing based on the usage scenario;

[0074] According to the parameter index corresponding to the inference efficiency, candidate models that can meet the parameter index are screened out from the preliminary models.

[0075] In this implementation, as the number of model parameters increases, the inference efficiency decreases, which is reflected in the inference speed and delay. Therefore, according to the requirements of the actual application scenario for the model inference speed, a suitable number of model parameters is selected from the same series of large models.

[0076] Specifically, in order to screen the amount of parameters that meet the needs of the scenario, you also need to pay attention to the impact of the current video memory capacity on the fastest reasoning efficiency. The hardware parameters involved include the number of graphics cards, the communication bandwidth between graphics cards, and the video memory RAM. First, select the model parameter amount suitable for reasoning deployment based on the video memory of the graphics card, and then test the maximum number of concurrency and reasoning efficiency that the current hardware can support. Commonly used large model reasoning efficiency indicators include the number of words generated per second (token / s), the number of queries supported per second (query / s), and reasoning latency. In the same series of large models, the reasoning efficiency is closely related to the amount of model parameters, so you need to choose a large model parameter version that can meet the needs of the scenario.

[0077] In the implementation process, it is assumed that the graphics card memory in the current hardware environment can only meet the inference deployment requirements of models with less than 32B parameters, so the Qwen-72B model cannot meet the requirements and is removed from the candidate model list. At the same time, the inference efficiency test results are shown in Table 2 below. According to the scene deployment requirements, the candidate model list v0.2 is screened out: including InternLM-7B, Qwen-1.8B and Qwen-7B models.

[0078] Table 2 Inference efficiency test results

[0079]

[0080] In the above implementation, preferably, the specific process of evaluating the capability of the candidate model using the test data of the corresponding scenario includes:

[0081] Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task;

[0082] For each candidate model, test it with test data and use corresponding evaluation indicators to evaluate the candidate model;

[0083] According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

[0084] In this implementation, scenario data is used to evaluate the capabilities of candidate models, thereby ensuring that the selected large model can demonstrate excellent performance and adaptability in practical applications.

[0085] During the implementation process, the performance of the large model is evaluated by constructing scenario-oriented test data, and the model with the best performance is selected as the final selection using quantitative evaluation indicators. Specifically, the scenario capabilities are decomposed into the capabilities of specific tasks, and specific test data and evaluation indicators are selected for each task. For the specific parameter quantity of each candidate large model, quantitative evaluation results are obtained by testing each task separately, and then the evaluation results are weighted averaged according to the preset task weights to obtain the final scenario-oriented evaluation results.

[0086] In the above implementation, preferably, the specific process of determining the best model according to the evaluation results includes:

[0087] The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

[0088] During the implementation, it is assumed that according to the needs of the dialogue robot scenario, the selected evaluation tasks include: multi-round dialogue, semantic understanding, knowledge question and answer, and logical reasoning. The weights of the tasks are 0.35, 0.35, 0.15, and 0.15 respectively. The evaluation index uses ROUGE-L / Accuracy, and a public and easily accessible general data set is used to evaluate the task capabilities. The evaluation results are shown in Table 3 below. According to the final weighted result ranking, the model with the strongest effect is InternLM-7B. Therefore, InternLM-7B was finally selected as the selection of the large model for the dialogue robot scenario.

[0089] Table 3 Scenario test evaluation results

[0090] Model Multi-round dialogue Semantic Understanding Trivia Logical Reasoning Weighted results InternLM-7B 30.5 57.7 58.2 63.7 49.2 Qwen-7B 23.1 45.6 51.3 28.0 35.9 Qwen-1.8B 17.9 43.5 50.3 24.8 32.8

[0091] like Figure 3 As shown, the enterprise big model selection method for scenario application disclosed in the above implementation mainly includes the following two parts of the testing process:

[0092] Hardware testing environment: including hardware devices such as graphics card, CPU, memory, and the operating environment of the reasoning and training framework. In this environment, use the commercial licensed candidate model list v0.1 listed in step S1 to test the adaptability, maximum concurrency, and reasoning efficiency of the large model, and determine the appropriate candidate large model and parameter amount.

[0093] Application scenario test environment: including the inference interface caller of the large model, scenario task test data and evaluation model. By using the candidate model list v0.2 output by the hardware test environment, actual scenario testing is carried out to evaluate the performance of the model in a specific scenario, and the final selection model is selected based on the evaluation results.

[0094] like Figure 4 As shown, the present invention further proposes a large enterprise model selection system for scenario applications, which uses the large enterprise model selection method for scenario applications disclosed in any one of the above embodiments, including:

[0095] Model list acquisition module 1, used to acquire large models available for enterprise selection according to the commercial license agreement of the large models;

[0096] Model hardware preliminary selection module 2 is used to select preliminary models that meet the hardware usage requirements from the large model list based on the company's existing hardware resources;

[0097] The model scenario candidate module 3 is used to select candidate models that meet the parameter range from the preliminary selected models according to the reasoning speed required by the usage scenario;

[0098] The model capability evaluation module 4 is used to evaluate the capabilities of candidate models using test data of corresponding scenarios and determine the best model based on the evaluation results.

[0099] In the above implementation, preferably, the model hardware preliminary selection module 2 is specifically used for:

[0100] Determine the CPU model, GPU graphics card model, and NPU graphics card model of the enterprise's existing hardware;

[0101] Determine the adaptable inference and training frameworks based on the hardware resource models owned by the enterprise;

[0102] In the big model list, select the big model types corresponding to the inference framework and training framework to obtain the preliminary models that can be adapted.

[0103] In the above implementation, preferably, the model scene candidate module 3 is specifically used for:

[0104] The inference efficiency that can be supported by hardware testing based on the usage scenario;

[0105] According to the parameter index corresponding to the inference efficiency, candidate models that can meet the parameter index are screened out from the preliminary models.

[0106] In the above implementation, preferably, the model capability evaluation module 4 is specifically used for:

[0107] Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task;

[0108] For each candidate model, test it with test data and use corresponding evaluation indicators to evaluate the candidate model;

[0109] According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

[0110] In the above implementation, preferably, the model capability evaluation module 4 is also used for:

[0111] The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

[0112] According to the enterprise big model selection system for scenario applications disclosed in the above-mentioned implementation mode, the functions to be implemented by each module thereof correspond to the steps of the enterprise big model selection method for scenario applications disclosed in the above-mentioned implementation mode. During the implementation process, operations are performed in reference to the above-mentioned implementation mode, which will not be repeated here.

[0113] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for selecting a large enterprise model for scenario applications, characterized in that: include: Obtain large models for enterprises to choose from according to the commercial license agreement of large models; According to the existing hardware resources of the enterprise, a preliminary model that meets the hardware usage requirements is selected from the large model list; According to the inference speed required by the usage scenario, a candidate model satisfying the parameter range is screened from the preliminary selected models; The candidate models are evaluated for their capabilities using test data from the corresponding scenarios, and the best model is determined based on the evaluation results.

2. The method for selecting a large enterprise model for scenario application according to claim 1 is characterized in that: According to the existing hardware resources of the enterprise, a preliminary model that meets the hardware usage requirements is screened from the large model list. The specific process includes: Determine the CPU model, GPU graphics card model and NPU graphics card model of the enterprise's existing hardware; Determine the adaptable reasoning framework and training framework based on the hardware resource model owned by the enterprise; In the large model list, the types of large models corresponding to the reasoning framework and the training framework are screened to obtain the initially selected models that can be adapted.

3. The method for selecting enterprise large models for scenario applications according to claim 1 is characterized in that: The process of selecting candidate models that meet the parameter range from the preliminary selected models according to the inference speed required by the usage scenario includes: The inference efficiency that can be supported by hardware testing based on the usage scenario; According to the parameter index corresponding to the inference efficiency, a candidate model that can meet the parameter index is screened out from the preliminary selected models.

4. The method for selecting a large enterprise model for scenario application according to claim 1 is characterized in that: The specific process of evaluating the capability of the candidate model using the test data of the corresponding scenario includes: Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task; For each candidate model, the test data is used to test it, and the corresponding evaluation index is used to evaluate the candidate model; According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

5. The method for selecting a large enterprise model for scenario application according to claim 4 is characterized in that: The specific process of determining the best model according to the evaluation results includes: The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

6. A large enterprise model selection system for scenario applications, characterized in that: The method for selecting a large enterprise model for scenario applications according to any one of claims 1 to 5 comprises: A model list acquisition module is used to acquire large models available for enterprise selection according to the commercial license agreement of the large models; A model hardware preliminary selection module is used to select a preliminary model that meets the hardware usage requirements from a large model list according to the existing hardware resources of the enterprise; A model scenario candidate module is used to select candidate models that meet the parameter range from the preliminary selected models according to the reasoning speed required by the usage scenario; The model capability evaluation module is used to evaluate the capabilities of the candidate models using the test data of the corresponding scenario and determine the best model based on the evaluation results.

7. The enterprise large model selection system for scenario application according to claim 6 is characterized in that: The model hardware preliminary selection module is specifically used for: Determine the CPU model, GPU graphics card model and NPU graphics card model of the enterprise's existing hardware; Determine the adaptable reasoning framework and training framework based on the hardware resource model owned by the enterprise; In the large model list, the types of large models corresponding to the reasoning framework and the training framework are screened to obtain the initially selected models that can be adapted.

8. The enterprise large model selection system for scenario application according to claim 6 is characterized in that: The model scene candidate module is specifically used for: The inference efficiency that can be supported by hardware testing based on the usage scenario; According to the parameter index corresponding to the inference efficiency, a candidate model that can meet the parameter index is screened out from the preliminary selected models.

9. The enterprise large model selection system for scenario application according to claim 6 is characterized in that: The model capability evaluation module is specifically used for: Decompose scenario-oriented capabilities into task-oriented capabilities, and construct corresponding test data and evaluation indicators for each task; For each candidate model, the test data is used to test it, and the corresponding evaluation index is used to evaluate the candidate model; According to the preset task weights, the evaluation results are weighted averaged to obtain the capability evaluation results of each candidate model.

10. The enterprise large model selection system for scenario application according to claim 9 is characterized in that: The model capability assessment module is also used to: The weighted average capability evaluation results of all candidate models are ranked, and the model with the best result is determined as the best model.

Images