Large model fine tuning method and device, electronic equipment and storage medium

Abstract

The invention provides a large model fine tuning method, and relates to the technical field of artificial intelligence, in particular to the technical field of large models and the technical field of deep learning. According to the specific implementation scheme, a to-be-deployed large model is determined from a plurality of large models of different types according to a model application scene; inputting the initial prompt data into the to-be-deployed large model to obtain a first output result; according to at least one of a preset database corresponding to the model application scene and the first output result, adjusting the initial prompt data to obtain adjusted prompt data; according to the training sample data corresponding to the model application scene, finely adjusting the to-be-deployed large model to obtain a target large model; according to the adjusted prompt data and the target large model, a first performance evaluation value of the target large model is determined, and the first performance evaluation value is used for representing the performance of the target large model. The invention further provides a large model fine adjustment device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of large model technology and deep learning technology. More specifically, the present disclosure provides a large model fine-tuning method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application scenarios of large models are increasing. Large models can be pre-trained models with large parameters, which can handle tasks at different semantic granularities and perform complex logical reasoning. Summary of the invention

[0003] The present disclosure provides a large model fine-tuning method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a large model fine-tuning method is provided, the method comprising: determining a large model to be deployed from a plurality of different types of large models according to a model application scenario, wherein the plurality of different types of large models include a conversational large model and a basic large model; inputting initial prompt data into the large model to be deployed to obtain a first output result; adjusting the initial prompt data according to a preset database corresponding to the model application scenario and at least one of the first output results to obtain adjusted prompt data; fine-tuning the large model to be deployed according to training sample data corresponding to the model application scenario to obtain a target large model; determining a first performance evaluation value of the target large model according to the adjusted prompt data and the target large model, wherein the first performance evaluation value is used to characterize the performance of the target large model.

[0005] According to another aspect of the present disclosure, a large model fine-tuning device is provided, and the device includes: a first determination module, used to determine a large model to be deployed from multiple different types of large models according to a model application scenario, wherein the multiple different types of large models include a conversational large model and a basic large model; a first acquisition module, used to input initial prompt data into the large model to be deployed to obtain a first output result; an adjustment module, used to adjust the initial prompt data according to a preset database corresponding to the model application scenario and at least one of the first output results to obtain adjusted prompt data; a fine-tuning module, used to fine-tune the large model to be deployed according to training sample data corresponding to the model application scenario to obtain a target large model; a second determination module, used to determine a first performance evaluation value of the target large model according to the adjusted prompt data and the target large model, wherein the first performance evaluation value is used to characterize the performance of the target large model.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which a large model fine-tuning method and apparatus can be applied according to an embodiment of the present disclosure;

[0012] Figure 2 is a flow chart of a large model fine-tuning method according to an embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of a large model fine-tuning method according to an embodiment of the present disclosure;

[0014] Figure 4 is a block diagram of a large model fine-tuning apparatus according to an embodiment of the present disclosure; and

[0015] Figure 5 is a block diagram of an electronic device to which a large model fine-tuning method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0017] The big model can be a pre-trained model with a large number of parameters, such as the Wenxin big model, the Wenxin one-word model, etc. The big model has powerful natural language processing capabilities and multimodal information processing capabilities, and can cope with tasks at different semantic granularities and perform complex logical reasoning. The big model also has strong transfer learning and few-sample learning capabilities, and can quickly master the processing methods of new tasks and adapt to different fields and different data modes. These characteristics enable the big model to empower thousands of industries and improve industry efficiency. The application scenarios of the big model are wide and diverse, covering multiple fields from search questions and answers, content creation and production to industry-specific applications. Exploring the combination of actual business applications and big models has received widespread attention. Building business applications based on big models and maximizing the accuracy and performance of big models in business is a technical issue worth exploring.

[0018] In order to improve the accuracy of the large model, the performance of the large model itself can be optimized, which will be explained below.

[0019] In some embodiments, the data for training the model can be preprocessed. Data preprocessing can include data cleaning, data enhancement, and data standardization. Data cleaning can remove noise data and outliers to ensure the accuracy and consistency of the training data. Data enhancement refers to increasing the diversity and complexity of the data set through random cropping, rotation, translation, scaling, etc., to improve the robustness and generalization ability of the model. Data standardization can make it easier for the model to learn the characteristics of the data.

[0020] In some embodiments, the hyperparameters of the large model can be adjusted. The hyperparameters may include, for example, the batch size (BatchSize), and adjusting the batch size may balance memory usage efficiency and training stability. The optimal solution for the hyperparameters may also be determined based on at least one of a grid search and a random search. The grid search may find the optimal solution by traversing all combinations in the hyperparameter space. The random search may randomly select a hyperparameter combination for trial to determine the optimal solution. The Bayesian probability model may also be used for hyperparameter search. The probability model may be iteratively updated to obtain the optimal solution for the hyperparameters. Compared with grid search and random search, the Bayesian probability model may utilize limited computing resources more efficiently. Various optimizers may also be used to dynamically adjust the learning rate of the large model to accelerate model convergence and avoid falling into a local optimal solution.

[0021] In some embodiments, the pre-trained model can be fine-tuned, and multiple fine-tuned models can be fused to improve model performance.

[0022] In some embodiments, large models can be efficiently trained based on data parallelism, model parallelism, or gradient accumulation. Based on data parallelism, the data set can be divided into multiple small parts, each of which is processed by a hardware computing node. Different hardware computing nodes exchange gradient information through communication to achieve parallel training. Based on model parallelism, different parts of the model can be assigned to different hardware computing nodes for processing. Model parallelism is suitable for situations where the model scale is very large and a single hardware computing node cannot accommodate the entire model. Based on gradient accumulation technology, gradient information can be accumulated on small batches of data. When a certain amount of data is accumulated, the parameters are updated. Gradient accumulation technology can reduce communication overhead and improve the utilization of computing resources.

[0023] In some embodiments, the large model can be continuously optimized and iterated. For example, the performance of the model can be continuously monitored after deployment, and optimized based on actual feedback. For another example, new data and new features can also be introduced. As new data is continuously generated, the training data is regularly updated and new features are introduced to improve the adaptability and accuracy of the model.

[0024] The performance optimization of a large pre-trained model is a comprehensive process that requires comprehensive consideration of data, models, hyperparameters, training methods, and continuous optimization. Through continuous trial and error, the accuracy and performance of the model can be gradually improved. However, the performance optimization method of the large pre-trained model mentioned above mainly optimizes the accuracy of the pre-trained model itself, and does not involve actual business scenarios and application scenarios.

[0025] Therefore, in order to improve the performance of large models in actual business scenarios and application scenarios, the present disclosure provides a large model fine-tuning method, which will be described below.

[0026] Figure 1 is a schematic diagram of an exemplary system architecture to which a large model fine-tuning method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0029] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0030] It should be noted that the large model fine-tuning method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the large model fine-tuning device provided in the embodiment of the present disclosure can generally be set in the server 105. The large model fine-tuning method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the large model fine-tuning device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0031] It can be understood that the above describes the system architecture of the present disclosure, and the following will describe the method of the present disclosure.

[0032] Figure 2 is a flowchart of a large model fine-tuning method according to an embodiment of the present disclosure.

[0033] like Figure 2 As shown, the method 200 may include operations S210 to S250.

[0034] In operation S210, a large model to be deployed is determined from a plurality of large models of different types according to a model application scenario.

[0035] In the embodiments of the present disclosure, the model application scenario may be a scenario where the model is to be deployed. The model application scenario may include scenarios that require model generalization capabilities and scenarios that communicate with users in natural language. Scenarios that require strong model generalization capabilities may include: automatic content generation, text classification, entity recognition, sentiment analysis, summarization, understanding, translation, and other scenarios. Scenarios that communicate with users in natural language may include: chatbots, virtual assistants, customer support, and other scenarios.

[0036] In the embodiment of the present disclosure, the multiple different types of macro models include a conversational macro model and a base macro model. The conversational (chat) macro model can be a Wenxin Yiyan macro model. The base (base) macro model can be a Wenxin macro model.

[0037] In the embodiments of the present disclosure, for scenarios that require strong model generalization capabilities, the basic large model can be used as the large model to be deployed. For scenarios that require natural language communication with users, the basic large model can be used as the large model to be deployed.

[0038] In operation S220, the initial prompt data is input into the large model to be deployed to obtain a first output result.

[0039] In the embodiment of the present disclosure, the prompt data may include various data such as text, image, audio, etc.

[0040] In operation S230, the initial prompt data is adjusted according to at least one of a preset database corresponding to the model application scenario and the first output result to obtain adjusted prompt data.

[0041] In the disclosed embodiment, the preset database may include a plurality of background knowledge data corresponding to the model application scenario. If the initial prompt data is adjusted according to the preset database, one or more background knowledge data in the preset database may be merged with the initial prompt data to adjust the initial prompt data.

[0042] In the embodiment of the present disclosure, according to the first output result, the initial prompt data may be adjusted based on the prompt engineering technology.

[0043] In operation S240, the large model to be deployed is fine-tuned according to the training sample data corresponding to the model application scenario to obtain a target large model.

[0044] In the disclosed embodiment, based on the training sample data, the large model to be deployed can be fine-tuned in a supervised or unsupervised fine-tuning manner.

[0045] In operation S250 , a first performance evaluation value of the target large model is determined according to the adjusted prompt data and the target large model.

[0046] In the disclosed embodiment, the adjusted prompt data may be input into the target large model to obtain an adjusted output result. According to the adjusted output result, a first performance evaluation value may be obtained. The first performance evaluation value may characterize the performance of the target large model. The first performance evaluation value may include an accuracy value, a response time, etc.

[0047] Through the embodiments of the present disclosure, the big model to be deployed is determined according to the application scenario of the model, and a big model of a more matching type can be determined for the actual application scenario, which helps to realize the combination of the big model with the actual business application. Based on the model application scenario, the prompt data is adjusted, and the big model is fine-tuned, which can efficiently optimize the big model so that the big model can process the data in the relevant application scenarios more efficiently. When the performance evaluation value of the big model is greater than or equal to the preset performance threshold, the big model is deployed to the relevant hardware equipment, which can fully improve the data processing efficiency and production efficiency of the big model in the relevant application scenarios. In this way, the combination of the big model and the actual business application scenario can be efficiently realized, and the performance of the big model in the relevant application scenarios can be improved.

[0048] In some embodiments, the above method may also include: in response to determining that the first performance evaluation value is greater than or equal to a preset performance threshold, deploying the target large model to a hardware device corresponding to the model application scenario.

[0049] In the disclosed embodiment, the model data of the target large model may be written into relevant hardware devices to deploy the target large model.

[0050] It can be understood that the method of the present disclosure is described above, and the method of the present disclosure will be further described below.

[0051] Figure 3 is a schematic diagram of a large model fine-tuning method according to an embodiment of the present disclosure.

[0052] like Figure 3 As shown, different types of big models may include a conversational big model chat30 and a basic big model base30.

[0053] In some embodiments, in some implementations of S210 above, determining a big model to be deployed from a plurality of different types of big models according to a model application scenario may include: determining the type of the big model to be deployed according to the model application scenario. The pre-trained big model may include a basic big model and a conversational big model. The pre-trained big model may be a big language model (LLM), an image big model, an audio big model, a video big model, etc.

[0054] In the embodiments of the present disclosure, taking the large language model as an example, the basic large model can be a standard version of the large model, which has the basic architecture and parameter configuration of the model. During the training process, the basic large model is developed and optimized earlier. The basic large model is trained on a wider range of text data and has the basic ability to handle a variety of natural language tasks, which is suitable for the above-mentioned scenarios that require strong model generalization capabilities.

[0055] In the disclosed embodiment, taking the large language model as an example, the conversational large model can be obtained by performing supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) on the basis of the basic large model. The conversational large model has stronger conversational ability and natural language understanding ability, and is a model optimized for conversational and interactive tasks. During the training process, the conversational large model uses conversation-related data sets and training strategies. After adjusting the relevant parameters, the conversational large model has a strong ability to understand the context and maintain the coherence of the conversation, so that it can provide more natural, fluent and conversational responses, which is more suitable for the above-mentioned scenario of natural language communication with users.

[0056] In some embodiments, in some implementations of S210 above, determining the large model to be deployed from a plurality of different types of large models according to the model application scenario may include: determining the parameter amount of the large model to be deployed according to the hardware resources of the hardware device corresponding to the model application scenario. For example, the parameter amount of the large model may be 7 billion (7B), 13 billion (13B), and 70 billion (70B). The parameter amount of the large model to be deployed may be determined according to the capacity of the storage resources of the hardware device and the number of processing units.

[0057] In some embodiments, in some implementations of S210 above, determining a large model to be deployed from a plurality of different types of large models according to a model application scenario may include: determining the size of a training sample data set of the large model to be deployed according to the model application scenario. The training sample data set includes a plurality of training sample data. For example, the training sample data may include training prompt data and manually annotated data results.

[0058] like Figure 3 As shown, according to the model application scenario, the large model lm30 to be deployed can be determined from the conversational large model chat30 and the basic large model base30.

[0059] In some embodiments, in some implementations of the above operation S220, the initial prompt data prompt30 may be input into the large model to be deployed to obtain a first output result. The initial prompt data prompt30 may not enable the large model to fully understand and perform related tasks. The above operation S230 may be performed to adjust the initial prompt data, for example, the initial prompt data may be adjusted based on prompt engineering and retrieval-augmented generation (RAG), and prompt engineering will be described below.

[0060] In an embodiment of the present disclosure, adjusting the initial prompt data may include: adjusting iteratively N times based on the initial prompt data according to the first output result. Taking the prompt data as a prompt text as an example, the purpose of the prompt project includes: clearly and accurately providing the large model with the tasks to be performed and the concepts related to the tasks, and avoiding providing vague or ambiguous expressions. The adjusted prompt data may include the necessary details of the task so that the large model can fully understand and correctly perform the task. During the N iterations, the prompt data may be modified and tested multiple times to continuously optimize its effect and accuracy. During the test process, it may be determined whether the model correctly understands and performs the task based on the similarity between the output result and the label of the large model. The prompt data may also be adjusted based on the similarity. The adjustment of the prompt data may be performed manually or by the large model. When adjusting using the large model, the large model may be used to identify inaccurate prompt data and adjust the prompt data.

[0061] For example, according to the first output result, the initial prompt data is adjusted to obtain the first level prompt data to be adjusted. The first level prompt data to be adjusted is input into the large model to be deployed to obtain the first level second output result and the first level second performance evaluation value. According to the first level second output result and the label of the initial prompt data, the first level second performance evaluation value can be determined. It can be determined whether the first level second performance evaluation value is greater than or equal to the preset performance threshold. If the first level second performance evaluation value is less than the preset performance threshold, a second adjustment can be made. If the first level second performance evaluation value is greater than or equal to the preset performance threshold, the first level prompt data to be adjusted can be used as the adjusted prompt data.

[0062] For example, when N is greater than 1, in response to determining that the second performance evaluation value of the nth level is less than the preset performance threshold, the nth level prompt data to be adjusted is adjusted for the n+1th time according to the second output result of the nth level, and the n+1th level second output result and the n+1th level second performance evaluation value are obtained. n is an integer greater than or equal to 1 and less than N. Taking n=1 as an example, when the second performance evaluation value of the first level is less than the preset performance threshold, the first level prompt data to be adjusted can be adjusted for the second time to obtain the second level prompt data to be adjusted. The second level prompt data to be adjusted is input into the large model to be deployed, and the second level second output result and the second level second performance evaluation value can be obtained. Next, multiple subsequent adjustments can be made based on the same or similar methods as the first adjustment and the second adjustment until the second performance evaluation value is greater than or equal to the preset performance threshold or the preset number of adjustments (N) is reached. If there is a second performance evaluation value greater than or equal to the preset performance threshold during the N iterations, the prompt data to be adjusted corresponding to the second performance evaluation value can be used as the adjusted prompt data. Through the embodiments of the present disclosure, prompt data is adjusted based on prompt engineering technology, which helps to decompose complex tasks into simple subtasks and provides a benchmark for subsequent processing.

[0063] For example, in response to determining that the second performance evaluation value of the Nth level is less than a preset performance threshold, the Nth level of prompt data to be adjusted can be adjusted using the retrieval enhancement generation technology. The prompt data may require more contextual information. In this case, the Nth level of prompt data to be adjusted can be determined as prompt data to be enhanced. Based on the retrieval enhancement technology, data other than the data entered by the user can be added to the prompt data, so that the model references real, reference-worthy data that is not used in the training process. The retrieval enhancement generation will be explained below.

[0064] In some embodiments, in other implementations of the above operation S230, the initial prompt data is adjusted according to at least one of the preset database and the first output result corresponding to the model application scenario, and the adjusted prompt data is obtained, including: retrieving the adjustment data corresponding to the prompt data to be enhanced from the preset database. The adjustment data is merged with the prompt data to be enhanced to obtain the adjusted prompt data. For example, the preset database may include background knowledge data corresponding to the model application scenario. The entirety of the prompt data to be enhanced or one or more word segments of the prompt data to be enhanced can be used as a query text (query) to retrieve in the preset database to obtain the adjustment data. The adjustment data is spliced ​​with the prompt data to be enhanced, or natural language text is generated based on the adjustment data and the prompt data to be enhanced, to obtain the adjusted prompt data prompt31. Through the embodiments of the present disclosure, the data in the preset database is merged with the prompt data, which can provide new information for the model body, provide richer context data, effectively reduce the model "hallucination", and provide the precision and accuracy of model reasoning.

[0065] It can be understood that the above describes the adjustment methods of the prompt data, and the following will explain some methods of fine-tuning the large model.

[0066] In some embodiments, in some implementations of the above operation S240, fine-tuning the large model to be deployed according to the training sample data corresponding to the model application scenario to obtain the target large model may include: inputting the training sample data into the large model to be deployed multiple times to obtain multiple third output results. Fine-tuning the large model to be deployed so that the similarity between the multiple third output results is greater than or equal to a preset similarity threshold.

[0067] For example, the training sample data comes from the above-mentioned training sample data set. The training sample data set may be related to the model application scenario and may include a large amount of training sample data to avoid overfitting. After the initial data is collected, the format of the data may be converted to obtain the training sample data. The training sample data may include input training data and labels. The input training data may also be referred to as the prompt text used for training. During the fine-tuning process, the same input training data may be input into the large model to be deployed multiple times so that the large model to be deployed generates multiple third output results. The parameters of the large model to be deployed may be fine-tuned so that the similarity between the multiple third output results is greater than or equal to a preset similarity threshold. In addition, in the process of fine-tuning the large model to be deployed, the difference between the third output result and the label of the input training data may be less than or equal to a preset difference threshold. Through the embodiments of the present disclosure, the consistency of the model can be provided, the performance of the model can be effectively improved, the complexity of the prompt data can be reduced, and the task execution mode of the model can be improved, so that the model can achieve a more professional and accurate response in the relevant application scenarios.

[0068] like Figure 3 As shown, after fine-tuning the large model lm30 to be deployed, the target large model lm31 can be obtained. Next, the above operation S250 can be performed to obtain a first performance evaluation value. It can be determined whether the first performance evaluation value is greater than or equal to a preset performance threshold. If the first performance evaluation value is less than the preset performance threshold, data can be re-collected and the large model can be fine-tuned again. If the first performance evaluation value is greater than or equal to the performance threshold, the target large model lm31 can be deployed to the hardware device dev30.

[0069] In some embodiments, the above method may further include: after deploying the target large model to a hardware device corresponding to the model application scenario, in response to receiving the input prompt data, adjusting the input prompt data according to the data format of the adjusted prompt data to obtain the prompt data to be processed. For example, a prompt data template may be determined based on one or more adjusted prompt data. The prompt data template corresponds to the data format of the adjusted prompt data. Based on the prompt data template, the large model may adjust the input prompt data to obtain the prompt data to be processed.

[0070] In some embodiments, the method may further include: inputting the prompt data to be processed into the target large model to obtain an inference result. Through the embodiments of the present disclosure, the inference of the large model in actual application scenarios can be realized.

[0071] It can be understood that the above text adjusts the prompt data in combination with the prompt engineering and the search enhancement generation technology. However, the present disclosure is not limited to this, and the prompt data can also be adjusted based on the prompt engineering and the search enhancement generation. In the case of adjustment based on the prompt engineering, the above-mentioned Nth level prompt data to be adjusted can be used as the adjusted prompt data. The following will explain the way to adjust the prompt data based on the search enhancement generation.

[0072] In some embodiments, adjusting the initial prompt data according to at least one of a preset database and a first output result corresponding to the model application scenario to obtain the adjusted prompt data includes: retrieving the adjustment data corresponding to the initial prompt data from the preset database. Merging the adjustment data with the initial prompt data to obtain the adjusted prompt data. For example, the preset database may include background knowledge data corresponding to the model application scenario. The initial prompt data as a whole or one or more word segments of the initial prompt data may be used as a query text to retrieve in the preset database to obtain the adjustment data. The adjustment data may be spliced ​​with the initial prompt data, or natural language text may be generated based on the adjustment data and the initial prompt data to obtain the adjusted prompt data.

[0073] It should be noted that the above operations S230 and S240 may be performed in parallel, but the embodiments of the present disclosure are not limited thereto, and the two groups of operations may also be performed in other orders, for example, first performing operation S230 and then performing operation S240, or first performing operation S240 and then performing operation S230.

[0074] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be described below.

[0075] Figure 4 It is a block diagram of a large model fine-tuning apparatus according to an embodiment of the present disclosure.

[0076] like Figure 4 As shown, the apparatus 400 may include a first determining module 410 , a first obtaining module 420 , an adjusting module 430 , a fine-tuning module 440 and a second determining module 450 .

[0077] The first determination module 410 is used to determine a large model to be deployed from a plurality of different types of large models according to a model application scenario. The plurality of different types of large models include a conversational large model and a basic large model.

[0078] The first obtaining module 420 is used to input the initial prompt data into the large model to be deployed to obtain a first output result.

[0079] The adjustment module 430 is used to adjust the initial prompt data according to at least one of the preset database corresponding to the model application scenario and the first output result to obtain adjusted prompt data.

[0080] The fine-tuning module 440 is used to fine-tune the large model to be deployed according to the training sample data corresponding to the model application scenario to obtain the target large model.

[0081] The second determination module 450 is used to determine a first performance evaluation value of the target large model according to the adjusted prompt data and the target large model. The first performance evaluation value is used to characterize the performance of the target large model.

[0082] In some embodiments, the above-mentioned device also includes: a deployment module, which is used to deploy the target large model to a hardware device corresponding to the model application scenario in response to determining that the first performance evaluation value is greater than or equal to a preset performance threshold.

[0083] In some embodiments, the first determination module includes at least one of the following: a first determination unit, used to determine the type of the large model to be deployed according to the model application scenario. A second determination unit, used to determine the parameter quantity of the large model to be deployed according to the hardware resources of the hardware device corresponding to the model application scenario. A third determination unit, used to determine the scale of the training sample data set of the large model to be deployed according to the model application scenario. The training sample data set includes multiple training sample data.

[0084] In some embodiments, the adjustment module includes: a first retrieval unit, configured to retrieve adjustment data corresponding to the initial prompt data from a preset database; and a first fusion unit, configured to fuse the adjustment data with the initial prompt data to obtain adjusted prompt data.

[0085] In some embodiments, the adjustment module includes: an adjustment unit, configured to perform N iterative adjustments based on the initial prompt data according to the first output result to obtain prompt data to be enhanced. N is an integer greater than or equal to 1. A second retrieval unit, configured to retrieve adjustment data corresponding to the prompt data to be enhanced from a preset database. A second fusion unit, configured to fuse the adjustment data with the prompt data to be enhanced to obtain adjusted prompt data.

[0086] In some embodiments, the adjustment unit includes: a first adjustment subunit, for adjusting the initial prompt data according to the first output result to obtain the first level prompt data to be adjusted. An acquisition subunit, for inputting the first level prompt data to be adjusted into the large model to be deployed to obtain the first level second output result and the first level second performance evaluation value. A second adjustment subunit, for, in response to determining that the nth level second performance evaluation value is less than a preset performance threshold, adjusting the nth level prompt data to be adjusted for the nth+1th time according to the nth level second output result to obtain the nth level second output result and the nth level second performance evaluation value. n is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1. A determination subunit, for, in response to determining that the Nth level second performance evaluation value is less than the preset performance threshold, determining the Nth level prompt data to be adjusted as the prompt data to be enhanced.

[0087] In some embodiments, the fine-tuning module includes: an obtaining unit, configured to input the training sample data into the large model to be deployed multiple times to obtain multiple third output results. Fine-tune the large model to be deployed so that the similarity between the multiple third output results is greater than or equal to a preset similarity threshold.

[0088] In some embodiments, the method further includes: a second obtaining module for adjusting the input prompt data according to the data format of the adjusted prompt data in response to receiving the input prompt data after the target large model is deployed to the hardware device corresponding to the model application scenario, and obtaining the prompt data to be processed. A third obtaining module for inputting the prompt data to be processed into the target large model to obtain the inference result.

[0089] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0090] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0091] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0092] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0093] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0094] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the large model fine-tuning method. For example, in some embodiments, the large model fine-tuning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the large model fine-tuning method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the large model fine-tuning method in any other appropriate manner (eg, by means of firmware).

[0095] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0097] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0099] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0100] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0101] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0102] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large model fine-tuning method, comprising: Determine a big model to be deployed from a plurality of different types of big models according to a model application scenario, wherein the plurality of different types of big models include a conversational big model and a basic big model; Inputting the initial prompt data into the large model to be deployed to obtain a first output result; adjusting the initial prompt data according to at least one of a preset database corresponding to the model application scenario and the first output result to obtain adjusted prompt data; Fine-tune the large model to be deployed according to the training sample data corresponding to the model application scenario to obtain a target large model; A first performance evaluation value of the target large model is determined according to the adjusted prompt data and the target large model, wherein the first performance evaluation value is used to characterize the performance of the target large model.

2. The method according to claim 1, further comprising: In response to determining that the first performance evaluation value is greater than or equal to a preset performance threshold, the target large model is deployed to a hardware device corresponding to the model application scenario.

3. The method according to claim 1, wherein: Determining the large model to be deployed from a plurality of large models of different types according to the model application scenario includes at least one of the following: Determine the type of the large model to be deployed according to the model application scenario; Determining the parameter quantity of the large model to be deployed according to the hardware resources of the hardware device corresponding to the model application scenario; According to the model application scenario, the scale of the training sample data set of the large model to be deployed is determined, wherein the training sample data set includes a plurality of the training sample data.

4. The method according to claim 1, wherein: The adjusting the initial prompt data according to at least one of the preset database corresponding to the model application scenario and the first output result to obtain the adjusted prompt data comprises: Retrieving adjustment data corresponding to the initial prompt data from the preset database; The adjusted data is merged with the initial prompt data to obtain the adjusted prompt data.

5. The method according to claim 1, wherein: The adjusting the initial prompt data according to at least one of the preset database corresponding to the model application scenario and the first output result to obtain the adjusted prompt data comprises: According to the first output result, performing N iterative adjustments based on the initial prompt data to obtain prompt data to be enhanced, where N is an integer greater than or equal to 1; Retrieving adjustment data corresponding to the prompt data to be enhanced from the preset database; The adjusted data is merged with the prompt data to be enhanced to obtain the adjusted prompt data.

6. The method according to claim 5, wherein: The step of performing N iterative adjustments based on the initial prompt data according to the first output result to obtain prompt data to be enhanced includes: According to the first output result, the initial prompt data is adjusted to obtain the first level prompt data to be adjusted; Inputting the first-level prompt data to be adjusted into the large model to be deployed to obtain a first-level second output result and a first-level second performance evaluation value; In response to determining that the second performance evaluation value of the nth level is less than the preset performance threshold, according to the second output result of the nth level, the nth level prompt data to be adjusted is adjusted for the n+1th time to obtain the n+1th level second output result and the n+1th level second performance evaluation value, where n is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1; In response to determining that the Nth level second performance evaluation value is less than the preset performance threshold, the Nth level prompt data to be adjusted is determined as the prompt data to be enhanced.

7. The method according to claim 1, wherein: The step of fine-tuning the large model to be deployed according to the training sample data corresponding to the model application scenario to obtain the target large model comprises: Inputting the training sample data multiple times into the large model to be deployed to obtain multiple third output results; Fine-tune the large model to be deployed so that the similarity between the plurality of the third output results is greater than or equal to a preset similarity threshold.

8. The method according to claim 1, further comprising: After the target large model is deployed to a hardware device corresponding to the model application scenario, in response to receiving the input prompt data, the input prompt data is adjusted according to a data format of the adjusted prompt data to obtain prompt data to be processed; The prompt data to be processed is input into the target large model to obtain an inference result.

9. A large model fine-tuning device, comprising: A first determination module is used to determine a big model to be deployed from a plurality of different types of big models according to a model application scenario, wherein the plurality of different types of big models include a conversational big model and a basic big model; A first obtaining module, used for inputting initial prompt data into the large model to be deployed to obtain a first output result; An adjustment module, configured to adjust the initial prompt data according to at least one of a preset database corresponding to the model application scenario and the first output result to obtain adjusted prompt data; A fine-tuning module, used to fine-tune the large model to be deployed according to the training sample data corresponding to the model application scenario to obtain a target large model; The second determination module is used to determine a first performance evaluation value of the target large model according to the adjusted prompt data and the target large model, wherein the first performance evaluation value is used to characterize the performance of the target large model.

10. The apparatus according to claim 9, further comprising: A deployment module is used to deploy the target large model to a hardware device corresponding to the model application scenario in response to determining that the first performance evaluation value is greater than or equal to a preset performance threshold.

11. The device according to claim 9, wherein: The first determining module includes at least one of the following: A first determining unit, configured to determine the type of the large model to be deployed according to the model application scenario; A second determining unit, configured to determine the parameter amount of the large model to be deployed according to the hardware resources of the hardware device corresponding to the model application scenario; The third determination unit is used to determine the scale of the training sample data set of the large model to be deployed according to the model application scenario, wherein the training sample data set includes a plurality of the training sample data.

12. The device according to claim 9, wherein: The adjustment module comprises: A first retrieval unit, configured to retrieve adjustment data corresponding to the initial prompt data from the preset database; The first fusion unit is used to fuse the adjusted data with the initial prompt data to obtain the adjusted prompt data.

13. The device according to claim 9, wherein: The adjustment module comprises: an adjusting unit, configured to perform N iterative adjustments based on the initial prompt data according to the first output result, to obtain prompt data to be enhanced, where N is an integer greater than or equal to 1; A second retrieval unit, configured to retrieve adjustment data corresponding to the prompt data to be enhanced from the preset database; The second fusion unit is used to fuse the adjusted data with the prompt data to be enhanced to obtain the adjusted prompt data.

14. The device according to claim 13, wherein: The adjustment unit comprises: A first adjustment subunit, configured to adjust the initial prompt data according to the first output result to obtain first-level prompt data to be adjusted; An obtaining subunit is used to input the first level prompt data to be adjusted into the large model to be deployed to obtain the first level second output result and the first level second performance evaluation value; a second adjustment subunit, configured to, in response to determining that the second performance evaluation value of the nth level is less than a preset performance threshold, perform an n+1th adjustment on the prompt data to be adjusted at the nth level according to the second output result of the nth level, to obtain the second output result of the n+1th level and the second performance evaluation value of the n+1th level, where n is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1; The determining subunit is configured to, in response to determining that the Nth level second performance evaluation value is less than the preset performance threshold, determine the Nth level prompt data to be adjusted as the prompt data to be enhanced.

15. The device according to claim 9, wherein: The fine-tuning module comprises: An obtaining unit, used for inputting the training sample data into the large model to be deployed multiple times to obtain multiple third output results; Fine-tune the large model to be deployed so that the similarity between the plurality of the third output results is greater than or equal to a preset similarity threshold.

16. The apparatus according to claim 9, further comprising: A second obtaining module is used for, after deploying the target large model to a hardware device corresponding to the model application scenario, in response to receiving the input prompt data, adjusting the input prompt data according to a data format of the adjusted prompt data to obtain prompt data to be processed; The third acquisition module is used to input the prompt data to be processed into the target large model to obtain the inference result.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images