Target business model generation and data processing method and device based on large model

Abstract

The invention provides a target business model generation and data processing method and device based on a large model, and relates to the technical field of artificial intelligence, in particular to the technical fields of intelligent office, big data, large models and the like. The target business model generation method based on the large models comprises the steps of performing knowledge distillation on at least two pre-trained large models to obtain a base model of a target scene; each pre-trained large model corresponds to one business type in at least two business types included in the target scene; performing knowledge distillation on the base model to obtain a target business model of a target business type in the at least two business types; the target business model is used for processing the data of the target business type.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of smart office, big data, big models, etc., and in particular to a method and device for generating a target business model and processing data based on a big model. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, large language models (LLM, referred to as large models for short) have been widely used in various scenarios.

[0003] In the enterprise office scenario, how to apply large models efficiently and at low cost is a problem that needs to be solved. Summary of the invention

[0004] The present invention provides a method and device for generating a target business model and processing data based on a large model.

[0005] According to one aspect of the present disclosure, a method for generating a target business model based on a large model is provided, comprising: performing knowledge distillation on at least two pre-trained large models to obtain a base model of a target scenario; each pre-trained large model corresponds to one of at least two business types included in the target scenario; performing knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types; and the target business model is used to process data of the target business type.

[0006] According to another aspect of the present disclosure, a data processing method is provided, comprising: obtaining target data of a target business type; processing the target data using a target business model corresponding to the target business type to obtain a data processing result; wherein the target business model is generated using a method as described in any one of the above aspects.

[0007] According to another aspect of the present disclosure, a target business model generation device based on a large model is provided, including: a first generation module, used to perform knowledge distillation on at least two pre-trained large models to obtain a base model of a target scenario; each pre-trained large model corresponds to one of at least two business types included in the target scenario; a second generation module, used to perform knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type.

[0008] According to another aspect of the present disclosure, a data processing device is provided, including: an acquisition module for acquiring target data of a target business type; a processing module for processing the target data using a target business model corresponding to the target business type to obtain a data processing result; wherein the target business model is generated using a method as described in any one of the above aspects.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method as described in any of the above aspects.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0011] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the methods described in any one of the above aspects.

[0012] According to the embodiments of the present disclosure, data processing can be performed with high efficiency and low cost.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0015] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram of an application scenario for implementing an embodiment of the present disclosure;

[0017] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure;

[0018] Figure 4 is a schematic diagram of the first stage of knowledge distillation provided according to an embodiment of the present disclosure;

[0019] Figure 5is a schematic diagram of the second stage of knowledge distillation provided according to an embodiment of the present disclosure;

[0020] Figure 6 is a schematic diagram according to a third embodiment of the present disclosure;

[0021] Figure 7 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0022] Figure 8 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0023] Fig. 9 It is a schematic diagram of an electronic device used to implement the target business model generation method or data processing method based on a large model in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] In vertical scenarios, the pre-trained large model is usually fine-tuned to obtain a fine-tuned model for the corresponding vertical category, and the fine-tuned model is used for reasoning.

[0026] Enterprise office scenarios are different from vertical scenarios. Vertical scenarios usually only target a single type of data processing, such as code generation. However, enterprise office scenarios face multiple business types, such as customer service conversations, enterprise document processing, and intelligent recommendations. If each business type fine-tunes the pre-trained large model to obtain the corresponding type of fine-tuned model, there will be problems such as high computing resource overhead and complex deployment.

[0027] Therefore, in the enterprise office scenario, it is necessary to solve the problem of how to obtain the target model suitable for the target business type efficiently and at low cost.

[0028] In order to obtain a model suitable for enterprise office scenarios efficiently and at low cost, the present disclosure provides the following embodiments.

[0029] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a method for generating a target business model based on a large model, such as Figure 1 As shown, the method includes:

[0030] 101. Perform knowledge distillation on at least two pre-trained large models to obtain a base model of a target scenario; each pre-trained large model corresponds to one of at least two business types included in the target scenario.

[0031] 102. Perform knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type.

[0032] Among them, the pre-trained large model refers to an existing large model obtained through pre-training.

[0033] In the disclosed embodiment, processing is performed based on multiple (at least two) pre-trained large models.

[0034] These pre-trained large models include, for example, GPT-o, GPT-4, EB4, Qwen3.5, etc. GPT-o and GPT-4 are respectively one of the Generative Pre-trained Transformer (GPT) series models; EB4 is one of the Wenxinyiyan series models; Qwen3.5 is one of the Qwen series models.

[0035] The core idea of ​​knowledge distillation is to use a well-trained complex model (teacher model) to guide the training of a simple model (student model), so that the student model is close to the teacher model in performance, but the number of parameters is greatly reduced.

[0036] In the first stage, the pre-trained large model is used as the teacher model, and the base model of the target scene is used as the student model. Through knowledge distillation, the base model of the target scene is generated from multiple existing pre-trained large models.

[0037] The target scenario is the scenario in which the model is to be applied, and the target scenario involves multiple business types.

[0038] Taking the target scenario of an enterprise office scenario as an example, this scenario involves multiple business types, such as customer service conversations, enterprise document processing, intelligent recommendations, etc.

[0039] If the pre-trained large model is fine-tuned for each business type to obtain a business model of the corresponding business type, there will be problems such as low efficiency and high cost.

[0040] To this end, in this embodiment, a base model shared by different business types can be first obtained based on a pre-trained large model, and then a business model corresponding to each business type can be obtained based on the base model.

[0041] Since the base model is shared by different business types, the base model needs to have a certain degree of generalization to meet the data processing requirements of different business types.

[0042] In order to make the base model generalizable, knowledge distillation is performed on multiple pre-trained large models to learn knowledge from each pre-trained large model and improve the generalization of the base model.

[0043] Multiple pre-trained large models correspond to different business types, which is conducive to the base model learning knowledge of different business types.

[0044] Assuming that the target scenario includes three business types, such as the above-mentioned enterprise office scenario includes the following three business types: customer service dialogue, enterprise document processing, and intelligent recommendation, then three pre-trained large models can be selected to correspond to the above three business types. For example, Qwen3.5 has good dialogue capabilities, so the pre-trained large model Qwen3.5 can be used as the pre-trained large model corresponding to the customer service dialogue business type.

[0045] In this way, the base model can learn different knowledge from the pre-trained large models corresponding to different business types, thereby improving the generalization and accuracy of the base model.

[0046] After obtaining the base model, knowledge distillation can also be used to perform knowledge distillation on the base model of the target scenario to obtain a target business model of the target business type in the target scenario.

[0047] That is, in the second stage, the base model is used as the teacher model and the target business model is used as the student model, and the target business model is generated from the base model through knowledge distillation.

[0048] The target business type is one or more of the multiple business types included in the target scenario.

[0049] For example, samples corresponding to the first business type may be used to perform knowledge distillation on the base model to obtain a first business model, which is used in the reasoning process of the first business type. Specifically, if the first business type is customer service dialogue, a first business model for customer service dialogue is generated, and in the reasoning stage, the first business model is used for customer service dialogue. For another example, samples corresponding to the second business type may be used to perform knowledge distillation on the base model to obtain a second business model, which is used in the reasoning process of the second business type. Specifically, if the second business type is enterprise document processing, a second business model for enterprise document processing is generated, and in the reasoning stage, the second business model is used for enterprise document processing, such as document classification, document summary generation, etc.

[0050] Through the knowledge distillation process, a student model that is smaller in size than the teacher model can be obtained, but the student model can continue the performance of the teacher model, so that a model with better performance and smaller size can be obtained.

[0051] In the model application stage, the target business model can be used for reasoning, such as processing business data of types such as text, images, and application programming interfaces (APIs), to obtain accurate and efficient data processing results.

[0052] In this embodiment, the base model of the target scenario is obtained by performing knowledge distillation on the pre-trained large model, and then the target business model of the target business type is obtained by performing knowledge distillation on the base model. This allows the target business model to be generated at low cost and efficiently. In addition, the base model is obtained by performing knowledge distillation on multiple pre-trained large models, and each pre-trained large model corresponds to a business type. This allows the base model to learn knowledge of different business types, thereby improving the generalization and accuracy of the base model, and thereby improving the accuracy of the target business model.

[0053] In specific implementation, the model generation method and the data processing method based on the generated target business model can be specifically executed by a physical device, which can be a terminal device, a server, a processing unit, etc. Since the target business model is obtained through two stages of knowledge distillation, the target business model has a smaller scale, occupies fewer resources, and due to its small scale, it can also improve data processing efficiency, thus saving storage resources and computing resources of the physical device, improving processing speed, and improving the internal performance of the physical device.

[0054] Taking text data as an example, the following scheme can be used to obtain the target business model applicable to text data:

[0055] Based on the first text data sample, knowledge distillation is performed on at least two pre-trained large models to obtain a base model of a target scenario; each pre-trained large model corresponds to one of at least two business types included in the target scenario; and,

[0056] Based on the second text data sample, knowledge distillation is performed on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type;

[0057] The first text data sample includes: text data samples corresponding to the at least two business types; the second text data sample includes: text data samples corresponding to the target business type.

[0058] Specifically, in the first stage: assuming that the target scenario involves two types of business, namely customer service dialogue and document processing, then a pre-trained large model suitable for customer service dialogue can be selected as the first pre-trained large model, and a large model suitable for document processing can be selected as the second pre-trained large model. Specifically, since Qwen3.5 has good dialogue capabilities, Qwen3.5 is used as the first pre-trained large model, and since EB4 has good document processing capabilities such as document classification, EB4 is used as the second pre-trained large model.

[0059] In addition, samples of various business types may be collected as the first text data samples. For example, existing conversation data samples and document classification samples may be collected to form the first text data samples.

[0060] After determining the first pre-trained large model and the second pre-trained large model and obtaining the first text data sample, the first text data sample is respectively input into the above two pre-trained large models and into the base model to be trained. These models process the first text data, such as extracting features of the first text data, generating or classifying data based on the extracted features, etc., to obtain corresponding output results. For example, the first pre-trained large model obtains a first dialogue prediction result, and the second pre-trained large model obtains a first document processing prediction result. The base model will obtain prediction results corresponding to multiple business types, which are respectively referred to as second dialogue prediction results and second document processing prediction results. These dialogue prediction results and document processing prediction results are all text data. Afterwards, a first loss function can be constructed based on the first dialogue prediction result and the second dialogue prediction result, and a second loss function can be constructed based on the first document processing prediction result and the second document processing prediction result. The sum of the first loss function and the second loss function is used as the loss function of the first stage, and the loss function is used to adjust the parameters of the base model until the preset end condition is reached to obtain the final base model.

[0061] This process involves text data and its processing. Since it involves large pre-trained models corresponding to multiple business types to process text data, and the base model will obtain text prediction results corresponding to multiple business types, a base model that is suitable for processing text data of multiple business types can be obtained, thereby improving the accuracy and generalization of the base model's processing of text data of multiple business types.

[0062] In the second stage: Assuming that the target business type is a customer service conversation, the existing conversation data samples can be collected as the second text data samples, and the second text data samples are respectively input into the base model obtained in the first stage and the target business model corresponding to the customer service conversation to be trained. The base model and the target business model process the second text data, such as extracting text features of the second text data, generating conversations based on the extracted text features, etc., to obtain corresponding output results. For example, the base model obtains the third conversation prediction result, and the target business model obtains the fourth conversation prediction result. These conversation prediction results are all text data. Afterwards, the loss function of the second stage can be constructed based on the third conversation prediction result and the fourth conversation prediction result. The loss function is used to adjust the parameters of the target business model until the preset end condition is reached to obtain the final target business model. When applied, the target business model is used in a customer service conversation.

[0063] In this way, by performing processing such as text data generation on the second text data sample (dialogue data sample), a target business model suitable for customer service dialogue can be obtained, thereby improving the accuracy of the target business model.

[0064] In order to better understand the present disclosure, the application scenarios involved in the present disclosure are described as follows:

[0065] Figure 2 It is a schematic diagram of an application scenario for implementing the embodiment of the present disclosure.

[0066] like Figure 2 As shown, the application scenario includes: a pre-trained large model 201, a base model 202 of the target scenario, and a business model 203 corresponding to each business type.

[0067] Among them, a large model library can be pre-constructed, which stores existing pre-trained large models for selection. When targeting a certain target scenario, multiple corresponding pre-trained large models can be selected according to the various business types included in the target scenario.

[0068] In this embodiment, taking the target scenario as an enterprise office scenario, in the enterprise office scenario, it is assumed that the multiple business types involved include: customer service dialogue, enterprise document processing and intelligent recommendation. For this purpose, a pre-trained large model suitable for the above three business types can be selected from the large model library. It is assumed that the selected pre-trained large models are represented as the first pre-trained large model to the third pre-trained large model, so that the base model of the target scenario can learn knowledge from multiple teacher models (pre-trained large models) to improve the accuracy, stability and generalization of the base model. The business models corresponding to the above business types are respectively called the first business model, the second business model and the third business model.

[0069] The base model is obtained by performing knowledge distillation on the pre-trained large model.

[0070] The business model, in its initial state, is obtained by performing knowledge distillation on the base model. Furthermore, in order to improve the performance (such as accuracy or stability) of the business model, each business model can also perform evolution operations to achieve self-evolution.

[0071] The core of model evolution is to continuously optimize and improve the model to adapt to new needs or environmental changes. This process may involve improvements in model structure and / or parameters. In this embodiment, the business model can specifically use a preset evolutionary algorithm to perform evolution operations.

[0072] In combination with the above application scenarios, the present disclosure also provides the following embodiments.

[0073] Figure 3 is a schematic diagram according to the second embodiment of the present disclosure. This embodiment provides a method for generating a target business model based on a large model, the method comprising:

[0074] 301. Perform knowledge distillation on at least two pre-trained large models to obtain a base model of a target scenario; each pre-trained large model corresponds to one of at least two business types included in the target scenario.

[0075] Specifically, each pre-trained large model can be used to process the first training samples of the at least two business types to obtain a first output result; the base model can be used to process the first training sample to obtain at least two second output results; each second output result corresponds to one of the business types; a first loss function is constructed based on the first output result and the second output result; and based on the first loss function, the model parameters of the base model are adjusted.

[0076] Among them, the knowledge distillation from the pre-trained large model to the base model can be called the first stage of knowledge distillation. Figure 4 It is a schematic diagram of the first stage of knowledge distillation provided according to an embodiment of the present disclosure.

[0077] like Figure 4 As shown, the training samples used in the first stage of knowledge distillation are called first training samples, and the first training samples include data of various business types involved in the target scenario.

[0078] For example, the target scenario is an enterprise office scenario, which involves multiple business types including customer service conversations, enterprise document processing, and intelligent recommendations. In this case, data of these three business types need to be collected as the first training samples.

[0079] Specifically, data of various business types may be collected from the log data of the enterprise, thereby obtaining the first training sample.

[0080] In addition, in order to improve data quality, the collected raw data can be preprocessed and the preprocessed data can be used as training samples. The preprocessing includes, for example, removing sensitive data, removing garbled data or data that does not meet the requirements of the format, removing semantic duplication, and removing data of poor quality (such as sentences that are too short or too long, or data with poor semantic content).

[0081] After obtaining the first training sample, the first training sample is input into each pre-trained large model and the base model to be trained. The output of the pre-trained large model is called the first output result, and the output of the base model is called the second output result.

[0082] Each pre-trained large model corresponds to a business type, so the first output result of each pre-trained large model is one, corresponding to a business type, for example, the first pre-trained large model outputs a conversation result, the second pre-trained large model outputs a document classification result, and so on.

[0083] The base model can process data of multiple business types, and for this purpose, the second output results are multiple, corresponding to each business type. For example, the base model can obtain dialogue results and document classification results, etc. Specifically, the base model can include multiple output layers, each output layer is used to obtain an output result corresponding to a business type.

[0084] like Figure 4 As shown, taking three business types as an example, for this purpose, the pre-trained large models are respectively the first pre-trained large model to the third pre-trained large model, and each pre-trained large model outputs a first output result; the base model outputs three second output results.

[0085] Afterwards, a first loss function is constructed based on the first output result and the second output result. Specifically, the first output result and the second output result corresponding to each business type can be used to construct a loss function for each business type, and then the first loss function is constructed based on the loss function of each business type.

[0086] For example, the first output results of the three pre-trained large models are represented as y11, y12, and y13, and the three second output results of the base model are represented as p21, p22, and p23. Assuming that y11 and p21 correspond to the same business type, such as conversation results, and the rest are similar, loss1 can be constructed based on y11 and p21, loss2 can be constructed based on y12 and p22, and loss3 can be constructed based on y13 and p23. After that, loss1, loss2 and loss3 are directly added or weightedly added to obtain the first loss function.

[0087] After obtaining the first loss function, the model parameters of the base model can be adjusted using the first loss function using algorithms such as back propagation until a preset end condition is reached, and the base model that reaches the end condition is used as the final base model.

[0088] In this embodiment, a first loss function is constructed based on the first output result of each pre-trained large model and the second output result of the corresponding business type output by the base model. The parameters of the base model are adjusted using the first loss function. This allows the base model to accurately learn the knowledge of each pre-trained large model without the need to understand the internal structure of the pre-trained large model, thereby efficiently and accurately generating the base model.

[0089] 302. Perform knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type.

[0090] Assuming that the target scenario includes three business types and the business model of the first business type needs to be obtained, knowledge distillation can be performed on the base model to obtain the first business model.

[0091] Specifically, the base model can be used to process the second training sample to obtain a third output result; the second training sample includes: data of the target business type; the target business model is used to process the second training sample to obtain a fourth output result; a second loss function is constructed based on the third output result and the fourth output result; based on the second loss function, the model parameters of the target business model are adjusted.

[0092] Among them, the knowledge distillation from the base model to the business model can be called the second stage of knowledge distillation. Figure 5 It is a schematic diagram of the second stage of knowledge distillation provided according to an embodiment of the present disclosure.

[0093] like Figure 5 As shown, the training samples used in the second stage of knowledge distillation are called second training samples, and the second training samples include data of the target business type.

[0094] For example, the target scenario is an enterprise office scenario, which involves multiple business types including customer service conversations, enterprise document processing, and intelligent recommendations. Assuming that the target business type is customer service conversations, then it is necessary to collect data on the customer service conversation business type as the second training sample.

[0095] Specifically, data of the required business type may be collected from the log data of the enterprise, thereby obtaining the second training sample.

[0096] In addition, in order to improve data quality, the collected raw data can be preprocessed and the preprocessed data can be used as training samples. The preprocessing includes, for example, removing sensitive data, removing garbled data or data that does not meet the requirements of the format, removing semantic duplication, and removing data of poor quality (such as sentences that are too short or too long, or data with poor semantic content).

[0097] like Figure 5 As shown, after obtaining the second training sample, the second training sample is input into the base model obtained by the first stage of knowledge distillation and the business model to be trained (target business model). The output of the base model is called the third output result, and the output of the target business model is called the fourth output result.

[0098] Afterwards, a second loss function is constructed based on the third output result and the fourth output result. Specifically, the third output result can be specifically selected as the output result corresponding to the base model and the target business type. For example, the base model has three output layers, each of which corresponds to a business type. Assuming that the target business type is the first business type, the output result of the output layer corresponding to the first business type can be selected as the third output result. Each business model corresponds to a business type, so each target business model can output a fourth output result corresponding to the business type. In this way, the third output result and the fourth output result correspond to the same business type, such as both are dialogue results. Therefore, the second loss function corresponding to each target business model can be constructed based on the third output result and the fourth output result, and then the corresponding target business model is adjusted according to the second loss function, such as adjusting the business model of the customer service dialogue.

[0099] The specific adjustment process may include: using the second loss function to adjust the model parameters of the target business model using algorithms such as back propagation until a preset end condition is reached, and using the target business model when the end condition is reached as the final target business model.

[0100] In this embodiment, a second loss function is constructed based on the third output result of the base model and the fourth output result of the target business model. The parameters of the target business model are adjusted using the second loss function. This allows the target business model to accurately learn the knowledge of the base model without having to understand the internal structure of the base model, thereby efficiently and accurately generating the target business model.

[0101] The above process involves the generation of a base model and a target business model. In some cases, the base model and the target business model may also be updated.

[0102] This updating process can be called an evolutionary process.

[0103] Furthermore, for the base model, its evolution process is mainly to re-distill knowledge based on the new pre-trained large model to obtain an updated base model.

[0104] For the business model, its evolution process is mainly to use a preset evolutionary algorithm to perform self-evolution based on the reasoning results (data processing results) of the business model to obtain an updated business model.

[0105] To this end, in some embodiments, the following may also be included:

[0106] 303. In response to determining that at least part of the at least two pre-trained large models are updated, perform knowledge distillation on the updated pre-trained large models to obtain an updated base model; perform knowledge distillation on the updated base model to obtain an updated target business model.

[0107] For example, there are three pre-trained large models, and at least one of them is updated. Then, knowledge distillation can be performed again on these updated pre-trained large models, such as the knowledge distillation in the first stage mentioned above, to obtain an updated base model.

[0108] Since the base model has been updated, the updated base model can be further re-distilled to obtain an updated target business model.

[0109] Furthermore, in the first stage, in order to reduce the amount of computation, the update process can use the same first training sample, so that only the output results of the updated pre-trained large model need to be adjusted, and the other output results remain unchanged, thereby reducing the amount of computation and saving resources.

[0110] For example, refer to Figure 4 , assuming that only the first pre-trained large model is updated, the first training sample can be input into the updated first pre-trained large model to obtain a new first output result of the first pre-trained large model, while the output results of the other pre-trained large models and the benchmark model remain unchanged. The previously calculated results can be directly used without recalculation, thereby reducing the amount of calculation and saving resources.

[0111] In addition, after the base model is updated, the same second training sample can also be used in the second stage, so that the output result of the previously calculated target business model can be directly used, thereby reducing the amount of calculation and saving resources.

[0112] In this embodiment, the base model is updated based on the updated pre-trained large model, and the target business model is updated according to the updated base model, so that the target business model can be updated in time and the accuracy of the target business model can be improved.

[0113] 304. Process the data of the target business type using the target business model to obtain a data processing result; in response to determining that the data processing result satisfies a preset evolution condition, evolve the target business model to obtain an evolved target business model.

[0114] After the target business model is generated, the target business model can be deployed, and then, in the online process, the target business model is used to process data to obtain data processing results. For example, the customer service dialogue process is executed using the customer service dialogue business model to obtain customer service dialogue results.

[0115] After obtaining the data processing result, if the data processing result triggers an evolution operation, the target business model performs a self-evolution operation.

[0116] Specifically, evolution conditions can be preset. For example, when the accuracy of the data processing results is lower than a set threshold, the target business model can perform evolution operations according to a preset evolutionary algorithm. The evolutionary algorithm, for example, includes an update process for model parameters. Based on the evolutionary algorithm, the target business model can be automatically updated (evolved) to obtain the evolved target business model.

[0117] Afterwards, the evolved target business model is used to perform subsequent data processing of the target business type.

[0118] In this embodiment, when the data processing result triggers an evolution operation, the target business model is evolved, so that the target business model can be updated in time and the accuracy of the target business model can be improved.

[0119] Figure 6 is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides a data processing method, including:

[0120] 601. Obtain target data of a target business type.

[0121] 602. Use a target business model corresponding to the target business type to process the target data to obtain a data processing result.

[0122] The target business model is generated by using any of the methods shown in the above embodiments.

[0123] For example, the customer service conversation data can be processed using a customer service conversation model to obtain the customer service conversation results.

[0124] In this embodiment, since the target business model is obtained through knowledge distillation and is small in scale, it can reduce resource usage and improve processing efficiency; since the target business model has the advantage of high accuracy, the accuracy of data processing results can be improved based on this model.

[0125] Figure 7 It is a schematic diagram according to the fourth embodiment of the present disclosure. This embodiment provides a target business model generation device based on a large model. The device 700 includes: a first generation module 701 and a second generation module 702.

[0126] The first generation module 701 is used to perform knowledge distillation on at least two pre-trained large models to obtain a base model of the target scenario; each pre-trained large model corresponds to one business type among at least two business types included in the target scenario; the second generation module 702 is used to perform knowledge distillation on the base model to obtain a target business model of the target business type among the at least two business types; the target business model is used to process data of the target business type.

[0127] In this embodiment, the base model of the target scenario is obtained by performing knowledge distillation on the pre-trained large model, and then the target business model of the target business type is obtained by performing knowledge distillation on the base model. This allows the target business model to be generated at low cost and efficiently. In addition, the base model is obtained by performing knowledge distillation on multiple pre-trained large models, and each pre-trained large model corresponds to a business type. This allows the base model to learn knowledge of different business types, thereby improving the generalization and accuracy of the base model, and thereby improving the accuracy of the target business model.

[0128] In some embodiments, the first generating module 701 is further used to:

[0129] Using each of the pre-trained large models, a first training sample is processed to obtain a first output result; the first training sample includes: data of at least two business types;

[0130] Using the base model, processing the first training sample to obtain at least two second output results; each second output result corresponds to the one business type;

[0131] Constructing a first loss function based on the first output result and the second output result;

[0132] Based on the first loss function, model parameters of the base model are adjusted.

[0133] In this embodiment, a first loss function is constructed based on the first output result of each pre-trained large model and the second output result of the corresponding business type output by the base model. The parameters of the base model are adjusted using the first loss function. This allows the base model to accurately learn the knowledge of each pre-trained large model without the need to understand the internal structure of the pre-trained large model, thereby efficiently and accurately generating the base model.

[0134] In some embodiments, the second generating module 702 is further configured to:

[0135] The base model is used to process a second training sample to obtain a third output result; the second training sample includes: data of the target business type;

[0136] Using the target business model, processing the second training sample to obtain a fourth output result;

[0137] Constructing a second loss function based on the third output result and the fourth output result;

[0138] Based on the second loss function, the model parameters of the target business model are adjusted.

[0139] In this embodiment, a second loss function is constructed based on the third output result of the base model and the fourth output result of the target business model. The parameters of the target business model are adjusted using the second loss function. This allows the target business model to accurately learn the knowledge of the base model without having to understand the internal structure of the base model, thereby efficiently and accurately generating the target business model.

[0140] In some embodiments, the apparatus 700 may further include:

[0141] The first updating module is used to, in response to determining that at least part of the at least two pre-trained large models are updated, perform knowledge distillation on the updated pre-trained large model to obtain an updated base model; and perform knowledge distillation on the updated base model to obtain an updated target business model.

[0142] In this embodiment, the base model is updated based on the updated pre-trained large model, and the target business model is updated according to the updated base model, so that the target business model can be updated in time and the accuracy of the target business model can be improved.

[0143] In some embodiments, the apparatus 700 may further include:

[0144] The second updating module is used to process the data of the target business type using the target business model to obtain a data processing result; in response to determining that the data processing result meets a preset evolution condition, evolve the target business model to obtain an evolved target business model.

[0145] In this embodiment, when the data processing result triggers an evolution operation, the target business model is evolved, so that the target business model can be updated in time and the accuracy of the target business model can be improved.

[0146] Figure 8It is a schematic diagram according to the fifth embodiment of the present disclosure. This embodiment provides a data processing device. The device 800 includes: an acquisition module 801 and a processing module 802.

[0147] The acquisition module 801 is used to acquire target data of a target business type; the processing module 802 is used to process the target data using a target business model corresponding to the target business type to obtain a data processing result.

[0148] The target business model is generated by using any of the methods shown in the above embodiments.

[0149] For example, the customer service conversation data can be processed using a customer service conversation model to obtain the customer service conversation results.

[0150] In this embodiment, since the target business model is obtained through knowledge distillation and is small in scale, it can reduce resource usage and improve processing efficiency; since the target business model has the advantage of high accuracy, the accuracy of data processing results can be improved based on this model.

[0151] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0152] It can be understood that the “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0153] It is understandable that unless there is any special limitation on the sequence of steps in the process, it means that the timing relationship between these steps is not limited.

[0154] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0155] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0156] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0157] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0158] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a model generation method or a data processing method. For example, in some embodiments, the model generation method or the data processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the model generation method or the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the model generation method or the data processing method in any other appropriate manner (eg, by means of firmware).

[0160] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable task processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0162] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0164] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0166] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0167] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a target business model based on a large model, comprising: Perform knowledge distillation on at least two pre-trained large models to obtain a base model for the target scene; Each pre-trained large model corresponds to one of at least two business types included in the target scenario; Knowledge distillation is performed on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type.

2. The method according to claim 1, wherein: The performing knowledge distillation on at least two pre-trained large models to obtain a base model of the target scene includes: Using each of the pre-trained large models, a first training sample is processed to obtain a first output result; the first training sample includes: data of the at least two business types; Using the base model, processing the first training sample to obtain at least two second output results; each second output result corresponds to the one business type; Constructing a first loss function based on the first output result and the second output result; Based on the first loss function, model parameters of the base model are adjusted.

3. The method according to claim 1, wherein: The performing knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types includes: The base model is used to process a second training sample to obtain a third output result; the second training sample includes: data of the target business type; Using the target business model, processing the second training sample to obtain a fourth output result; Constructing a second loss function based on the third output result and the fourth output result; Based on the second loss function, the model parameters of the target business model are adjusted.

4. The method according to claim 1, further comprising: In response to determining that at least part of the at least two pre-trained large models is updated, performing knowledge distillation on the updated pre-trained large models to obtain an updated base model; Knowledge distillation is performed on the updated base model to obtain an updated target business model.

5. The method according to claim 1, further comprising: Processing the data of the target business type using the target business model to obtain a data processing result; In response to determining that the data processing result meets a preset evolution condition, the target business model is evolved to obtain an evolved target business model.

6. A data processing method, comprising: Obtain target data of target business type; Using a target business model corresponding to the target business type, the target data is processed to obtain a data processing result; Wherein, the target business model is generated by using the method according to any one of claims 1-5.

7. A target business model generation device based on a large model, comprising: A first generation module is used to perform knowledge distillation on at least two pre-trained large models to obtain a base model of the target scene; Each pre-trained large model corresponds to one of at least two business types included in the target scenario; The second generation module is used to perform knowledge distillation on the base model to obtain a target business model of a target business type among the at least two business types; the target business model is used to process data of the target business type.

8. The device according to claim 7, wherein: The first generating module is further used for: Using each of the pre-trained large models, processing the first training sample to obtain a first output result; The first training sample includes: data of at least two business types; Using the base model, processing the first training sample to obtain at least two second output results; each second output result corresponds to the one business type; Constructing a first loss function based on the first output result and the second output result; Based on the first loss function, model parameters of the base model are adjusted.

9. The device according to claim 7, wherein: The second generating module is further used for: Using the base model, processing the second training sample to obtain a third output result; The second training sample includes: data of the target service type; Using the target business model, processing the second training sample to obtain a fourth output result; Constructing a second loss function based on the third output result and the fourth output result; Based on the second loss function, the model parameters of the target business model are adjusted.

10. The apparatus according to claim 7, further comprising: The first updating module is used to, in response to determining that at least part of the at least two pre-trained large models are updated, perform knowledge distillation on the updated pre-trained large model to obtain an updated base model; and perform knowledge distillation on the updated base model to obtain an updated target business model.

11. The apparatus according to claim 7, further comprising: A second updating module, used to process the data of the target business type using the target business model to obtain a data processing result; In response to determining that the data processing result meets a preset evolution condition, the target business model is evolved to obtain an evolved target business model.

12. A data processing device, comprising: An acquisition module, used to acquire target data of a target business type; A processing module, used to process the target data using a target business model corresponding to the target business type to obtain a data processing result; Wherein, the target business model is generated by using the method according to any one of claims 1-5.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images