Model training methods and apparatus, data processing methods and apparatus

Abstract

This specification discloses a model training method and apparatus, and a data processing method and apparatus. The training method includes: training a configuration prediction model based on model feature vectors and device feature vectors, capable of outputting configuration numbers for each layer of the model. The processing method includes: using the configuration prediction model to predict the configuration numbers of the model to be loaded and run on the device, thereby predicting the configuration numbers for each layer of the complete model. The device performs loading inference based on the predicted configuration numbers for each layer of the model, and can generate target content based on user input information.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of artificial intelligence technology, and in particular to model training methods and apparatus, and data processing methods and apparatus. Background Technology

[0002] AIGC (Artificial Intelligence Generated Content) refers to content generated by artificial intelligence. AIGC technology can automatically analyze and understand various types of data, such as text, images, audio, and video, and automatically generate mathematical models based on the data, thereby achieving automated modeling.

[0003] Taking image data as an example, existing AIGC technology can generate a large amount of creative and high-quality image content, but its computing resources consumption is tens of times that of traditional artificial intelligence algorithms (e.g., image generation methods based on diffusion models) (e.g., memory consumption is greater than 10G, and it takes more than 1 second to generate an image). Summary of the Invention

[0004] This specification describes a model training method and apparatus, a data processing method and apparatus, through one or more embodiments, which can solve the problem of high computing power consumption during AIGC model runtime while ensuring good quality of generated content.

[0005] Firstly, the embodiments of this specification provide a model training method, including:

[0006] Acquire training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment;

[0007] The training model is loaded and run on the training device to obtain the prediction results, and the number combination of the preferred results in the prediction results is obtained. The number combination contains one or two or more training configuration numbers of the model layer.

[0008] Obtain the model feature vector of the test model and the device feature vector of the test device, input the model feature vector and the device feature vector into the configuration prediction model, and the configuration prediction model outputs the test configuration number of each layer of the model; wherein, the configuration prediction model is a deep learning network model;

[0009] Using the number combination under the preferred result as the supervision signal, the result of the configuration prediction model is converged using the loss function. If it does not converge, the above test process is repeated until it converges; if it converges, the configuration prediction model is trained.

[0010] Secondly, embodiments of this specification provide a model training apparatus, including:

[0011] A data acquisition module is used to acquire training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment;

[0012] The loading module is used to load and run the training model on the training device to obtain prediction results;

[0013] The training module is used to obtain the number combination of the preferred result in the prediction results, and the number combination includes the training configuration number of one, two or more model layers;

[0014] The feature acquisition module is used to acquire the model feature vector of the test model and the device feature vector of the test device;

[0015] The prediction module is configured with a configuration prediction model, which is used to input the model feature vector and the device feature vector into the configuration prediction model. The configuration prediction model outputs the test configuration number of each layer of the model. The configuration prediction model is a deep learning network model.

[0016] The testing module is used to use the number combination under the preferred result as a supervision signal and to use the loss function to judge the convergence of the result of the configuration prediction model. If it fails to converge, the feature acquisition module and the prediction module repeat the above testing process until convergence. If it converges, the configuration prediction model is trained.

[0017] Thirdly, the embodiments of this specification provide a data processing method, including:

[0018] The system acquires device information and operational status information of the running equipment, as well as model information of the running model; wherein, the model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operational status information includes system utilization and storage occupancy rate.

[0019] Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector;

[0020] The device feature vector and model feature vector are input into the configuration prediction model to obtain the configuration number of each layer of the running model; wherein, the configuration prediction model is a deep learning network model.

[0021] Fourthly, embodiments of this specification provide a model training apparatus, including:

[0022] A data acquisition module is used to acquire training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment;

[0023] The loading module is used to load and run the training model on the training device to obtain prediction results;

[0024] The training module is used to obtain the number combination of the preferred result in the prediction results, and the number combination includes the training configuration number of one, two or more model layers;

[0025] The feature acquisition module is used to acquire the model feature vector of the test model and the device feature vector of the test device;

[0026] The prediction module is configured with a configuration prediction model, which is used to input the model feature vector and the device feature vector into the configuration prediction model. The configuration prediction model outputs the test configuration number of each layer of the model. The configuration prediction model is a deep learning network model.

[0027] The testing module is used to use the number combination under the preferred result as a supervision signal and to use the loss function to judge the convergence of the result of the configuration prediction model. If it fails to converge, the feature acquisition module and the prediction module repeat the above testing process until convergence. If it converges, the configuration prediction model is trained.

[0028] Fifthly, embodiments of this specification provide an electronic device, including a processor and a memory;

[0029] The processor is connected to the memory;

[0030] The memory is used to store executable program code;

[0031] The processor runs a program corresponding to the executable program code stored in the memory to perform the method described in any of the above aspects.

[0032] Sixthly, embodiments of this specification provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in one or more embodiments of this specification.

[0033] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0034] In one or more embodiments of this specification, a configuration prediction model is trained based on model feature vectors and device feature vectors, capable of outputting configuration numbers for each layer of the model. By using the configuration prediction model to predict the configuration numbers of the model to be loaded and run on the device, the configuration numbers for each layer of the complete model can be predicted. The device performs loading inference based on the predicted configuration numbers for each layer of the model, generating target content based on user input information. When the number of models to be loaded exceeds the device's loading capacity, the model is lightweighted before obtaining the configuration numbers for each layer through the configuration prediction model to improve computational efficiency. The embodiments of this specification can reduce the computational consumption of overall model loading and inference for the terminal, improve the computational efficiency of terminal inference and target content generation, and ensure good quality of generated target content. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a global schematic diagram illustrating an application scenario of a data processing method provided in an embodiment of this specification.

[0037] Figure 2 This is a user-oriented interactive diagram illustrating an application scenario of a data processing method provided in the embodiments of this specification.

[0038] Figure 3 A schematic diagram illustrating the deployment scenario of the model provided in the embodiments of this specification;

[0039] Figure 4 A flowchart illustrating a data processing method provided in an embodiment of this specification;

[0040] Figure 5 A flowchart illustrating yet another data processing method provided in the embodiments of this specification;

[0041] Figure 6 A flowchart illustrating yet another data processing method provided in the embodiments of this specification;

[0042] Figure 7 A flowchart illustrating yet another data processing method provided in the embodiments of this specification;

[0043] Figure 8 A schematic flowchart illustrating a model training method provided in an embodiment of this specification;

[0044] Figure 9 This is a schematic diagram of a model lightweighting process provided in an embodiment of this specification;

[0045] Figure 10 A schematic diagram of the distillation architecture during the model lightweighting process provided in the embodiments of this specification;

[0046] Figure 11 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this specification;

[0047] Figure 12 This is a schematic diagram of the structure of another data processing device provided in the embodiments of this specification;

[0048] Figure 13 This is a schematic diagram of the structure of another data processing device provided in the embodiments of this specification;

[0049] Figure 14 This is a schematic diagram of the structure of another data processing device provided in the embodiments of this specification;

[0050] Figure 15 This is a schematic diagram illustrating the process of model training in offline mode and data processing in online mode in an embodiment of this specification;

[0051] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0052] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0053] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0054] AIGC is a new artificial intelligence technology that uses AI models to automatically generate various types of text, images, audio, video, and other content based on given conditions such as topics, keywords, formats, and styles (inputted to the model in the form of AI prompts "Prompt").

[0055] Figure 1The diagram illustrates an AIGC content generation application scenario. The AIGC model in the diagram can be loaded onto an AIGC platform or APP application on terminal devices (such as smartphones, tablets, desktops, laptops, ultra-mobile personal computers (UMPCs), handheld computers, PC devices, personal digital assistants (PDAs), virtual reality devices, etc.) for user use.

[0056] Users can choose according to their needs. Figure 2 Enter the AI ​​prompt word on the interface shown. The AIGC platform or APP application will intelligently process the received input information and generate content corresponding to the AI ​​prompt word. Figure 2 Taking image data as an example, when a user enters the text "sunset at the seaside" on the input interface, the AIGC platform or APP application outputs the image information of "sunset at the seaside" on the interface based on the AI ​​prompt. Figure 2 The purpose is to illustrate the interaction process in application scenarios, and it is not limited to image data. That is, the generated content can be one or more types of data such as text, images, audio, and video (i.e., the target content mentioned in the embodiments of this specification).

[0057] See Figure 3 , Figure 3 The diagram illustrates a scenario for deploying an AIGC model. This scenario includes a server 100 and terminal devices 200. The server 100 and the terminal devices 200 are communicatively linked. Multiple terminal devices 200 can be connected to the server 100. The server can store AIGC models, which can be single models or a group of models.

[0058] The server 100, on the one hand, can respond to instructions from the terminal device 200 and load the model onto the terminal device. For example, based on the user's actual operation, it is determined that a model needs to be retrieved from the server. If an AI prompt word is received, it is determined that a model needs to be retrieved from the server 100 and loaded onto the terminal device 200.

[0059] The server 100, on the other hand, can be used for model training. For example, the server 100 is equipped with a training platform that can train an initial model using a training set and a test set to obtain a target model. The trained model can be stored on the server or locally using a software development kit (SDK).

[0060] AIGC's capabilities are provided by machine learning models, which are large models pre-trained on massive amounts of data and are often referred to as foundation models. With the improvement and optimization of AI models, and the enrichment and refinement of data resources, AIGC can generate higher-quality, more diverse, and more personalized content to meet various user needs and scenarios. This results in its computational resource consumption being dozens of times that of traditional AI algorithms.

[0061] To address this issue, existing technologies propose two schemes to optimize the computational efficiency of AIGC models. Scheme 1 involves optimizing the AIGC model using third-party acceleration libraries (such as xformer, deepspeed, and other general-purpose deep learning model acceleration libraries) before performing inference on the device. This scheme has low adaptation costs, often requiring only minor code modifications. However, its drawbacks are also significant: the efficiency improvement is not substantial, typically ranging from 20% to 50%, failing to adequately meet the requirements for running on terminal devices. Scheme 2 involves pruning the original model into smaller models, then using the unpruned model to distill the pruned model, improving the performance of the smaller models. The advantage of this scheme is that the entire model can be deployed and run on the terminal. The disadvantage is that while the pruned smaller models can meet the requirements of terminal devices, the quality of generated AIGC content drops significantly, falling far short of the quality generated using the unpruned larger model.

[0062] Based on this, this specification provides one or more embodiments to solve the technical problem of high computational resource consumption while ensuring good AIGC content generation quality.

[0063] Please see Figure 4 , Figure 4 A flowchart illustrating a data processing method provided in an embodiment of this specification is shown.

[0064] like Figure 4 As shown, the data processing method may specifically include the following steps:

[0065] Step 102: Obtain the device information and operating status information of the running equipment, as well as the model information of the running model;

[0066] The model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

[0067] Step 104: Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector;

[0068] Step 106: Input the device feature vector and model feature vector into the configuration prediction model to obtain the configuration number of each layer of the running model;

[0069] The configuration prediction model is a deep learning network model.

[0070] The data processing method described in this specification utilizes a configuration prediction model to output the configuration number of each layer of the running model based on the input "information about the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This method can generate high-quality target content with relatively low GPU / RAM consumption and controllable time costs.

[0071] The data processing method described in this specification refers to a running model that meets the loading capacity of the running device. Loading capacity (also applicable to other embodiments of this specification) refers to the performance and resource consumption capacity of the running device. The running device can complete model loading within a certain time, for example, with short loading time, without interfering with the normal operation of other functions of the running device; it also does not occupy excessive memory / video memory, which is often affected by the size of the running model. In other words, the data processing method described in this specification can directly perform sub-model division for running models that meet the loading capacity of the running device.

[0072] The following examples and embodiments illustrate... Figure 4 Each step in the process will be explained separately.

[0073] In step 102, the running device is a terminal device used to load the AIGC model and generate AIGC content; the running model is an AIGC model, which is an AIGC model obtained from the server based on the user's actual operation. The running model can be a single model or a group of models.

[0074] The device information includes the device model and storage capacity. For example, the device's CPU model and GPU model, and correspondingly, the storage capacity is RAM and video memory. The operating status information includes system utilization and storage occupancy. For example, the device's CPU utilization, GPU utilization, and RAM and video memory occupancy.

[0075] The model information includes layer type, layer parameters, etc. Each model has a layer structure with a predetermined configuration number (i.e., layer number). Different models have different numbers of layers, and model layers can be of different types (such as convolutional layers, linear layers, BN layers, etc.). Each model layer has its corresponding layer parameters (such as model weights, etc.).

[0076] The device information and the operating status information are obtained by the running device acquiring its own data. The model information is obtained by the server. When the above data processing method is executed on the running device, when the running device obtains the operating model from the server, it can simultaneously obtain the model information of that operating model. The running device also reads its own data to obtain device information and the current operating status information of the device. When the above data processing method is executed on the server, when the running device obtains the operating model from the server, the running device reads its own data to obtain device information and the current operating status information of the device and sends them to the server. The server obtains the user's actual operation instructions through the running device, can determine the operating model to be loaded, and then determine the model information of the operating model.

[0077] In step 104, the feature encoder is used to perform feature recognition on the input information to obtain a feature vector. The device feature encoder and the model feature encoder are pre-trained before executing the data method of the embodiments of this specification. Based on the device dataset (containing device information and operating status information) and the model dataset (containing parameter information), their respective feature encoders are trained to obtain feature vectors that are closest to the original data. After training, they are applied in the embodiments of this specification.

[0078] In step 106, the configuration prediction model can output the configuration number of each layer of the running model based on the input of the device feature vector and the model feature vector. In the embodiments of this specification, when executing step 106, the trained configuration prediction model can be obtained from the server, and step 106 can be executed using the configuration prediction model on the server; alternatively, the trained configuration prediction model can be obtained from the server and sent to the terminal device, where the terminal device executes step 106 using the configuration prediction model.

[0079] After obtaining the configuration number of each layer of the running model in step S106, it can be used to perform specific analysis on each layer of the running model structure. Alternatively, the terminal device can perform sequential stacking and inference according to the configuration number to generate target content. It can also be used in other aspects, which can reduce the impact of excessive model size on the execution efficiency of terminal devices or servers.

[0080] Please see Figure 5 , Figure 5 A flowchart illustrating another data processing method provided in an embodiment of this specification is shown.

[0081] Step 202: Obtain the device information and operating status information of the running equipment, as well as the model information of the running model;

[0082] The model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

[0083] Step 204: Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector;

[0084] Step 206: Input the device feature vector and model feature vector into the configuration prediction model to obtain the configuration number of each layer of the running model;

[0085] The configuration prediction model is a deep learning network model.

[0086] Step 208: Based on the obtained configuration number, load the layers of the running model into the running device in sequence, and perform inference based on the user input information to generate the target content corresponding to the user input information on the running device.

[0087] The data processing method described in this specification utilizes a configuration prediction model to output the configuration number of each layer of the running model based on the input "information about the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This method can generate high-quality target content with relatively low GPU / RAM consumption and controllable time costs.

[0088] The data processing method described in this specification refers to an operating model that conforms to the loading capacity of the operating device. This allows for direct sub-model division of the operating model.

[0089] Figure 5 The illustrated embodiment is compared to Figure 4 The illustrated embodiment differs in that, after obtaining the configuration number for each layer of the running model, the running device loads and infers based on the configuration number to generate the target content. Taking image data as an example, the running device generates the target content in image form based on AI prompts, model information, device information, and running status information.

[0090] The following examples and embodiments illustrate this. Figure 5 Each step in the process will be explained separately. For detailed explanations of steps 202 to 206, please refer to [link / reference needed]. Figure 4 Steps 102 to 106 in the process.

[0091] In step 208, the model loading and inference process is implemented through stacked loading and inference.

[0092] Based on the configuration number of the model layer, the model layers are loaded into the running device sequentially. Specifically, first, the model layer with configuration number 1 is loaded into the running device, and then the AI ​​prompt is input into the already loaded part of the model in the running device to obtain the output of configuration number 1; then the model layer with configuration number 2 is loaded into the running device (the model with configuration number 1 is removed from the running device), and the output of configuration number 1 is input into the already loaded part of the model in the running device to obtain the output of configuration number 2; and so on, until all model layers have been sequentially traversed, and the inferred target content is output.

[0093] In the two examples above, when model layers have the same configuration number, the model layers with the same configuration number are loaded only once. For example, if there are 3 model layers with configuration number 1, only one needs to be loaded, and the rest can be directly shared and reused (including reusing the weight parameters of the model layers).

[0094] Please see Figure 6 , Figure 6 A flowchart illustrating another data processing method provided in an embodiment of this specification is shown.

[0095] Step 300: Lightweight processing is performed on the running model to obtain a lightweight model that meets the loading capacity of the running device;

[0096] Step 302: Obtain the device information and operating status information of the running equipment, as well as the model information of the running model;

[0097] The model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

[0098] Step 304: Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector;

[0099] Step 306: Input the device feature vector and model feature vector into the configuration prediction model to obtain the configuration number of each layer of the running model;

[0100] The configuration prediction model is a deep learning network model.

[0101] Compared to Figure 4 The embodiments shown in this specification, Figure 6The embodiments shown in this specification require lightweighting of the running model before acquiring relevant information about the running device and the running model. Often, the running model requiring lightweighting is one that exceeds the loading capacity of the running device. For example, if the model is a single model or a group of models, its size is very large, exceeding the running device's operating conditions, or the running device can barely load the model, but the loading speed is very slow, even affecting the implementation of other functions on the running device. Therefore, lightweighting of the running model is necessary. This speeds up the process of dividing the model into sub-models while minimizing computational resource consumption, thus optimizing the entire subsequent data processing flow from the start of the data processing method.

[0102] After lightweighting, the data processing method in this embodiment can utilize a configuration prediction model to output the configuration number of each layer of the running model based on the input "information about the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This method can generate high-quality target content with relatively low GPU / RAM consumption and controllable time costs.

[0103] The following examples and embodiments illustrate this. Figure 6 Each step in the process will be explained separately. For detailed explanations of steps 302 to 306, please refer to [link / reference needed]. Figure 4 Steps 102 to 106 in the process.

[0104] In step 300, model lightweighting refers to processing the original model to reduce the number of parameters, computational cost, and memory usage. Model lightweighting can be implemented in various ways, such as matrix quantization, Kronecker inner product, Huffman coding, and model pruning, to quantize the parameters in the original model and achieve model compression. The original model is an AIGC model obtained from the server, determined based on the user's actual operations.

[0105] Step 300 further includes: determining whether the running model meets the lightweight processing conditions. If it does, lightweight processing is performed; otherwise, step 302 is executed. The lightweight processing conditions can be determined based on the size of the running model and the time taken for the running device to load the running model. Lightweight processing is performed when either the size condition, the time condition, or both are met. Step 302 is executed when none of these conditions are met. The size and time thresholds for the size and time conditions can be preset as needed. Different size and time thresholds can be set for different running devices.

[0106] In one or more embodiments of this specification, the lightweight model is obtained through distillation. Model distillation is the process of transferring the generalization ability "knowledge" of a trained complex model to a simpler network, or learning the "knowledge" of a complex model through a simpler network.

[0107] The distillation process typically begins by training a high-performing Teacher model using a dataset. Then, a lighter Student model is selected, which is trained using both the dataset and the "knowledge" transferred from the Teacher model.

[0108] In one example, a single-step compressive distillation can be performed. First, the original model is pruned to obtain a lightweight model that meets the requirements of the terminal device. Then, the unpruned model is used as the Teacher model, and the pruned model, which is now the Student model, is distilled. Specifically, user input information and random noise are input into the Teacher model, and the output is the teacher's target content corresponding to the user input information. User input information and random noise are input into the Student model, and the output is the student's target content corresponding to the user input information. Using the teacher's target content as a supervision signal, a loss function is used to determine the convergence of the student's target content. When the distance between the Student model's output and the Teacher model's output reaches the convergence condition, training is complete, and the trained Student model is output as the lightweight model.

[0109] In another example, progressive distillation can be performed. For example... Figure 9 The process of lightweighting the operating model to obtain a lightweight model that meets the loading capabilities of the operating device includes:

[0110] Step 301: Use the user input information and random noise input as the operating model of the teacher network, and output the teacher's target content corresponding to the user input information;

[0111] Step 303: Input the teacher's target content into the teacher selection model and output the optimal teacher's target content based on the user's input information;

[0112] Step 305: Use the user input information and random noise input as a lightweight model of the student network, and output the student target content corresponding to the user input information.

[0113] Step 307: Using the selected teacher target content as the supervision signal, the result of the lightweight model is judged by the loss function. If it does not converge, the above process is repeated until it converges. If it converges, the first round of distillation ends and the converged lightweight model is obtained. Steps 301 to 305 are repeated to perform a new round of distillation.

[0114] When a new round of distillation is performed, the lightweight model used in the new round of distillation is based on the converged lightweight model obtained in the previous round of distillation, and is obtained by reducing the number of channels; the teacher network used in the new round of distillation is the network of the converged lightweight model obtained in the previous round of distillation within a certain number of steps.

[0115] Step 309: After multiple rounds of distillation, when the lightweight model meets the loading capacity of the operating equipment, stop distillation and output the distilled lightweight model.

[0116] The above process is based on Figure 10 The distillation architecture shown in the diagram is implemented as follows. In the diagram, the teacher network transmits knowledge to the teacher selection model, and the teacher selection model transmits knowledge to the student network.

[0117] In the above process, the teacher network in the first round of distillation is the AIGC model obtained from the server, determined by the user's actual operation. This teacher network is generally an open-source diffusion model. The teacher selection model, determined based on this teacher network, is a multilayer perceptron (MLP). The teacher selection model selects the preferred output from the teacher network as supervision information for the student network. The lightweight model can be obtained by adjusting the teacher network based on experience, for example, by reducing the number of channels by half; it can also be obtained based on the teacher network using pruning techniques.

[0118] After the first round of distillation, a converged lightweight model is obtained. This lightweight model is then processed by reducing the number of channels for use in the next round of distillation. For example, the number of channels can be reduced by 1 / 2, 1 / 3, or 1 / 4, etc., thus reducing the overall computational cost of the model as the number of channels in each layer decreases. Correspondingly, the teacher network is also adjusted, using the network from the converged lightweight model obtained in the previous round of distillation within a certain number of steps as the teacher network. Generally, the number of steps K is greater than or equal to 2; for example, choosing K as 3 means using the networks from the three most recent processes as the teacher network.

[0119] After multiple rounds of cyclic distillation, when the lightweight model meets the loading capacity of the operating equipment, i.e., satisfies the performance and resource consumption requirements, the lightweight model processing is complete, and the distilled lightweight model is output. Specifically, the operating equipment loading capacity refers to the size of the operating model and / or the time taken for the operating equipment to load the operating model. When it is determined that the lightweight model meets the volume condition, or the time condition, or both the volume condition and the time condition, the progressive distillation is complete.

[0120] In each round of distillation, the loss function is used to determine model convergence. For example, the distance between the output of the teacher-selected model and the output of the lightweight model is calculated, which can be Euclidean distance, Mahalanobis distance, cosine distance, etc. Convergence is achieved when the two distances are closest.

[0121] Please see Figure 7 , Figure 7 A flowchart illustrating another data processing method provided in an embodiment of this specification is shown.

[0122] Step 400: Lightweight processing is performed on the running model to obtain a lightweight model that meets the loading capacity of the running device;

[0123] Step 402: Obtain the device information and operating status information of the running equipment, as well as the model information of the running model;

[0124] The model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

[0125] Step 404: Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector;

[0126] Step 406: Input the device feature vector and model feature vector into the configuration prediction model to obtain the configuration number of each layer of the running model;

[0127] The configuration prediction model is a deep learning network model.

[0128] Step 408: Based on the obtained configuration number, load the layers of the running model into the running device in sequence, and perform inference based on the user input information to generate the target content corresponding to the user input information on the running device.

[0129] Compared to Figure 6 In the embodiments shown in this specification, after obtaining the configuration number of each layer of the running model, the running device loads and infers based on the configuration number to generate target content. Taking image data as an example, the running device generates target content in image form based on AI prompts, model information, device information, and running status information.

[0130] The following examples and embodiments illustrate this. Figure 7 Each step in the process will be explained separately. For detailed explanations of steps 400 to 406, please refer to [link / reference needed]. Figure 6 Steps 300 to 306 in the process.

[0131] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0132] In one or more embodiments of this specification, the configuration prediction model needs to be pre-trained. Figure 8 A flowchart illustrating the training method for the configured prediction model is shown.

[0133] The model training method in the embodiments of this specification includes:

[0134] Step 500: Obtain training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment;

[0135] Step 502: Load and run the training model on the training device to obtain the prediction results, and obtain the number combination of the preferred results in the prediction results. The number combination includes one or two or more training configuration numbers of the model layers.

[0136] Step 504: Obtain the model feature vector of the test model and the device feature vector of the test device; input the model feature vector and the device feature vector into the configuration prediction model; the configuration prediction model outputs the test configuration number of each layer of the model; wherein, the configuration prediction model is a deep learning network model.

[0137] Step 506: Using the number combination under the preferred result as the supervision signal, the result of the configuration prediction model is converged using the loss function. If convergence is not achieved, the above test process is repeated until convergence is achieved. If convergence is achieved, the configuration prediction model is trained.

[0138] The method described in this specification is for training a configuration prediction model that can partition the model structure based on how the running model input to the configuration prediction model performs stacked inference on the terminal device. Specifically, it can predict the stacking sequence number of each layer of the running model (i.e., the configuration number as referred to in this specification). After obtaining the configuration number, the terminal device performs loading inference sequentially according to the order of the configuration numbers. For example, it first loads several layers with configuration number 1, then loads several layers with configuration number 2, and so on, until the target content is finally generated. Inference is performed on one layer with a configuration number at a time.

[0139] The training process will be explained in detail below.

[0140] In step 500, sample data is acquired, including several models and several devices for loading and running the models. Depending on the training objective, the sample data is divided into training data and test data in a certain ratio, such as 7:3 or 8:2. The models are models for different purposes, such as text generation models, image generation models, speech generation models, video generation models, etc. The devices have the capability to load the aforementioned models. The devices are selected from terminal devices used in various application scenarios.

[0141] In step 502, prediction is performed using training data. The step of loading and running the training model on the training device to obtain prediction results, and acquiring the preferred combination of numbers from the prediction results, includes:

[0142] Step 5021: Generate multiple sets of number combinations based on the training configuration number of the training model;

[0143] Different layers of different trained models have corresponding training configuration numbers. Based on this, permutations and combinations are used to generate all possible combinations of numbers.

[0144] Step 5023: Each training device loads multiple training models according to the number combination to obtain multiple prediction results;

[0145] Before loading, different configurations are set for each training device, allowing each device to run under different configurations. These configurations include memory and video memory, CPU utilization, GPU utilization, and memory and video memory usage.

[0146] The training device loads the training model according to the number combination to obtain the prediction results (including the predicted value and prediction efficiency).

[0147] The process of loading and running the training model on the training device can be achieved by loading and running it on the actual device, or by simulating it on a simulation platform.

[0148] Step 5025: When the prediction result meets certain conditions, the prediction result is the preferred result;

[0149] The prediction result meeting certain conditions refers to the predicted value deviating from the actual value within a certain range, and the prediction efficiency being at its highest. If both of these conditions are met, the prediction result is considered the preferred result.

[0150] Step 5027: Obtain the number combination under the optimal result. Use this number combination under the optimal result as a supervision signal for subsequent supervised training.

[0151] In step 504, obtaining the model feature vector of the test model and the device feature vector of the test device includes: inputting the model information of each layer of the test model into the model feature encoder, and the model feature encoder outputting the model feature vector; wherein, the model information includes layer type and layer parameters; inputting the device information and operating status information of the test device into the device feature encoder, and the device feature encoder outputting the device feature vector; wherein, the device information includes device model and storage capacity; the operating status information includes system utilization and storage occupancy rate.

[0152] The device feature encoder and the model feature encoder are pre-trained before executing the data method of the embodiments of this specification. Based on the device dataset (containing device information and operating status information) and the model dataset (containing parameter information), their respective feature encoders are trained to obtain feature vectors that are closest to the original data. After training, they are applied in the embodiments of this specification.

[0153] Next, the obtained device feature vectors and model feature vectors are input into the configuration prediction model, which outputs the test configuration number for each layer of the model. In step 506, the convergence of the configuration prediction model is determined using a loss function. Specifically, a regression loss function, such as mean squared error, mean absolute error, HuberLoss, etc., is used to calculate the regression loss on the configuration number. The loss function is used to calculate the loss between the predicted value and the true value. If convergence is not achieved, the above testing process is repeated until convergence is achieved; if convergence is achieved, the configuration prediction model is trained successfully.

[0154] The configuration prediction model in the embodiments of this specification, in addition to being... Figure 8 Besides the example shown, training can also be performed in other ways. For instance, historical data from the device's loading of the model can be used for training, with optimized data obtained from the historical data serving as a supervision signal. Then, regression loss can be calculated for the configuration numbers. Training is complete when the test results converge.

[0155] Figure 3 , Figure 4 , Figure 5 , Figure 6The data processing method of the illustrated embodiment can be executed on a terminal device, such as a terminal device with high computing power. After the user inputs an AI prompt word on the running device, it can trigger communication between the running device and the server to obtain the running model to be loaded onto the running device from the server. The running model can be stored locally on the device. As a terminal device, the running device obtains the model information of the running model (if lightweight processing is required, it is performed before obtaining the model information), as well as the device's own device information and running status information. Using the configuration prediction model, the running model is parsed to obtain the configuration number of each layer of the running model. In this way, the running device can quickly and efficiently load the running model from the hard drive into memory / video memory, so that the device can load and infer the model to generate high-quality content.

[0156] Figure 3 , Figure 4 , Figure 5 , Figure 6 The data processing method in the illustrated embodiment can also be executed on a server (if the method involves loading and inference steps, such as step 208 or step 408, it needs to be executed on the terminal device). When the user inputs an AI prompt word on the running device, it can trigger communication between the running device and the server to obtain the running model to be loaded onto the running device from the server. The server obtains the device information and running status of the running device to be loaded with the running model, and combines it with the model information of the running model (if lightweight processing is required, it is performed before obtaining the model information), and uses the configuration prediction model to parse the running model to obtain the configuration number of each layer of the running model. The server sends the obtained configuration number of each layer of the running model to the running device. Based on the obtained configuration number, the running device loads and infers the model to generate content of relatively good quality. The model training and data processing involved in the embodiments of this specification can be executed on the same server, the same server group, or the same cloud server, or they can be executed on two independent servers or cloud servers respectively, and even terminal devices with strong computing power can be set up. Due to the limited computing power of terminal devices, the above process is preferentially executed on the server. After obtaining the layer configuration number of the running model, the stacking and inference are performed in sequence based on the configuration number at the running device to efficiently generate the target content.

[0157] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0158] Please see Figure 11 The diagram shown is a structural schematic of a data processing apparatus according to an embodiment of this specification.

[0159] like Figure 11 As shown, the data processing device 1000 may include at least an information acquisition module 1001, a feature vector acquisition module 1002, and a configuration number acquisition module 1003, wherein:

[0160] The information acquisition module 1001 is used to acquire equipment information and operating status information of the operating equipment, as well as model information of the operating model.

[0161] The running device is a terminal device used to load the AIGC model and generate AIGC content. The running model is an AIGC model obtained from the server, determined based on the user's actual operations. This running model can be a single model or a group of models.

[0162] The device information includes the device model and storage capacity.

[0163] The model information includes layer type, layer parameters, etc.

[0164] The feature vector acquisition module 1002 is configured with a device feature encoder and a model feature encoder. The device feature encoder receives device information and operating status information to obtain device feature vectors, while the model feature encoder receives model information to obtain model feature vectors. Both the device feature encoder and the model feature encoder are pre-trained before feature vector acquisition.

[0165] The configuration number acquisition module 1003 is equipped with a configuration prediction model, used to input device feature vectors and model feature vectors into the configuration prediction model to obtain the configuration number of each layer of the running model; wherein, the configuration prediction model is a deep learning network model. The configuration prediction model is pre-trained on the server.

[0166] After obtaining the configuration number of each layer of the running model, it can be used to perform specific analysis on each layer of the running model structure. It can also be used by the terminal device to sequentially stack and load and infer according to the configuration number to generate target content. It can also be used in other aspects, which can reduce the impact of excessive model size on the execution efficiency of terminal devices or servers.

[0167] Please see Figure 12 The diagram shown is a structural schematic of another data processing apparatus according to an embodiment of this specification.

[0168] like Figure 12As shown, the data processing device 2000 may include at least an information acquisition module 2001, a feature vector acquisition module 2002, a configuration number acquisition module 2003, and a loading inference module 2004.

[0169] Compared to Figure 11 The example shown, Figure 12 The example shown includes an additional loading inference module 2004. This loading inference module 2004, after obtaining the configuration number of each layer of the running model, sequentially loads the layers of the running model into the running device according to the obtained configuration numbers, and performs inference based on user input information to generate target content corresponding to the user input information on the running device.

[0170] Based on the configuration number of the model layer, the model layers are loaded into the running device sequentially. When model layers have the same configuration number, they are loaded only once.

[0171] The information acquisition module 2001, the feature vector acquisition module 2002, and the configuration number acquisition module 2003 respectively refer to Figure 11 The information acquisition module 1001, the feature vector acquisition module 1002, and the configuration number acquisition module 1003 mentioned in the example will not be described again here.

[0172] The data processing apparatus described in this specification can utilize a configuration prediction model to output the configuration number of each layer of the running model based on the input "information about the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This apparatus can generate high-quality target content with relatively low video memory / memory consumption and controllable time costs.

[0173] Figure 11 , Figure 12 The data processing apparatus of the illustrated embodiment refers to an operating model that conforms to the loading capacity of the operating device.

[0174] Please see Figure 13 The diagram shown is a structural schematic of another data processing apparatus according to an embodiment of this specification.

[0175] like Figure 13 As shown, the data processing device 3000 may include at least a lightweight processing module 30001, an information acquisition module 3002, a feature vector acquisition module 3003, and a configuration number acquisition module 3004.

[0176] Compared to Figure 11 The example shown, Figure 13 The example shown includes an additional lightweight processing module 3001. This lightweight processing module is used to perform lightweight processing on the running model before obtaining its model information when the specifications of the running model exceed the loading capacity of the running device, thereby obtaining a lightweight model that conforms to the loading capacity of the running device.

[0177] The information acquisition module 3001, the feature vector acquisition module 3002, and the configuration number acquisition module 3003 respectively refer to Figure 11 The information acquisition module 1001, the feature vector acquisition module 1002, and the configuration number acquisition module 1003 mentioned in the example will not be described again here.

[0178] Before using the information acquisition module to obtain relevant information about the running device and the running model, the lightweight processing module needs to perform lightweight processing on the running model to be loaded. Often, the running model that needs lightweight processing is one that exceeds the loading capacity of the running device. Lightweighting the running model can speed up the process of dividing the model into sub-models and avoid consuming excessive computing resources, thus optimizing the overall data processing efficiency.

[0179] After lightweighting, the data processing apparatus of this embodiment can utilize a configuration prediction model to output the configuration number of each layer of the running model based on the input "information of the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This method can generate high-quality target content with relatively low video memory / memory consumption and controllable time costs.

[0180] The lightweight processing module may include a judgment unit and a lightweight processing unit. The judgment unit determines whether the running model meets the lightweight processing conditions. If it does, the lightweight processing unit is triggered to perform lightweight processing. If it does not meet the conditions, the information acquisition module works to acquire relevant information for model layer-by-layer parsing. The lightweight processing conditions can be determined based on the size of the running model and the time taken for the running device to load the running model.

[0181] In one embodiment of this specification, the lightweight processing unit may be a distillation unit used to distill the running model to obtain a lightweight model.

[0182] In one example, the distillation unit employs a single-step distillation mode. Specifically, the original model is first pruned to obtain a lightweight model that meets the requirements of the terminal device. Then, the unpruned model is used as the Teacher model, and the pruned model, which is used as the Student model, is distilled.

[0183] In another example, the distillation unit employs a progressive distillation mode. Multiple distillation processes are based on... Figure 10 The distillation architecture shown is implemented as follows:

[0184] The teacher network operates by using user input and random noise as input, and outputs the teacher target content corresponding to the user input. The teacher target content is then input into the teacher selection model, and the optimal teacher target content corresponding to the user input is output.

[0185] The user input information and random noise input are used as a lightweight model of the student network, and the output is the student target content corresponding to the user input information.

[0186] Using the target content of the selected teachers as the supervision signal, the convergence of the result of the lightweight model is judged by the loss function. If it does not converge, the above process is repeated until it converges. If it converges, the first round of distillation ends and the converged lightweight model is obtained. The above process is repeated to perform a new round of distillation.

[0187] When a new round of distillation is performed, the lightweight model used in the new round of distillation is based on the converged lightweight model obtained in the previous round of distillation, and is obtained by reducing the number of channels; the teacher network used in the new round of distillation is the network of the converged lightweight model obtained in the previous round of distillation within a certain number of steps.

[0188] After multiple rounds of distillation, when the lightweight model meets the loading capacity of the operating equipment, the distillation is stopped and the distilled lightweight model is output.

[0189] Please see Figure 14 The present specification provides a schematic diagram of the structure of another data processing device according to an embodiment.

[0190] like Figure 14 As shown, the data processing device 4000 may include at least a lightweight processing module 40001, an information acquisition module 4002, a feature vector acquisition module 4003, a configuration number acquisition module 4004, and a loading inference module 4005.

[0191] Compared to Figure 13 The example shown, Figure 14 The example shown includes an additional loading inference module 4005. This loading inference module 4005, after obtaining the configuration number of each layer of the running model, sequentially loads the layers of the running model into the running device according to the obtained configuration numbers, and performs inference based on user input information to generate target content corresponding to the user input information on the running device.

[0192] Figure 14The lightweight processing module 40001, information acquisition module 4002, feature vector acquisition module 4003, and configuration number acquisition module 4004 shown in the example are referenced. Figure 13 The lightweight processing module 30001, information acquisition module 3002, feature vector acquisition module 3003, and configuration number acquisition module 3004 shown in the example will not be described in detail here.

[0193] The data processing apparatus described in this specification can utilize a configuration prediction model to output the configuration number of each layer of the running model based on the input "information about the running device and the running model". In this way, the running model does not need to be loaded entirely onto the running device. Instead, it is divided into several sub-models, which are then loaded one by one on the running device through a stacking loading method, and inference is performed step by step to finally generate the target content. This apparatus can generate high-quality target content with relatively low video memory / memory consumption and controllable time costs.

[0194] Figure 11 , Figure 12 , Figure 13 , Figure 14 The data processing apparatus in the illustrated embodiment can be a terminal device, such as a terminal device with supercomputing power. Figure 11 , Figure 12 , Figure 13 , Figure 14 When the data processing device in the illustrated embodiment obtains the configuration number of the model layer, the data processing device can be understood as a server, or it may include a server and a terminal device (the server performs the functions of obtaining the configuration number and the functions before obtaining the configuration number, and the terminal device performs the function of loading the inference module when there is one).

[0195] In one or more embodiments of this specification, the configuration prediction model is achieved through... Figure 15 The model training device shown has completed training.

[0196] like Figure 15 As shown, the model training device 5000 may include at least a data acquisition module 5001, a loading module 5002, a training module 5003, a feature acquisition module 5004, a prediction module 5005, and a testing module 5006, wherein:

[0197] The data acquisition module 5001 is used to acquire training data and test data. The training data includes a training model and a training device, and the test data includes a test model and a test device. The model is a model for different purposes, such as a text generation model, an image generation model, a speech generation model, a video generation model, etc. The device has the capability to load the aforementioned models. Several devices are selected from terminal devices used in various application scenarios.

[0198] The loading module 5002 is used to load and run the training model on the training device to obtain prediction results. The loading module can be executed by a simulation platform or by a specific device.

[0199] The loading module 5002 includes: a number combination forming unit, used to generate multiple number combinations based on the training configuration number of the training model; a loading execution unit, used to control each training device to load multiple training models according to the number combination to obtain multiple prediction results; and a prediction result judgment unit, used to determine that the prediction result is the preferred result when the prediction result meets certain conditions.

[0200] The training module 5003 is used to obtain the number combination of the preferred result in the prediction result, and the number combination includes the training configuration number of one, two or more model layers.

[0201] The feature acquisition module 5004 is used to acquire the model feature vector of the test model and the device feature vector of the test device. The feature acquisition module includes at least a model feature encoder and a device feature encoder. The model feature encoder is used to output a model feature vector based on the model information of each layer of the input test model; wherein the model information includes layer type and layer parameters. The device feature encoder is used to output a device feature vector based on the device information and operating status information of the input test device; wherein the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

[0202] The prediction module 5005 is configured with a configuration prediction model, which is used to input the model feature vector and the device feature vector into the configuration prediction model. The configuration prediction model outputs the test configuration number of each layer of the model. The configuration prediction model is a deep learning network model.

[0203] The testing module 5006 is used to use the preferred combination of numbers as a supervision signal and to determine the convergence of the configuration prediction model using a loss function. If convergence fails, the feature acquisition module and the prediction module repeat the above testing process until convergence occurs. If convergence occurs, the configuration prediction model is trained. Specifically, regression loss functions, such as mean squared error, mean absolute error, Huber Loss, etc., are used to calculate the regression loss for the configuration numbers.

[0204] The model training apparatus described in this specification is for training a configuration prediction model. This model can divide the model structure according to how the running model input to the configuration prediction model performs stacked inference on the terminal device. Specifically, it can predict the stacking sequence number of each layer of the running model (i.e., the configuration number as referred to in this specification). After obtaining the configuration number, the terminal device performs loading inference sequentially according to the order of the configuration numbers.

[0205] The model training device and data processing device involved in the embodiments of this specification can be the same server, the same server group, or the same cloud server, or they can be two independent servers or cloud servers, or even terminal devices with strong computing capabilities. Since the computing power of terminal devices is limited, it is preferred to implement the above devices on a server. After obtaining the layer configuration number of the running model, the running device performs stacking loading and inference in sequence based on the layer configuration number to efficiently generate the target content. (Refer to...) Figure 15 The model training device can train the configuration prediction model offline and then load it onto the terminal device online. The configuration prediction model outputs the target content based on the Prompt input, model information, device information, and running status information.

[0206] Please see Figure 16 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0207] like Figure 16 As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, user interface 1103, memory 1105, and at least one communication bus 1102.

[0208] The communication bus 1102 can be used to realize the connection and communication of the above components.

[0209] The user interface 1103 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0210] The network interface 1104 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0211] The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines. It executes instructions, programs, code sets, or instruction sets stored in the memory 1105, and calls data stored in the memory 1105 to perform various functions and process data within the routing device 1100. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1101 and may be implemented as a separate chip.

[0212] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. Figure 11 As shown, the memory 1105, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 can be used to call the application programs stored in the memory 1105 and execute the methods in one or more of the above embodiments.

[0213] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the above-described instructions. Figure 4 or Figure 5 or Figure 6 or Figure 7 or Figure 8 One or more steps in the illustrated embodiment. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0214] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0215] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0216] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. Model training methods, including: Acquire training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment; The training model is loaded and run on the training device to obtain prediction results, and the number combination of the preferred results in the prediction results is obtained. The number combination contains one or two or more training configuration numbers of the model layer; including: generating multiple number combinations based on the training configuration numbers of the training model. Each training device loads multiple training models according to the number combination, and obtains multiple prediction results; When a prediction result meets certain conditions, it is considered the preferred result. Obtain the number combination under the preferred result; Obtain the model feature vector of the test model and the device feature vector of the test device, input the model feature vector and the device feature vector into the configuration prediction model, and the configuration prediction model outputs the test configuration number of each layer of the model; wherein, the configuration prediction model is a deep learning network model; Using the number combination under the preferred result as the supervision signal, the result of the configuration prediction model is converged using the loss function. If it does not converge, the above test process is repeated until it converges; if it converges, the configuration prediction model is trained.

2. The method according to claim 1, wherein the process of loading and running the training model on the training device is simulated by a simulation platform.

3. The method according to claim 1, wherein obtaining the model feature vector of the test model and the device feature vector of the test device includes: The model information of each layer of the test model is input into the model feature encoder, and the model feature encoder outputs the model feature vector; wherein, the model information includes layer type and layer parameters; The device information and operating status information of the test equipment are input into the device feature encoder, and the device feature encoder outputs the device feature vector; wherein, the device information includes the device model and storage capacity; the operating status information includes system utilization and storage occupancy rate.

4. A model training device, comprising: A data acquisition module is used to acquire training data and test data, wherein the training data includes a training model and training equipment, and the test data includes a test model and test equipment; The loading module is used to load and run the training model on the training device to obtain prediction results; The loading module includes: Number combination forming unit, used to generate multiple sets of number combinations based on the training configuration number of the training model; The loading execution unit is used to control each training device to load multiple training models according to the number combination, and obtain multiple prediction results; The prediction result judgment unit is used to determine the prediction result as the preferred result when the prediction result meets certain conditions; The training module is used to obtain the number combination of the preferred result in the prediction results, and the number combination includes the training configuration number of one, two or more model layers; The feature acquisition module is used to acquire the model feature vector of the test model and the device feature vector of the test device; The prediction module is configured with a configuration prediction model, which is used to input the model feature vector and the device feature vector into the configuration prediction model. The configuration prediction model outputs the test configuration number of each layer of the model. The configuration prediction model is a deep learning network model. The testing module is used to use the number combination under the preferred result as a supervision signal and to use the loss function to judge the convergence of the result of the configuration prediction model. If it fails to converge, the feature acquisition module and the prediction module repeat the above testing process until convergence. If it converges, the configuration prediction model is trained.

5. The apparatus according to claim 4, wherein the loading module is executed by the simulation platform to realize the process of loading and running the training model on the training device.

6. The apparatus according to claim 4, wherein the feature acquisition module comprises: A model feature encoder is used to output a model feature vector based on the model information of each layer of the input test model; wherein the model information includes layer type and layer parameters; The device feature encoder is used to output a device feature vector based on the input device information and operating status information of the test device; wherein, the device information includes device model and storage capacity; and the operating status information includes system utilization and storage occupancy rate.

7. Data processing methods, including: The system acquires device information and operational status information of the running equipment, as well as model information of the running model; wherein, the model information includes layer type and layer parameters; the device information includes device model and storage capacity; and the operational status information includes system utilization and storage occupancy rate. Input the device information and operating status information into the device feature encoder to obtain the device feature vector; input the model information into the model feature encoder to obtain the model feature vector; The device feature vector and model feature vector are input into the configuration prediction model to obtain the configuration number of each layer of the running model; wherein, the configuration prediction model is a deep learning network model; Based on the obtained configuration number, the layers of the running model are loaded into the running device in sequence, and reasoning is performed in combination with user input information to generate the target content corresponding to the user input information on the running device.

8. The method according to claim 7, wherein the configuration prediction model is trained according to any one of claims 1-3.

9. The method of claim 7, further comprising, prior to obtaining the model information of the operational model: The operating model is lightweighted to obtain a lightweight model that meets the loading capacity of the operating equipment.

10. The method according to claim 9, wherein the lightweight model is obtained by distillation.

11. The method according to claim 9, wherein the step of lightweighting the operating model to obtain a lightweight model that meets the loading capacity of the operating device includes: The system uses user input information and random noise input as the operating model of the teacher network, and outputs the first teacher target content corresponding to the user input information. Input the first teacher's target content into the teacher selection model, and output the second teacher's target content corresponding to the user's input information; The user input information and random noise input are used as a lightweight model of the student network, and the output is the student target content corresponding to the user input information. Using the second teacher's target content as the supervision signal, the convergence of the lightweight model is judged by the loss function. If it does not converge, the above process is repeated until it converges. If it converges, the first round of distillation ends and the converged lightweight model is obtained. The above process is repeated to perform a new round of distillation. When a new round of distillation is performed, the lightweight model used in the new round of distillation is based on the converged lightweight model obtained in the previous round of distillation, and is obtained by reducing the number of channels; the teacher network used in the new round of distillation is the network of the converged lightweight model obtained in the previous round of distillation within a certain number of steps. After multiple rounds of distillation, when the lightweight model meets the loading capacity of the operating equipment, the distillation is stopped and the distilled lightweight model is output.

12. A data processing device, comprising: The information acquisition module is used to acquire equipment information and operating status information of the running equipment, as well as model information of the running model; wherein, the model information includes layer type and layer parameters; the equipment information includes equipment model and storage capacity; and the operating status information includes system utilization and storage occupancy rate. The feature vector acquisition module is equipped with a device feature encoder and a model feature encoder. It is used to input device information and operating status information into the device feature encoder to obtain device feature vectors; and to input model information into the model feature encoder to obtain model feature vectors. The configuration number acquisition module is configured with a configuration prediction model, which is used to input the device feature vector and the model feature vector into the configuration prediction model to obtain the configuration number of each layer of the running model; wherein, the configuration prediction model is a deep learning network model; The loading inference module is used to load the layers of the running model into the running device sequentially according to the configuration number of each layer of the running model after obtaining the configuration number, and to perform inference in combination with user input information to generate target content corresponding to the user input information on the running device.

13. The apparatus of claim 12, wherein the configuration prediction model is trained by any one of claims 1-3.

14. The apparatus of claim 12, further comprising: The lightweight processing module is used to perform lightweight processing on the running model before obtaining the model information when the specifications of the running model exceed the loading capacity of the running device, so as to obtain a lightweight model that meets the loading capacity of the running device.

15. The apparatus of claim 14, wherein the lightweight processing module comprises: The distillation unit is used to distill the running model to obtain a lightweight model.

16. The apparatus of claim 14, wherein the lightweight processing module comprises: The distillation unit is used to distill the running model multiple times to obtain a lightweight model; The multiple distillation process is as follows: The user input information and random noise input are used as the operating model of the teacher network, and the output is the first teacher target content corresponding to the user input information; the first teacher target content is input into the teacher selection model, and the output is the second teacher target content corresponding to the user input information. The user input information and random noise input are used as a lightweight model of the student network, and the output is the student target content corresponding to the user input information. Using the second teacher's target content as the supervision signal, the convergence of the lightweight model is judged by the loss function. If it does not converge, the above process is repeated until it converges. If it converges, the first round of distillation ends and the converged lightweight model is obtained. The above process is repeated to perform a new round of distillation. When a new round of distillation is performed, the lightweight model used in the new round of distillation is based on the converged lightweight model obtained in the previous round of distillation, and is obtained by reducing the number of channels; the teacher network used in the new round of distillation is the network of the converged lightweight model obtained in the previous round of distillation within a certain number of steps. After multiple rounds of distillation, when the lightweight model meets the loading capacity of the operating equipment, the distillation is stopped and the distilled lightweight model is output.

17. Electronic devices, including processors and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-3 and 7-11.

18. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as claimed in any one of claims 1-3 and 7-11.

Images