Data processing methods, devices, and electronic equipment based on scaling models

Abstract

This application discloses a data processing method, apparatus, and electronic device based on a scaling model. The method, applied in the field of artificial intelligence, includes: receiving first text input by a target user; inputting the first text into a target model, generating target text to respond to the first text through the target model; and displaying the target text through a target device in response to the first text input by the target user. This application solves the problem in related technologies where using an inflated deep learning model to respond to user requests requires a significant amount of time for pruning the model based on device conditions and training the pruned model, resulting in long model training times and low efficiency in responding to requests.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a data processing method, apparatus, and electronic device based on a scaling model. Background Technology

[0002] Training and inference of large-scale language models place high demands on computing resources. In terms of training, training a language model with tens or hundreds of billions of parameters requires significant GPU training time, making it difficult to train even larger models (e.g., hundreds of billions of parameters) and slowing down model updates. In terms of inference, large-scale language models have high memory consumption and inference latency, making them difficult to deploy on resource-constrained devices and resulting in slower inference speeds.

[0003] To reduce the resource consumption of model training and the resource usage and latency of model inference, a feasible approach is to expand the network parameters of the model during training, i.e., inflate the model, then prune the inflated model, and finally use the pruned model to process user input questions, generate response text, and respond to user input based on the response text. However, existing model pruning methods directly delete some parameters by evaluating their importance, which generally requires a long training period to achieve a performance close to the original model, making it very inefficient and reducing the model's efficiency in responding to user requests. Furthermore, existing dynamic network methods applied to large-scale language models can only prune parts of the network structure (i.e., delete some layers), which is not flexible enough and can only compress the model to a few fixed parameter sizes, thus reducing the model's accuracy during inference.

[0004] There is currently no effective solution to the problem that when using an inflated deep learning model to respond to user requests in related technologies, it takes a long time to prune the model according to the device conditions and then train the pruned model, resulting in long model training time and low efficiency in responding to requests. Summary of the Invention

[0005] The main objective of this application is to provide a data processing method, apparatus, and electronic device based on a scaling model, in order to solve the problem in related technologies where, when using an inflated deep learning model to respond to user requests, it is necessary to spend a long time pruning the model according to the device conditions and training the pruned model, resulting in a long model training time and low efficiency in responding to requests.

[0006] To achieve the above objectives, according to one aspect of this application, a data processing method based on a scaling model is provided. The method includes: receiving first text input by a target user; inputting the first text into a target model; generating target text for responding to the first text through the target model; wherein the target model is a model that is pre-trained during model training, has its network parameters expanded after pre-training, and has its network parameters pruned during model inference; and displaying the target text through a target device in response to the first text input by the target user.

[0007] Further, the target model is trained by the following steps: pre-training a preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question-and-answer text; expanding the network parameters of the first model by copying the network parameters, and training the expanded first model to obtain a second model; calculating the space occupied by the model based on the device information of the target device to obtain a target value; compressing the network parameters of the second model based on the target value to obtain the target model.

[0008] Further, the network parameters of the first model are expanded by copying the network parameters, including: determining N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; updating the first model based on the N first parameter matrices to obtain a third model; determining M second network structures based on a second network structure, wherein the second network structure is the network structure in the third model, and M is a positive integer; updating the third model based on the M second network structures to obtain the expanded first model.

[0009] Further, the network parameters of the second model are compressed according to the target value to obtain the target model, including: calculating the number of pruning branches according to the target value; pruning the N first parameter matrices in the second model according to the number of pruning branches to obtain a fourth model; and pruning the M second network structures in the fourth model according to a preset number to obtain the target model.

[0010] Furthermore, the first model includes a router module. After determining N first parameter matrices based on a preset numerical matrix, the method further includes: determining the weight value of each of the N first parameter matrices through the router module; and calculating the output results of the N first network structures based on the weight value of each first parameter matrix when detecting first data input to the N first network structures corresponding to the N first parameter matrices.

[0011] Furthermore, after determining M second network structures based on the second network structure, the method further includes: when second data is detected input to the M second network structures, obtaining the output results of the M second network structures; inputting the output results of the M second network structures into a preset mapping matrix, and calculating the loss function value of each of the M second network structures; and updating the network parameters in the extended first model based on the loss function value of each second network structure.

[0012] Further, calculating the number of pruning branches based on the target value includes: obtaining training samples and constructing a calibration set based on the training samples; for each of the M second network structures in the second model, inputting the calibration set into the N first parameter matrices in the second network structure, and obtaining the weight value of each first parameter matrix in the N first parameter matrices; determining the number of pruning branches based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning branches is an integer greater than a preset value.

[0013] Furthermore, after pruning the N first parameter matrices in the second model according to the pruning quantity to obtain the fourth model, the method further includes: calculating the weight value of each first parameter matrix in the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; and training each first parameter matrix and the weight value of each first parameter matrix in the fourth model using training samples to update each first parameter matrix in the fourth model.

[0014] To achieve the above objectives, according to another aspect of this application, a data processing apparatus based on a scaling model is provided. The apparatus includes: a receiving unit for receiving first text input by a target user; a generating unit for inputting the first text into a target model and generating target text to respond to the first text through the target model, wherein the target model is a model pre-trained during model training, with its network parameters expanded after pre-training and pruned during model inference; and a response unit for displaying the target text through a target device in response to the first text input by the target user.

[0015] Furthermore, the device further includes: a training unit, used to pre-train a preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question-and-answer text; an expansion unit, used to expand the network parameters of the first model by copying the network parameters, and train the expanded first model to obtain a second model; a calculation unit, used to calculate the space occupied by the model based on the device information of the target device to obtain a target value; and a compression unit, used to compress the network parameters of the second model based on the target value to obtain the target model.

[0016] Further, the expansion unit includes: a first determining subunit, configured to determine N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​a parameter matrix of the first network structure in the first model, and N is a positive integer; a first updating subunit, configured to update the first model based on the N first parameter matrices to obtain a third model; a second determining subunit, configured to determine M second network structures based on a second network structure, wherein the second network structure is a network structure in the third model, and M is a positive integer; and a second updating subunit, configured to update the third model based on the M second network structures to obtain the expanded first model.

[0017] Further, the compression unit includes: a first calculation subunit, used to calculate the number of pruning branches based on the target value; a first pruning processing subunit, used to prune the N first parameter matrices in the second model based on the number of pruning branches to obtain a fourth model; and a second pruning processing subunit, used to prune the M second network structures in the fourth model based on a preset number to obtain the target model.

[0018] Furthermore, the first model includes a router module, and the extension unit further includes: a third determining subunit, used to determine the weight value of each of the N first parameter matrices through the router module after determining N first parameter matrices based on a preset numerical matrix; and a second calculation subunit, used to calculate the output results of the N first network structures based on the weight value of each first parameter matrix when the first data input to the N first network structures corresponding to the N first parameter matrices is detected.

[0019] Furthermore, the expansion unit further includes: an acquisition subunit, configured to acquire the output results of the M second network structures after determining the M second network structures based on the second network structure, and upon detecting second data input to the M second network structures; a third calculation subunit, configured to input the output results of the M second network structures into a preset mapping matrix and calculate the loss function value of each of the M second network structures; and a third update subunit, configured to update the network parameters in the expanded first model based on the loss function value of each second network structure.

[0020] Further, the first computational subunit includes: a construction module, configured to acquire training samples and construct a calibration set based on the training samples; an acquisition module, configured to input the calibration set into the N first parameter matrices in the second network structure for each of the M second network structures in the second model, and acquire the weight value of each first parameter matrix in the N first parameter matrices; and a determination module, configured to determine the number of pruning operations based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning operations is an integer greater than a preset value.

[0021] Furthermore, the compression unit further includes: a fourth calculation subunit, used to calculate the weight value of each first parameter matrix in the fourth model after pruning the N first parameter matrices in the second model according to the pruning quantity to obtain the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; and a fourth update subunit, used to train each first parameter matrix and the weight value of each first parameter matrix in the fourth model using training samples to update each first parameter matrix in the fourth model.

[0022] To achieve the above objectives, according to one aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the data processing method based on the scaling model described in any of the above claims, and when executed by a processor, implements the steps of the data processing method based on the scaling model described in various embodiments of this application.

[0023] To achieve the above objectives, according to one aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including stored computer instructions, wherein, when the computer instructions are executed by a processor, the data processing method based on the scaling model described in any one of the above claims is implemented.

[0024] To achieve the above objectives, according to one aspect of this application, an electronic device is provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data processing method based on the scaling model described in any of the above claims.

[0025] This application employs the following steps: receiving first text input by a target user; inputting the first text into a target model, generating target text to respond to the first text through the target model, wherein the target model is a model that is pre-trained during model training, its network parameters are expanded after pre-training, and its network parameters are pruned during model inference; and displaying the target text through a target device to respond to the first text input by the target user. This solves the problem in related technologies where, when using an expanded deep learning model to respond to user requests, a long time is required to prune the model according to the device conditions and train the pruned model, resulting in a long model training time and low efficiency in model response to requests. By pre-training the model before actual training and expanding the network parameters of the pre-trained model, the pre-trained model parameters can be initialized more efficiently, significantly reducing the training cost and computational consumption during training. Furthermore, by reducing the number of network parameters during inference based on the expanded network parameters, the model's resource consumption and inference latency are reduced while maintaining accuracy. This avoids the problem of low model response efficiency caused by spending excessive time training the pruned model, thus improving the model's efficiency in responding to user requests. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a flowchart of a data processing method based on a scaling model provided in Embodiment 1 of this application;

[0028] Figure 2 This is a schematic diagram of an optional data processing method based on a scaling model provided in Embodiment 1 of this application. Figure 1 ;

[0029] Figure 3 This is a schematic diagram of an optional data processing method based on a scaling model provided in Embodiment 1 of this application. Figure 2 ;

[0030] Figure 4 This is a schematic diagram of an optional data processing method based on a scaling model provided in Embodiment 1 of this application. Figure 3 ;

[0031] Figure 5 This is a schematic diagram of a data processing apparatus based on a scaling model provided in Embodiment 2 of this application;

[0032] Figure 6 This is a schematic diagram of a data processing electronic device based on a scaling model provided in Embodiment 5 of this application. Detailed Implementation

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] It should be noted that the user information (including but not limited to user device information, user personal information, collected data, used data, generated data, processed data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, collected information, used information, generated information, processed information, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, and necessary confidentiality measures have been taken. These measures do not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0035] It should be noted that this application provides users with a corresponding entry point for choosing to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0036] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0039] Transformer is a neural network architecture based on self-attention. Through self-attention and a layer-by-layer stacked encoder-decoder structure, it achieves the ability to globally model the input sequence and generate the target sequence.

[0040] Text pre-training: Text pre-training refers to the process of training a model on a large-scale text dataset. This initial training phase allows the model to learn general features and representations from the text data, which can then be fine-tuned for specific tasks.

[0041] Mixture of Experts (MoE): A mixture of expert models is a machine learning technique that uses multiple expert networks to partition the problem space into homogeneous regions. A significant advantage of mixture of expert models is their ability to be effectively pre-trained with far fewer computational resources than dense models. This means that, within the same computational budget, the model size can be significantly increased.

[0042] Early Exiting is a technique used in deep neural network (DNN) inference to reduce redundant computation and improve inference speed. Its core idea is to add conditional statements to the intermediate layers of the network so that results are output early when simple samples are identified, without having to execute the entire deep structure of the network.

[0043] Example 1

[0044] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a data processing method based on a scaling model provided in Embodiment 1 of this application, as follows: Figure 1 As shown, the method includes the following steps:

[0045] Step S101: Receive the first text input by the target user.

[0046] The data processing method based on the scaling model provided in Embodiment 1 of this application can be applied to a target device, which is a device that interacts with the user. For example, the user inputs a question or other text information into the target device. The target device receives the text information input by the user, processes the text information input by the user through the target model to generate a reply text, and displays the reply text in the target device in response to the text information input by the user.

[0047] In this first embodiment, in order to respond to a user request (i.e., to reply to the first text entered by the user), it is necessary to receive the first text entered by the user, generate a reply text based on the first text entered by the user, and respond to the user request based on the reply text.

[0048] It's important to note that the target model described above, besides being used for general knowledge-based question answering, can also be applied to many other projects such as code generation, knowledge embedding, long-form writing, intelligent data retrieval, intelligent customer service (or digital humans), and email generation. No specific limitations are imposed in this first embodiment. Therefore, the first text input by the user could be something like, "Please generate Python code for a convolutional neural network model for image classification," "Please explain what a Long Short-Term Memory (LSTM) network is and its application in time series forecasting," "Please write a paper on the application of deep learning in medical image analysis," "I need to obtain user behavior data from the past year, including click-through rates and conversion rates," "I need an AI customer service robot that can answer common questions," and "Please generate an automated reply email to inform the user that their order has been shipped," etc.

[0049] Step S102: Input the first text into the target model, and generate target text to respond to the first text through the target model. The target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference.

[0050] In existing technologies, to reduce the resource consumption of model training, a feasible approach is to expand the network parameters of the model during training, i.e., dilate the model, then prune the expanded model, and finally use the pruned model to process user input questions during inference, generating response text and responding to user input based on the response text. However, existing model pruning methods directly delete some parameters by evaluating their importance, which generally requires a long training period to achieve a performance close to the original model, making it very inefficient.

[0051] In this first embodiment, the model needs to be pre-trained before model training, and the network parameters of the pre-trained model need to be expanded. During model inference, the network parameters of the model are reduced accordingly based on the method of expanding the network parameters to obtain the target model. The target model is then used to generate response text, thereby combining the two processes of model expansion and model scaling. This enables the mutual conversion between small and large models, thereby accelerating the inference speed of the model without affecting the model's accuracy, reducing the response time of the model to user requests, and improving the model's response efficiency.

[0052] Step S103: Display the target text on the target device in response to the first text input by the target user.

[0053] In this first embodiment, in order to satisfy the user's request, the response text generated by the target model (i.e., the target text mentioned above) needs to be displayed on the target device that interacts with the target user. The target device displays the target text to the target user in response to the first text input by the target user.

[0054] In summary, the data processing method based on a scaling model provided in Embodiment 1 of this application receives first text input by a target user; inputs the first text into a target model; generates target text to respond to the first text through the target model; wherein the target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference; and displays the target text through a target device to respond to the first text input by the target user. This solves the problem in related technologies where, when using an expanded deep learning model to respond to user requests, a long time is required to prune the model according to the device conditions and train the pruned model, resulting in a long model training time and low efficiency in responding to requests. By pre-training the model before actual training and expanding the network parameters of the pre-trained model, the pre-trained model parameters can be initialized more efficiently, significantly reducing the training cost and computational consumption during training. Furthermore, by reducing the number of network parameters during inference based on the expanded network parameters, the model's resource consumption and inference latency are reduced while maintaining accuracy. This avoids the problem of low model response efficiency caused by spending excessive time training the pruned model, thus improving the model's efficiency in responding to user requests.

[0055] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, the target model is trained by the following steps: pre-training the preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question and answer text; expanding the network parameters of the first model by copying the network parameters, and training the expanded first model to obtain a second model; calculating the space occupied by the model based on the device information of the target device to obtain a target value; compressing the network parameters of the second model based on the target value to obtain the target model.

[0056] In this first embodiment, to improve model response efficiency, a pre-trained model with a small number of network parameters can be trained until convergence, resulting in a first model. The network parameters of the first model are then expanded, inflating it into a larger model with a larger number of parameters (i.e., the expanded first model described above). This process continues until convergence, resulting in the second model described above. During model training, the network parameters of the pre-trained first model can provide good initialization parameters for the expanded first model, making the training cost of the expanded first model significantly lower than training from scratch.

[0057] Then, to reduce the resource consumption and inference latency of the model inference, the actual amount of computing resources can be determined based on the device information of the target device. Based on this actual computing resource allocation, a suitable model size, i.e., the aforementioned target value, can be determined. Finally, based on the target value, the second model with a larger number of network parameters is scaled down to the target model with a smaller number of network parameters, while simultaneously ensuring that the target model possesses the performance characteristics of the second model.

[0058] In this first embodiment, the Transformer model (hereinafter referred to as the Transformer model) is used as an example to explain the expansion and scaling process of the model network parameters, that is, the Transformer model is used as the preset model mentioned above. It should be noted that in actual production activities, other deep learning models can be used as the preset model according to the actual production situation, and no specific restrictions are made in this first embodiment.

[0059] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, the network parameters of the first model are expanded by copying the network parameters, including: determining N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; updating the first model based on the N first parameter matrices to obtain a third model; determining M second network structures based on the second network structure, wherein the second network structure is the network structure in the third model, and M is a positive integer; updating the third model based on the M second network structures to obtain the expanded first model.

[0060] In this first embodiment, in order to reduce the model training cost, the pre-trained first model can be horizontally and vertically expanded.

[0061] In one alternative embodiment, the process of horizontally scaling the Transformer model can be as follows: Figure 2 As shown, by copying N copies of the first model's self-attention (e.g., ... Figure 2 The network parameter matrices of the attn1, attn2, attn3, and attn4 modules (i.e., the first parameter matrix mentioned above) and the MLP modules (such as...) Figure 2 The network parameter matrices of MLP1, MLP2, MLP3, and MLP4 (i.e., the first parameter matrix mentioned above) are used to form K independent expert modules. For example, the W parameter of the self-attention module in the first model can be... q W k W v The four network parameter matrices W are copied N times to obtain Where i = 1,...,N. For the input vector x of the self-attention module, the calculation process of the self-attention module can be shown in Equations 1 to 4.

[0062]

[0063] h (i) =attn(Q (i) k (i) V (i) )(three)

[0064]

[0065] Among them, y (i) W represents the output of the i-th self-attention module. q W k W v Let W represent the Q, K, and V mapping matrices in the self-attention module, each of size d×d. o This represents the output mapping matrix of the self-attention module, with a size of d×d, where d represents the hidden layer size and h represents the output mapping matrix of the self-attention module. (i) This typically represents the output after processing by the i-th self-attention module. The first model is updated based on the network parameter matrices of the self-attention modules (copied N times) and the network parameter matrices of the MLP modules (copied N times), along with N first parameter matrices, to obtain the third model.

[0066] In one alternative embodiment, the process of vertically scaling the Transformer model can be as follows: Figure 3As shown, for a Transformer model with L layers, a model with 2L layers can be obtained by vertical expansion (e.g., ...). Figure 3 In a larger parameter model (two Decoder Blocks × L), where i ≤ L, the parameters of the i-th layer can be directly copied to the (i+L)-th layer. For example, for the MLP module in the third model (i.e., the second network structure mentioned above), the W1 and W2 matrices of the third model are copied M times (i.e., M copies of the second network structure mentioned above), resulting in... Where i = 1, ..., M. For the input vector x of the MLP module, the calculation process of the MLP module can be shown in Formula 5.

[0067]

[0068] Among them, y (i) This represents the output of the i-th MLP module. Let W1 ∈ R represent two mapping matrices of the MLP layer. d×4d ,W2∈R 4d×d d represents the hidden layer size, and f represents the computation function corresponding to the MLP module. The third model is updated based on the M second network structures to obtain the expanded first model.

[0069] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, the network parameters of the second model are compressed according to the target value to obtain the target model, including: calculating the number of pruning branches according to the target value; pruning the N first parameter matrices in the second model according to the number of pruning branches to obtain the fourth model; and pruning the M second network structures in the fourth model according to a preset number to obtain the target model.

[0070] In this first embodiment, in order to reduce the memory usage and latency of inference, the number of network parameters that need to be pruned (i.e., the number of pruning parameters mentioned above) can be determined based on the actual computing resources of the target device used by the user, and then the trained second model can be pruned according to the number of pruning parameters.

[0071] In one alternative embodiment, before deploying the model, the required compression size of the model, i.e., the aforementioned target value, can be determined based on the video memory size and computing resource limitations of the target device used by the target user. Then, through horizontal and vertical compression, the space occupied by the second model is compressed to the target value.

[0072] In one alternative embodiment, lateral compression is the inverse process of lateral expansion. During model inference, the number of pruning parameters, i.e., the number of network parameters that need to be pruned, can be calculated based on the target value. Based on the number of pruning parameters, pruning is performed on the N first parameter matrices that are laterally expanded from the second model, pruning some layers of expert models to obtain the aforementioned fourth model. This transforms the N expert models into sparse models, reducing memory usage and inference latency.

[0073] Vertical compression is the inverse process of vertical expansion. After pruning the N first parameter matrices of the second model, the M second network structures in the fourth model can be pruned. Since the first L layers and the overall model (M×L layers) of the second model have token generation capabilities during training, for example, only the first L layers can be used for inference during model inference, that is, the network structure of the (i+L)th layer (i = 1, 2, ..., (M-1)×L) in the fourth model can be pruned to obtain the target model described above. It should be noted that 2×L layers can also be used for inference during model inference, and no specific limitation is made in this embodiment.

[0074] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, the first model mentioned above includes a router module. After determining N first parameter matrices based on a preset numerical matrix, the method further includes: determining the weight value of each of the N first parameter matrices through the router module; and when the first data input is detected to correspond to N first network structures corresponding to the N first parameter matrices, calculating the output results of the N first network structures based on the weight value of each first parameter matrix.

[0075] In this first embodiment, in order to avoid the expansion of network parameters having a negative impact on the accuracy of the target model, after horizontally expanding the network parameters, the output results of the N first network structures corresponding to the N first parameter matrices can be determined by adjusting the weight values ​​of the N first parameter matrices.

[0076] In an optional embodiment, after inputting N first network structures via vector x, a router module G is added. The router module calculates the weight value corresponding to each first network structure based on vector x, and performs a weighted sum of the outputs of the N first network structures to obtain the output results of the N first network structures. For example, the calculation process for the output results of the N first network structures can be shown in Formulas 6 and 7.

[0077] G = softmax(W) g x) (VI)

[0078]

[0079] Where G represents the weight matrix of the N first network structures, W g The router module mentioned above, W g ∈R d×N , i represents the i-th first network structure, y (i) Let G represent the output vector of the i-th first network structure, y represent the output results of N first network structures, and G... i represents the weight value of the i-th first network structure, and softmax represents the normalization process.

[0080] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, after determining M second network structures according to the second network structure, the method further includes: when the second data input to the M second network structures is detected, obtaining the output results of the M second network structures; inputting the output results of the M second network structures into a preset mapping matrix, and calculating the loss function value of each of the M second network structures; and updating the network parameters in the expanded first model according to the loss function value of each second network structure.

[0081] In this first embodiment, in order to maintain the accuracy of the model when scaling the network parameters after expanding the model, after copying M second network structures, the output vector of each of the M second network structures is input into the lm head mapping matrix, and the loss function value of the output vector of each second network structure is calculated using an independent language model loss module. The network parameters in the expanded first model are updated using the loss function value of the output vector of each second network structure, so that each of the M second network structures has the ability to generate tokens.

[0082] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, calculating the number of pruning branches according to the target value includes: obtaining training samples and constructing a calibration set based on the training samples; for each of the M second network structures in the second model, inputting the calibration set into the N first parameter matrices in the second network structure, and obtaining the weight value of each first parameter matrix in the N first parameter matrices; and determining the number of pruning branches based on the weight value of each first parameter matrix in each second network structure.

[0083] In this first embodiment, in order to ensure the accuracy of the model after pruning, the number of branches to be pruned can be determined according to the size of the compression target (i.e., the target value mentioned above).

[0084] In an optional embodiment, a small number of samples can be randomly drawn from the training set to construct the calibration set before model training. The sample data from the calibration set is then input into the N first parameter matrices of each of the M second network structures in the second model to obtain the weight values ​​of each first parameter matrix in each second network structure.

[0085] Then, the sum of the weight values ​​assigned to each first parameter matrix in the M second network structures by the routing module is calculated to obtain the sum of the weight values ​​of each first parameter matrix in the N first parameter matrices. The sum of the weight values ​​of each first parameter matrix in the N first parameter matrices is sorted in ascending order, and the P first parameter matrices with the smallest sum of weight values ​​in the N first parameter matrices are pruned.

[0086] It is important to note that after pruning the N first parameter matrices in the M second network structures, it is necessary to ensure that at least one first parameter matrix is ​​retained in each second network structure.

[0087] Optionally, in the data processing method based on the scaling model provided in Embodiment 1 of this application, after pruning the N first parameter matrices in the second model according to the number of prunings to obtain the fourth model, the above method further includes: calculating the weight value of each first parameter matrix in the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; and using training samples to train each first parameter matrix and the weight value of each first parameter matrix in the fourth model to update each first parameter matrix in the fourth model.

[0088] In this first embodiment, to ensure the accuracy of the model after pruning, when calculating the weights of the routing module, normalization (softmax) can be performed only on the retained first parameter matrices, so that the sum of the weight values ​​of all first network structures corresponding to all retained first parameter matrices is 1. Then, a small amount of sample data can be used to train each first parameter matrix and its weight value in the fourth model, updating each first parameter matrix in the fourth model to reduce the performance loss after model pruning.

[0089] Optionally, in this first embodiment, the process of using the target model with expanded and scaled network parameters to respond to the user's request can be as follows: Figure 4 As shown.

[0090] First, a randomly initialized Transformer model (i.e., the preset model mentioned above) is pre-trained from scratch. Then, it is trained to convergence using a dataset containing 1T tokens, resulting in model M1 (i.e., the first model mentioned above). For example, this embodiment uses a model structure with 7 billion (or simply 7B) parameters as the preset model. This model has 30 transformer layers, a hidden layer size of 4096, 32 attention heads per layer, a vocabulary size of 120,000, and is trained using the Megatron-Deepspeed framework with a training dataset of 1T tokens.

[0091] Then, each Transformer layer in model M1, which contains 7B parameters, is horizontally expanded to obtain the third model described above, where N=2 during the horizontal expansion process. The third model is trained using a pre-set training set. After a short training period, a decrease in the loss of the third model can be observed, and the rate of decrease is faster than when directly training the model with 7B parameters. After expanding model M1, the number of parameters is 13B. Training with 200B tokens is continued on the pre-training dataset until the model converges, resulting in model M2, whose parameter size has doubled.

[0092] Secondly, the Transformer layer of model M2, which contains 13B parameters, is vertically expanded to obtain the expanded model described above. After a short training period, a decrease in loss can be observed, and the rate of decrease is faster than when directly training the model with 13B parameters. After vertical expansion, the model has 25B parameters. Training with 100B tokens on the pre-training dataset continues until the model converges, resulting in model M3, whose parameter size has increased fourfold.

[0093] Next, based on the target device's memory information and computing resources, the model compression size (i.e., the target value mentioned above) is determined, and the number of pruning operations is also determined. The second model is compressed to the target value using lateral pruning and early stopping in the vertical direction, resulting in the compressed model M3'. Furthermore, model M3' is trained on a small amount of data for a small number of steps to improve its performance.

[0094] Finally, model M3' is deployed on the target device to enable interaction between the target user and the model M3' on the target device. For example, the target user inputs first text on the target device, model M3' generates target text, and the target text is displayed on the target device in response to the user's first text input.

[0095] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0096] Example 2

[0097] This application's second embodiment also provides a data processing apparatus based on a scaling model. It should be noted that the scaling model-based data processing apparatus of this application's second embodiment can be used to execute the scaling model-based data processing method provided in this application's first embodiment. The scaling model-based data processing apparatus provided in this application's second embodiment will be described below.

[0098] Figure 5 This is a schematic diagram of a data processing apparatus based on a scaling model according to Embodiment 2 of this application. Figure 5 As shown, the device includes a receiving unit 501, a generating unit 502, and a response unit 503.

[0099] Specifically, the receiving unit 501 is used to receive the first text input by the target user.

[0100] The generation unit 502 is used to input the first text into the target model and generate target text to respond to the first text through the target model. The target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference.

[0101] The response unit 503 is used to display target text through the target device in response to the first text input by the target user.

[0102] The data processing apparatus based on a scaling model provided in Embodiment 2 of this application receives first text input by a target user through a receiving unit 501; a generating unit 502 inputs the first text into a target model and generates target text to respond to the first text through the target model. The target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference; a response unit 503 displays the target text through a target device to respond to the first text input by the target user. This solves the problem in related technologies where, when using an expanded deep learning model to respond to user requests, it is necessary to spend a long time pruning the model according to the device conditions and training the pruned model, resulting in a long model training time and low efficiency in model response to requests. By pre-training the model before actual training and expanding the network parameters of the pre-trained model, the pre-trained model parameters can be initialized more efficiently, significantly reducing the training cost and computational consumption during training. Furthermore, by reducing the number of network parameters during inference based on the expanded network parameters, the model's resource consumption and inference latency are reduced while maintaining accuracy. This avoids the problem of low model response efficiency caused by spending excessive time training the pruned model, thus improving the model's efficiency in responding to user requests.

[0103] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the apparatus further includes: a training unit, used to pre-train a preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question-and-answer text; an expansion unit, used to expand the network parameters of the first model by copying the network parameters, and train the expanded first model to obtain a second model; a calculation unit, used to calculate the space occupied by the model based on the device information of the target device to obtain a target value; and a compression unit, used to compress the network parameters of the second model based on the target value to obtain a target model.

[0104] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the aforementioned expansion unit includes: a first determining subunit, configured to determine N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​a parameter matrix of the first network structure in the first model, and N is a positive integer; a first updating subunit, configured to update the first model based on the N first parameter matrices to obtain a third model; a second determining subunit, configured to determine M second network structures based on the second network structure, wherein the second network structure is a network structure in the third model, and M is a positive integer; and a second updating subunit, configured to update the third model based on the M second network structures to obtain the expanded first model.

[0105] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the compression unit includes: a first calculation subunit, used to calculate the number of prunings based on the target value; a first pruning processing subunit, used to prune the N first parameter matrices in the second model according to the number of prunings to obtain a fourth model; and a second pruning processing subunit, used to prune the M second network structures in the fourth model according to a preset number to obtain a target model.

[0106] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the first model mentioned above includes a router module, and the extension unit further includes: a third determining subunit, used to determine the weight value of each of the N first parameter matrices through the router module after determining N first parameter matrices according to a preset numerical matrix; and a second calculation subunit, used to calculate the output results of the N first network structures according to the weight value of each first parameter matrix when the N first network structures corresponding to the first data input N first parameter matrices are detected.

[0107] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the aforementioned expansion unit further includes: an acquisition subunit, configured to acquire the output results of the M second network structures after determining the M second network structures based on the second network structure, and upon detecting the second data input to the M second network structures; a third calculation subunit, configured to input the output results of the M second network structures into a preset mapping matrix and calculate the loss function value of each of the M second network structures; and a third update subunit, configured to update the network parameters in the expanded first model based on the loss function value of each second network structure.

[0108] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the first calculation subunit includes: a construction module, used to acquire training samples and construct a calibration set based on the training samples; an acquisition module, used to input the calibration set into N first parameter matrices in the second network structure for each of the M second network structures in the second model, and acquire the weight value of each first parameter matrix in the N first parameter matrices; and a determination module, used to determine the number of pruning branches based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning branches is an integer greater than a preset value.

[0109] Optionally, in the data processing apparatus based on the scaling model provided in Embodiment 2 of this application, the compression unit further includes: a fourth calculation subunit, used to calculate the weight value of each first parameter matrix in the fourth model after pruning the N first parameter matrices in the second model according to the number of prunings to obtain the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; and a fourth update subunit, used to train each first parameter matrix and the weight value of each first parameter matrix in the fourth model using training samples to update each first parameter matrix in the fourth model.

[0110] The data processing device based on the scaling model includes a processor and a memory. The receiving unit 501, generating unit 502 and responding unit 503 mentioned above are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0111] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve the efficiency of deep learning models in responding to user requests.

[0112] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0113] Embodiment 3 of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a data processing method based on a scaling model.

[0114] Embodiment 4 of the present invention provides a processor for running a program, wherein the program executes a data processing method based on a scaling model during runtime.

[0115] like Figure 6As shown, Embodiment 5 of the present invention provides an electronic device, the device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: receiving first text input by a target user; inputting the first text into a target model, generating target text to respond to the first text through the target model, wherein the target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference; and displaying the target text through the target device to respond to the first text input by the target user.

[0116] When the processor executes the program, it also performs the following steps: The target model mentioned above is trained by the following steps: Pre-training the preset model using a preset dataset until the model converges to obtain the first model, wherein the preset dataset is a dataset constructed based on the user's historical question and answer text; Expanding the network parameters of the first model by copying the network parameters, and training the expanded first model to obtain the second model; Calculating the space occupied by the model based on the device information of the target device to obtain the target value; Compressing the network parameters of the second model based on the target value to obtain the target model.

[0117] When the processor executes the program, it also performs the following steps: expanding the network parameters of the first model by copying the network parameters, including: determining N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; updating the first model based on the N first parameter matrices to obtain the third model; determining M second network structures based on the second network structure, wherein the second network structure is the network structure in the third model, and M is a positive integer; updating the third model based on the M second network structures to obtain the expanded first model.

[0118] When the processor executes the program, it also performs the following steps: compressing the network parameters of the second model according to the target value to obtain the target model, including: calculating the number of pruning branches according to the target value; pruning the N first parameter matrices in the second model according to the number of pruning branches to obtain the fourth model; and pruning the M second network structures in the fourth model according to a preset number to obtain the target model.

[0119] When the processor executes the program, it also performs the following steps: The first model mentioned above includes a router module. After determining N first parameter matrices based on a preset numerical matrix, the method also includes: determining the weight value of each of the N first parameter matrices through the router module; when the first data input is detected, corresponding to the N first network structures of the N first parameter matrices, calculating the output results of the N first network structures based on the weight value of each first parameter matrix.

[0120] When the processor executes the program, it also performs the following steps: After determining M second network structures based on the second network structure, the above method further includes: when the second data input M second network structures are detected, obtaining the output results of the M second network structures; inputting the output results of the M second network structures into a preset mapping matrix, and calculating the loss function value of each of the M second network structures; updating the network parameters in the expanded first model based on the loss function value of each second network structure.

[0121] When the processor executes the program, it also performs the following steps: calculating the number of pruning branches based on the target value, including: obtaining training samples and constructing a calibration set based on the training samples; for each of the M second network structures in the second model, inputting the calibration set into the N first parameter matrices in the second network structure, and obtaining the weight value of each first parameter matrix in the N first parameter matrices; determining the number of pruning branches based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning branches is an integer greater than a preset value.

[0122] When the processor executes the program, it also performs the following steps: After pruning the N first parameter matrices in the second model according to the number of prunings to obtain the fourth model, the above method further includes: calculating the weight value of each first parameter matrix in the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; training each first parameter matrix and the weight value of each first parameter matrix in the fourth model using training samples to update each first parameter matrix in the fourth model.

[0123] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0124] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program with the following initialization steps: receiving first text input by a target user; inputting the first text into a target model; generating target text to respond to the first text through the target model, wherein the target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference; and displaying the target text through the target device in response to the first text input by the target user.

[0125] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: The target model described above is trained by the following steps: pre-training a preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question-and-answer text; expanding the network parameters of the first model by copying the network parameters, and training the expanded first model to obtain a second model; calculating the space occupied by the model based on the device information of the target device to obtain the target value; compressing the network parameters of the second model based on the target value to obtain the target model.

[0126] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: expanding the network parameters of the first model by copying the network parameters, including: determining N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; updating the first model based on the N first parameter matrices to obtain a third model; determining M second network structures based on the second network structure, wherein the second network structure is the network structure in the third model, and M is a positive integer; updating the third model based on the M second network structures to obtain the expanded first model.

[0127] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: compressing the network parameters of the second model according to the target value to obtain the target model, including: calculating the number of pruning branches according to the target value; pruning the N first parameter matrices in the second model according to the number of pruning branches to obtain the fourth model; and pruning the M second network structures in the fourth model according to a preset number to obtain the target model.

[0128] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: the first model mentioned above includes a router module, and after determining N first parameter matrices based on a preset numerical matrix, the method further includes: determining the weight value of each of the N first parameter matrices through the router module; and when the first data input corresponding to the N first parameter matrices is detected, calculating the output results of the N first network structures based on the weight value of each first parameter matrix.

[0129] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: after determining M second network structures based on the second network structure, the above method further includes: when the second data input M second network structures are detected, obtaining the output results of the M second network structures; inputting the output results of the M second network structures into a preset mapping matrix, and calculating the loss function value of each of the M second network structures; updating the network parameters in the expanded first model based on the loss function value of each second network structure.

[0130] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: calculating the number of pruning branches based on the target value, including: obtaining training samples and constructing a calibration set based on the training samples; for each of the M second network structures in the second model, inputting the calibration set into the N first parameter matrices in the second network structure, and obtaining the weight value of each first parameter matrix in the N first parameter matrices; determining the number of pruning branches based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning branches is an integer greater than a preset value.

[0131] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: after pruning the N first parameter matrices in the second model according to the number of prunings to obtain the fourth model, the above method further includes: calculating the weight value of each first parameter matrix in the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; training each first parameter matrix and the weight value of each first parameter matrix in the fourth model using training samples to update each first parameter matrix in the fourth model.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0137] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0138] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data processing method based on a scaling model, characterized in that, include: Receive the first text input by the target user; The first text is input into the target model, and the target model generates target text to respond to the first text. The target model is a model that is pre-trained during model training, the network parameters of the model are expanded after pre-training, and the network parameters of the model are pruned during model inference. The target text is displayed on the target device in response to the first text input by the target user; The target model is trained using the following steps: A preset model is pre-trained using a preset dataset until the model converges to obtain the first model. The preset dataset is a dataset constructed based on the user's historical question and answer text. The network parameters of the first model are expanded by copying the network parameters, and the expanded first model is trained using the preset dataset to obtain the second model; The space occupied by the model is calculated based on the equipment information of the target device to obtain the target value; The network parameters of the second model are compressed based on the target value to obtain the target model; The network parameters of the first model are expanded by copying the network parameters, including: N first parameter matrices are determined based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; The first model is updated based on the N first parameter matrices to obtain the third model; Based on the second network structure, M second network structures are determined, where the second network structure is the network structure in the third model, and M is a positive integer; The third model is updated based on the M second network structures to obtain the extended first model; The network parameters of the second model are compressed based on the target value to obtain the target model, including: Calculate the number of pruning branches based on the target value; Based on the specified pruning amount, the N first parameter matrices in the second model are pruned to obtain the fourth model; The target model is obtained by pruning the M second network structures in the fourth model according to a preset number.

2. The method according to claim 1, characterized in that, The first model includes a router module. After determining N first parameter matrices based on a preset numerical matrix, the method further includes: The router module determines the weight value of each of the N first parameter matrices. When the first data input is detected, corresponding to the N first network structures of the N first parameter matrices, the output results of the N first network structures are calculated based on the weight values ​​of each first parameter matrix.

3. The method according to claim 1, characterized in that, After determining M second network structures based on the second network structure, the method further includes: When the second data input to the M second network structures is detected, the output results of the M second network structures are obtained; The output results of the M second network structures are all input into a preset mapping matrix, and the loss function value of each of the M second network structures is calculated. The network parameters in the expanded first model are updated based on the loss function value of each second network structure.

4. The method according to claim 1, characterized in that, The number of pruning branches is calculated based on the target value, including: Obtain training samples and construct a calibration set based on the training samples; For each of the M second network structures in the second model, the calibration set is input into the N first parameter matrices in the second network structure, and the weight value of each first parameter matrix in the N first parameter matrices is obtained; The number of pruning branches is determined based on the weight value of each first parameter matrix in each second network structure, wherein the number of pruning branches is an integer greater than a preset value.

5. The method according to claim 1, characterized in that, After pruning the N first parameter matrices in the second model according to the pruning amount to obtain the fourth model, the method further includes: Calculate the weight value of each first parameter matrix in the fourth model, wherein the sum of the weight values ​​of each first parameter matrix is ​​a preset value; The training samples are used to train each first parameter matrix and the weight value of each first parameter matrix in the fourth model, so as to update each first parameter matrix in the fourth model.

6. A data processing device based on a scaling model, characterized in that, include: The receiving unit is used to receive the first text input by the target user; A generation unit is used to input the first text into a target model and generate target text for responding to the first text through the target model. The target model is a model that is pre-trained during model training, has its network parameters expanded after pre-training, and has its network parameters pruned during model inference. A response unit is configured to display the target text on the target device in response to the first text input by the target user; The training unit is used to pre-train a preset model using a preset dataset until the model converges to obtain a first model, wherein the preset dataset is a dataset constructed based on the user's historical question-and-answer text; the expansion unit is used to expand the network parameters of the first model by copying the network parameters, and train the expanded first model using the preset dataset to obtain a second model; the calculation unit is used to calculate the space occupied by the model based on the device information of the target device to obtain a target value; the compression unit is used to compress the network parameters of the second model based on the target value to obtain the target model; The expansion unit includes: a first determining subunit, used to determine N first parameter matrices based on a preset numerical matrix, wherein the preset numerical matrix is ​​the parameter matrix of the first network structure in the first model, and N is a positive integer; a first updating subunit, used to update the first model based on the N first parameter matrices to obtain a third model; a second determining subunit, used to determine M second network structures based on the second network structure, wherein the second network structure is the network structure in the third model, and M is a positive integer; and a second updating subunit, used to update the third model based on the M second network structures to obtain the expanded first model. The compression unit includes: a first calculation subunit for calculating the number of pruning branches based on the target value; a first pruning processing subunit for pruning the N first parameter matrices in the second model based on the number of pruning branches to obtain a fourth model; and a second pruning processing subunit for pruning the M second network structures in the fourth model based on a preset number to obtain the target model.

7. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data processing method based on the scaling model as described in any one of claims 1 to 5.

Images