Model training method and device, computer equipment, readable storage medium and program product
By dividing the sample data set into multiple data subsets and independently updating the parameters of each expert network, the problem of high consumption of training resources of hybrid expert models is solved, and the training efficiency and model adaptability are improved.
Patent Information
- Application Number
- CN202411790381.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-18
AI Technical Summary
The existing hybrid expert model consumes a lot of resources during training, resulting in inefficient training.
By dividing the sample dataset into multiple data subsets, each expert network focuses on processing data subsets of different features and independently updates the parameters of each expert network during training.
This method reduces dependence on the full amount of data, reduces the computing resource consumption during training, improves the processing efficiency of the model on large-scale data sets, and reduces training time and cost.
Smart Images
Figure CN119646514A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] With the development of artificial intelligence technology, large models have emerged. Large models are machine learning models with a large number of parameters and complex structures. They can process massive amounts of data and complete various complex tasks (for example, natural language processing, computer vision, speech recognition, etc.). Therefore, they have been increasingly widely used in business scenarios such as intelligent question answering.
[0003] The architecture of large models is crucial to improving model performance. Mixed Expert Models (MoE) is a large model based on the Transformer architecture. By integrating multiple expert models / layers within the Transformer block, input samples (such as text fragments or image blocks) are assigned to one or more experts (independent models or networks) for processing, rather than all experts processing each sample, thereby improving the model's ability to handle complex tasks. However, processors consume more training resources to train existing mixed expert models based on large-scale datasets. Summary of the invention
[0004] Based on this, it is necessary to provide a model training method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the training resources consumed by the model and improve the efficiency of model reasoning in response to the above-mentioned technical problems.
[0005] In a first aspect, the present application provides a model training method, comprising:
[0006] Acquire a sample data set and label data corresponding to the sample data contained in the sample data set, and divide the sample data set into a plurality of sample data subsets; the sample data contained in each of the sample data subsets has different data features in at least one dimension;
[0007] Calling the hybrid expert model to be trained to process the multiple sample data subsets to obtain a first processing result; the hybrid expert model includes multiple expert networks, each of which has a different structure and is used to process different sample data subsets respectively;
[0008] Based on the gap between the first processing result and the label data, the hybrid expert model is updated.
[0009] In one embodiment, updating the hybrid expert model based on the gap between the first processing result and the label data includes:
[0010] A low-rank matrix is introduced to represent the parameters of each expert network in the hybrid expert model to be trained; at the initial moment, the low-rank matrix corresponding to each of the expert networks is the same;
[0011] Based on the gap between the first processing result and the label data, the low-rank matrix is updated.
[0012] In one embodiment, the hybrid expert model includes a gating network, and the hybrid expert model is updated based on the gap between the first processing result and the label data, and then includes:
[0013] The hybrid expert model is called to process the sample data set to obtain a second processing result; the gating network in the hybrid expert model is used to assign the sample data to the corresponding expert network for processing according to the data characteristics of the sample data;
[0014] Based on the gap between the second processing result and the label data, the hybrid expert model is updated.
[0015] In one embodiment, after updating the hybrid expert model based on the gap between the first processing result and the label data, the method further comprises:
[0016] The hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model.
[0017] In one embodiment, after the hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model, the method further includes:
[0018] Acquire a verification data set, and call the optimized hybrid expert model to process the verification data set to obtain a third processing result;
[0019] When the third processing result does not satisfy the preset verification pass condition, the parameters of each expert network in the hybrid expert model are adjusted.
[0020] In one embodiment, after the hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model, the method further includes:
[0021] Acquire a test data set, and call the optimized hybrid expert model to process the test data set to obtain a fourth processing result;
[0022] Based on the fourth processing result, generating a performance index value of the hybrid expert model;
[0023] When the performance indicator value falls within a preset range, it is determined that the hybrid expert model test has passed.
[0024] In a second aspect, the present application also provides a model training device, comprising:
[0025] A data acquisition module, used to acquire a sample data set and label data corresponding to the sample data contained in the sample data set, and divide the sample data set into a plurality of sample data subsets; the sample data contained in each of the sample data subsets has different data features in at least one dimension;
[0026] A data processing module, used for invoking the hybrid expert model to be trained to process the plurality of sample data subsets to obtain a first processing result; the hybrid expert model includes a plurality of expert networks, each of which has a different structure and is used for processing different sample data subsets respectively;
[0027] A model updating module is used to update the hybrid expert model based on the gap between the first processing result and the label data.
[0028] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above-described methods when executing the computer program.
[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the methods described above when executed by a processor.
[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the methods described above when executed by a processor.
[0031] The above-mentioned model training method, apparatus, computer equipment, computer-readable storage medium and computer program product, by assigning different data subsets to each expert network, enable the model to focus on specific features and data distribution, thereby improving the adaptability and generalization ability to unknown data; by independently training each expert network, the dependence on the full amount of data can be reduced, and the consumption of computing resources during training can be reduced, making the model more efficient when processing large-scale data sets, reducing training time and cost. Furthermore, by combining LoRA to fine-tune the hybrid expert network, it is possible to adapt to new tasks and the needs of larger-scale training data by adjusting fewer parameters, effectively reducing the parameter complexity of the hybrid expert model, while improving the efficiency of training and reasoning. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 A schematic diagram of a flow chart of a model training method in one embodiment;
[0034] Figure 2 A schematic diagram of the structure of a hybrid expert model in one embodiment;
[0035] Figure 3 is a flow chart of step S106 in one embodiment;
[0036] Figure 4 A schematic diagram of a flow chart of a model training method in another embodiment;
[0037] Figure 5 A schematic diagram of a flow chart of a model training method in yet another embodiment;
[0038] Figure 6 is a structural block diagram of a model training device in one embodiment;
[0039] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0041] In one embodiment, Figure 1 As shown, a model training method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0042] Step S102, obtaining a sample data set and label data corresponding to the sample data contained in the sample data set, and dividing the sample data set into a plurality of sample data subsets; the sample data contained in each sample data subset has different data features in at least one dimension.
[0043] Exemplarily, the acquired original sample data can be cleaned and preprocessed (including: processing missing values, standardization / normalization and data enhancement, etc.), and added to the sample data set; a suitable clustering algorithm (for example, K-means, hierarchical clustering, etc.) is selected to divide the sample data contained in the sample data set into multiple clusters; according to the clustering results, the sample data set is divided into sample data subsets consistent with the number of expert networks, and it is ensured that each sample data subset can represent different data characteristics or distributions. Among them, the label data corresponding to the sample data can be used for supervised learning of the model, and for each sample data subset, the corresponding label data can also be divided in the same way to ensure that the label data of each sample data subset corresponds to the sample data.
[0044] Step S104, calling the hybrid expert model to be trained to process multiple sample data subsets to obtain a first processing result; the hybrid expert model includes multiple expert networks, each expert network has a different structure and is used to process different sample data subsets.
[0045] Exemplarily, the processor may input multiple sample data subsets into the called hybrid expert model to be trained. Multiple network architectures may be selected according to the data characteristics of the sample data to respectively construct multiple expert networks included in the hybrid expert model. For example, different numbers of layers of convolutional neural networks (CNN) or recurrent neural networks (RNN) may be used. Each expert network may have different numbers of layers, activation functions, and hyperparameters (including: learning rate, batch size, number of iterations, etc.). In a possible implementation, please refer to Figure 2 , Figure 2 FIG. 1 is a schematic diagram of the structure of a hybrid expert model in an embodiment, wherein the dotted line is used to represent training freeze, and the line of sight is used to represent training non-freeze.
[0046] Step S106: updating the hybrid expert model based on the gap between the first processing result and the label data.
[0047] For example, a loss function can be used to calculate the gap between the first processing result and the corresponding label data; a back propagation algorithm is used to update the weights of the hybrid expert model; and the above steps are repeated until the model converges or a preset number of training rounds is reached.
[0048] In the above model training method, by assigning different data subsets to each expert network, the model can focus on specific features and data distribution, thereby improving its adaptability and generalization ability to unknown data; by independently training each expert network, the dependence on the full amount of data can be reduced, and the consumption of computing resources during training can be reduced, making the model more efficient when processing large-scale data sets, reducing training time and cost.
[0049] In an exemplary embodiment, Figure 3 As shown, the above step S106 may include:
[0050] Step S1061, introducing a low-rank matrix to represent the parameters of each expert network in the hybrid expert model to be trained; at the initial moment, the low-rank matrix corresponding to each expert network is the same.
[0051] Exemplarily, the parameters of the hybrid expert model can be fine-tuned based on LoRA (Low-Rank Adaptation of Large Language Models). Specifically, when training a specific expert network, other expert networks can be in a frozen state, that is, they do not participate in the training.
[0052] Step S1062: Update the low-rank matrix based on the gap between the first processing result and the label data.
[0053] For example, a low-rank expert network class can be defined, low-rank matrices A and B can be created using TensorFlow, and the product of A and B can be calculated in the forward propagation to obtain the final weight matrix of the expert network. Then multiple expert networks can be instantiated and integrated into a hybrid expert model. The gap between the output of each expert network and the label data can be compared to calculate the loss value; the gradient of the low-rank matrices A and B can be calculated using the back-propagation algorithm, and the low-rank matrix parameters can be updated through the optimizer (for example, Adam, etc.).
[0054] In this embodiment, by combining LoRA fine-tuning the hybrid expert network, it is possible to adapt to new tasks and meet the needs of larger-scale training data by adjusting fewer parameters, effectively reducing the parameter complexity of the hybrid expert model while improving the efficiency of training and reasoning.
[0055] In an exemplary embodiment, Figure 4 As shown, the hybrid expert model may include a gating network, and the above model training method may further include:
[0056] Step S1071, calling the hybrid expert model to process the sample data set to obtain a second processing result; the gating network in the hybrid expert model is used to assign the sample data to the corresponding expert network for processing according to the data characteristics of the sample data.
[0057] Step S1072: updating the hybrid expert model based on the gap between the second processing result and the label data.
[0058] The gating network may be a lightweight model (e.g., a fully connected layer, a convolutional neural network, etc.). The output of the gating network may include the activation probability of each expert network, which is used to characterize the weight of the sample data passing through each expert network. The second processing result may be obtained by weighted integration of the outputs of multiple expert networks through the gating network.
[0059] Exemplarily, a loss value may be calculated based on the difference between the second processing result and the label data (for example, a cross entropy loss or a mean square error function may be used to calculate the loss); a back propagation algorithm is used to calculate the gradients of the gating network and the expert network in the hybrid expert model; and the parameters of the gating network and the activated expert network are updated. In a possible implementation, the expert network in the hybrid expert model may be selectively updated (for example, only the expert network related to the sample data is trained).
[0060] In this embodiment, by introducing the gating network, the hybrid expert model can dynamically select and activate the appropriate expert network for processing according to the characteristics of the sample data, thereby optimizing the computational efficiency and improving the adaptability of the hybrid expert model.
[0061] In an exemplary embodiment, Figure 5 As shown, the above model training method may also include:
[0062] Step S1091, training the hybrid expert model in an end-to-end manner to obtain an optimized hybrid expert model.
[0063] For example, multiple expert networks and a gating network can be integrated into an end-to-end training framework. A sample data set is input into the training framework, and forward propagation is performed through the gating network and the expert network to obtain the model prediction output; an appropriate loss function is used to calculate the loss with the label data; the gradient of the model is calculated using the back-propagation algorithm, and the model parameters are updated through the optimizer.
[0064] In this embodiment, all parts of the hybrid expert model are integrated into a unified framework through end-to-end training, and the entire hybrid expert model is optimized at one time, so that the network can learn more effective feature representation as a whole, thereby improving the generalization ability of the hybrid expert model.
[0065] For further information, please refer to Figure 5 , the above model training method may also include:
[0066] Step S1092, obtaining a verification data set, calling the optimized hybrid expert model to process the verification data set, and obtaining a third processing result.
[0067] Step S1093: if the third processing result does not meet the preset verification pass condition, adjust the parameters of each expert network in the hybrid expert model.
[0068] Exemplarily, after the training is completed, an independent validation data set (including data different from the training data of the hybrid expert model) can be prepared; the optimized hybrid expert model is called to process the validation data set to obtain a prediction result, which is the third processing result; if the third processing result does not meet the preset validation pass conditions (for example, loss value threshold, accuracy, etc.), the poorly performing expert network is fine-tuned or the parameters of all experts and gating networks are updated at one time.
[0069] In this embodiment, the hybrid expert model is verified by verifying the data set, and the deviation of the model is timely identified and corrected, which can ensure that the hybrid expert model can adapt to different data and prevent overfitting.
[0070] For further information, please refer to Figure 5 , the above model training method may also include:
[0071] Step S1094, obtaining a test data set, calling the optimized hybrid expert model to process the test data set, and obtaining a fourth processing result.
[0072] Step S1095: Generate a performance indicator value of the hybrid expert model based on the fourth processing result.
[0073] Step S1096: When the performance indicator value is within the preset range, it is determined that the hybrid expert model test has passed.
[0074] Exemplarily, part of the sample data in the sample data set can be used as a test data set, or a test data set that is independent of the training data set and has a similar distribution can be obtained, the optimized hybrid expert model can be loaded into the computing environment, and it can be ensured that the parameters of the hybrid expert model have been correctly updated; the test data set is input into the hybrid expert model, and a forward propagation calculation is performed to obtain a fourth processing result; according to the specific application scenario and evaluation requirements of the hybrid expert model, appropriate performance indicators (for example, accuracy, precision, recall rate, F1 score, etc.) are selected for calculation; whether the hybrid expert model passes the test is determined based on the performance indicator value; if the performance indicator is within the preset range, it can be indicated that the hybrid expert model has good performance and can be deployed; if the performance indicator is within the preset range, it can be indicated that the hybrid expert model needs further adjustment.
[0075] In this embodiment, by calling the optimized hybrid expert model to process the test data set, the effectiveness and reliability of the hybrid expert model in actual application scenarios can be ensured.
[0076] In a possible implementation, the hybrid expert model can be used for intelligent question answering. For example, the question information input by the user (which may include: keywords of news events) can be received; the question information is input into a pre-trained question answering model to obtain reply information for the question information (which may include: description text of the news event); the question answering model is trained by the model training method in the above-mentioned method embodiments.
[0077] In summary, in the above model training method, by assigning different data subsets to each expert network, the model can focus on specific features and data distribution, improving the adaptability and generalization ability to unknown data; by independently training each expert network, the dependence on the full amount of data can be reduced, and the consumption of computing resources during training can be reduced, making the model more efficient when processing large-scale data sets, reducing training time and cost. Furthermore, by combining LoRA to fine-tune the hybrid expert network, it is possible to adapt to new tasks and meet the needs of larger-scale training data with only a few parameters adjusted, effectively reducing the parameter complexity of the hybrid expert model, while improving the efficiency of training and reasoning.
[0078] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0079] Based on the same inventive concept, the embodiment of the present application also provides a model training device for implementing the model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model training device embodiments provided below can refer to the limitations on the model training method above, and will not be repeated here.
[0080] In an exemplary embodiment, Figure 6As shown, a model training device 300 is provided, comprising: a data acquisition module 301, a data processing module 302 and a model updating module 303, wherein:
[0081] The data acquisition module 301 is used to acquire a sample data set and label data corresponding to the sample data contained in the sample data set, and divide the sample data set into multiple sample data subsets; the sample data contained in each sample data subset has different data features in at least one dimension.
[0082] The data processing module 302 is used to call the hybrid expert model to be trained to process the multiple sample data subsets to obtain a first processing result; the hybrid expert model includes multiple expert networks, each of which has a different structure and is used to process different sample data subsets.
[0083] The model updating module 303 is used to update the hybrid expert model based on the gap between the first processing result and the label data.
[0084] In an exemplary embodiment, the model updating module 303 is further used to:
[0085] A low-rank matrix is introduced to represent the parameters of each expert network in the hybrid expert model to be trained; at the initial moment, the low-rank matrix corresponding to each expert network is the same;
[0086] Based on the gap between the first processing result and the label data, the low-rank matrix is updated.
[0087] In an exemplary embodiment, the hybrid expert model includes a gating network, and the model updating module 303 is further used for:
[0088] The hybrid expert model is called to process the sample data set to obtain a second processing result; the gating network in the hybrid expert model is used to assign the sample data to the corresponding expert network for processing according to the data characteristics of the sample data;
[0089] Based on the gap between the second processing result and the label data, the hybrid expert model is updated.
[0090] In an exemplary embodiment, the model updating module 303 is further used to:
[0091] The hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model.
[0092] In an exemplary embodiment, the model training device 300 further includes:
[0093] The model verification module is used to obtain a verification data set, call the optimized hybrid expert model to process the verification data set, and obtain a third processing result;
[0094] When the third processing result does not satisfy the preset verification pass condition, the parameters of each expert network in the hybrid expert model are adjusted.
[0095] In an exemplary embodiment, the model training device 300 further includes:
[0096] A model testing module, used to obtain a test data set, call the optimized hybrid expert model to process the test data set, and obtain a fourth processing result;
[0097] Based on the fourth processing result, generating a performance indicator value of the hybrid expert model;
[0098] When the performance indicator value falls within the preset range, it is determined that the hybrid expert model test has passed.
[0099] Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0100] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sample data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model training method is implemented.
[0101] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0102] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0103] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0104] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0106] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0108] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A model training method, characterized in that: The method comprises: Acquire a sample data set and label data corresponding to the sample data contained in the sample data set, and divide the sample data set into a plurality of sample data subsets; the sample data contained in each of the sample data subsets has different data features in at least one dimension; Calling the hybrid expert model to be trained to process the multiple sample data subsets to obtain a first processing result; the hybrid expert model includes multiple expert networks, each of which has a different structure and is used to process different sample data subsets respectively; Based on the gap between the first processing result and the label data, the hybrid expert model is updated.
2. The method according to claim 1, characterized in that: The updating of the hybrid expert model based on the gap between the first processing result and the label data comprises: A low-rank matrix is introduced to represent the parameters of each expert network in the hybrid expert model to be trained; at the initial moment, the low-rank matrix corresponding to each of the expert networks is the same; Based on the gap between the first processing result and the label data, the low-rank matrix is updated.
3. The method according to claim 1, characterized in that The hybrid expert model includes a gating network, and the hybrid expert model is updated based on the gap between the first processing result and the label data, and then includes: The hybrid expert model is called to process the sample data set to obtain a second processing result; the gating network in the hybrid expert model is used to assign the sample data to the corresponding expert network for processing according to the data characteristics of the sample data; Based on the gap between the second processing result and the label data, the hybrid expert model is updated.
4. The method according to claim 1, characterized in that: After updating the hybrid expert model based on the gap between the first processing result and the label data, the method further comprises: The hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model.
5. The method according to claim 4, characterized in that After the hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model, the method further includes: Acquire a verification data set, and call the optimized hybrid expert model to process the verification data set to obtain a third processing result; When the third processing result does not satisfy the preset verification pass condition, the parameters of each expert network in the hybrid expert model are adjusted.
6. The method according to claim 5, characterized in that After the hybrid expert model is trained in an end-to-end manner to obtain an optimized hybrid expert model, the method further includes: Acquire a test data set, and call the optimized hybrid expert model to process the test data set to obtain a fourth processing result; Based on the fourth processing result, generating a performance index value of the hybrid expert model; When the performance indicator value falls within a preset range, it is determined that the hybrid expert model test has passed.
7. A model training device, characterized in that: The device comprises: A data acquisition module, used to acquire a sample data set and label data corresponding to the sample data contained in the sample data set, and divide the sample data set into a plurality of sample data subsets; the sample data contained in each of the sample data subsets has different data features in at least one dimension; A data processing module, used for invoking the hybrid expert model to be trained to process the plurality of sample data subsets to obtain a first processing result; the hybrid expert model includes a plurality of expert networks, each of which has a different structure and is used for processing different sample data subsets respectively; A model updating module is used to update the hybrid expert model based on the gap between the first processing result and the label data.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.