Data processing method and device, storage medium and computer program product thereof
By dimensionality reduction and clustering processing of expert matrices in artificial intelligence models, the problem of low data processing efficiency caused by excessive model parameters is solved, and more efficient data processing and computing resource optimization is achieved.
Patent Information
- Application Number
- CN202411583144.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-02-18
AI Technical Summary
The existing artificial intelligence model is less efficient in data processing, mainly due to the similar functions of some expert nodes, resulting in excessive parameters.
By obtaining the parameter matrix of the target network layer of the artificial intelligence model, dimensionality reduction processing expert matrix, clustering and merging, the reduced matrix is obtained, and a more efficient artificial intelligence model is built.
The amount of parameters of the entire artificial intelligence model is reduced, the processing efficiency of target data is improved, and the computing cost is reduced, while maintaining or improving the identification capabilities of expert nodes.
Smart Images
Figure CN119474924A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology and more particularly to a data processing method, device, storage medium and computer program product thereof. Background Art
[0002] With the continuous development of artificial intelligence technology, the application scope of artificial intelligence technology in people's lives is becoming more and more extensive. Current artificial intelligence technology is usually applied to artificial intelligence models, and artificial intelligence models usually contain a large number of expert nodes. Each of the above expert nodes can process different data, and the functions of some experts may be similar, which leads to a large number of parameters in the artificial intelligence model, which in turn leads to low data processing efficiency of the artificial intelligence model. Summary of the invention
[0003] The embodiments of the present application provide a data processing method, device, storage medium and computer program product thereof to solve the problem of low data processing efficiency of artificial intelligence models.
[0004] To solve the above problems, this application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides a data processing method, including:
[0006] Obtain a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, wherein the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds to the expert node one-to-one, and the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold;
[0007] Clustering and merging the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N;
[0008] The target data corresponding to the task to be processed is processed using an artificial intelligence model of the target network layer including the M second matrices.
[0009] In a second aspect, an embodiment of the present application provides a data processing device, including:
[0010] An acquisition module is used to acquire a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, wherein the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds to the expert node one-to-one, and the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold;
[0011] A clustering and merging module, used for clustering and merging the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N;
[0012] A processing module is used to process target data corresponding to the task to be processed using an artificial intelligence model of a target network layer including the M second matrices.
[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the method described in the first aspect above.
[0014] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.
[0015] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the method described in the first aspect above.
[0016] In an embodiment of the present application, since the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold, that is, the functions of at least some of the N expert nodes are similar, the N first matrices corresponding to the N expert nodes can be clustered and merged, thereby reducing the number of parameters of the entire artificial intelligence model, and the artificial intelligence model with reduced parameters is used to process the target data corresponding to the task to be processed, thereby improving the processing efficiency of the target data. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0018] Figure 1 It is one of the flow charts of the data processing method provided in the embodiment of the present application;
[0019] Figure 2 This is the second flowchart of the data processing method provided in the embodiment of the present application;
[0020] Figure 3 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;
[0021] Figure 4It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations in which A, B and C all exist.
[0024] See also Figure 1 , Figure 1 Schematic diagram of the data processing method provided in the embodiment of the present application. Figure 1 As shown, the data processing method may include the following steps:
[0025] Step 101: obtain a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, where the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds one-to-one to the expert node.
[0026] The specific type of the artificial intelligence model is not limited here. Optionally, the artificial intelligence model can be a large-scale language model (LLM), and the architecture of the large-scale language model can be a mixture of experts (Mixture of The large-scale language model of the MoE architecture can also be referred to as a MoE-LLM model or a MoE model, and the MoE-LLM model usually includes multiple expert nodes, each of which can be used to process tasks or data.
[0027] It should be noted that the functions of at least some of the N expert nodes corresponding to the N first matrices are similar, that is, it can be understood that the similarity of the data processed by at least some of the above N expert nodes is greater than the preset threshold, so it can be determined that at least some of the above N expert nodes are prone to redundant processing processes when processing data, affecting the overall processing efficiency of the artificial intelligence model and causing a waste of computing resources.
[0028] Among them, the parameter matrix can also be called matrix parameters. For example, when the artificial intelligence model is a MoE-LLM model, the target network layer in the MoE-LLM model can be a multilayer perceptron (MLP) layer, and the parameter matrix can be a gate matrix corresponding to an expert node, and the gate matrix can also be called an Expert_gate parameter matrix.
[0029] Step 102: cluster and merge the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N.
[0030] Among them, the N first matrices can be divided into M types of target matrices, and each type of target matrix can include at least one first matrix. The at least one first matrix included in each type of target matrix can be merged to obtain a second matrix. In this way, by merging M types of target matrices, M second matrices can be obtained.
[0031] The above can also be understood as: at least one first matrix included in each type of target matrix belongs to the same type of matrix, that is, the similarity of data processed by the expert nodes corresponding to at least one first matrix included in each type of target matrix is greater than a preset threshold, that is to say: the functions of the expert nodes corresponding to at least one first matrix included in each type of target matrix are similar.
[0032] Step 103: Use the artificial intelligence model of the target network layer including the M second matrices to process the target data corresponding to the task to be processed.
[0033] Among them, the artificial intelligence model of the target network layer including M second matrices can be understood as the artificial intelligence model after structural adjustment, and specifically, the N first matrices in the artificial intelligence model before the structural adjustment can be replaced by M second matrices to obtain the artificial intelligence model after structural adjustment.
[0034] It should be noted that since N first matrices are replaced by M second matrices, the network structure corresponding to the artificial intelligence model also needs to be adaptively changed.
[0035] In the embodiment of the present application, through steps 101 to 103, since the similarity of the data processed by at least some of the N expert nodes is greater than the preset threshold, that is, the functions of at least some of the N expert nodes are similar, the N first matrices corresponding to the N expert nodes respectively can be clustered and merged, thereby reducing the number of parameters of the entire artificial intelligence model, and the artificial intelligence model with reduced parameters is used to process the target data corresponding to the task to be processed, thereby improving the processing efficiency of the target data and reducing the computing cost.
[0036] It should be noted that since the functions of at least some of the above-mentioned N expert nodes are similar, the N first matrices corresponding to the N expert nodes are clustered and merged, so that the recognition ability of the expert nodes of the artificial intelligence model can be maintained unchanged while reducing the number of parameters of the entire artificial intelligence model, and can even improve the recognition ability of the expert nodes of the artificial intelligence model.
[0037] As an optional implementation manner, the N expert matrices in the parameter matrix are subjected to dimensionality reduction processing to obtain N first matrices, including:
[0038] Performing principal component analysis on the parameter matrix to obtain N expert matrices;
[0039] A distributed domain embedding algorithm is used to perform dimensionality reduction processing on the N expert matrices to obtain the N first matrices.
[0040] The process of principal component analysis can also be understood as a process of principal component dimensionality reduction. For example, the principal component analysis (PCA) algorithm can be used to reduce the dimension of the parameter matrix to obtain N expert matrices.
[0041] The specific type of the distributed domain embedding algorithm is not limited here. For example, the distributed domain embedding algorithm can adopt the T-Distribution Stochastic Neighbour Embedding (t-SNE) algorithm.
[0042] It should be noted that the N expert matrices may be three-dimensional matrices, and the N first matrices may be two-dimensional matrices, that is, the dimensions of the N expert matrices may be greater than the dimensions of the N first matrices.
[0043] In the implementation mode of the present application, the principal component analysis can be performed on the parameter matrix to obtain N expert matrices; then the distributed domain embedding algorithm is used to reduce the dimension of the N expert matrices to obtain N first matrices. In this way, the principal component analysis and the distributed domain embedding algorithm are used for dimensionality reduction, so as to further reduce the number of parameters of the entire artificial intelligence model, thereby further improving the processing efficiency of the target data.
[0044] As an optional implementation manner, performing principal component analysis on the parameter matrix to obtain N expert matrices includes:
[0045] Performing principal component analysis on the parameter matrix to obtain a target eigenvector in the parameter matrix, wherein the data amount of the target eigenvector is within a target numerical range;
[0046] The target feature vector is converted into the N expert matrices.
[0047] Among them, the specific value of the target numerical range is not limited here. For example, the target numerical range can be from 0 to 10% of the data volume of the eigenvector with the largest data volume in the parameter matrix. Because usually, the larger the data volume, the more data dimensions there are, so 10% of the data volume of the eigenvector with the largest data volume can also be understood as 10% of the data dimensions of the eigenvector with the largest data dimension.
[0048] In the implementation manner of the present application, since only the target feature vector in the parameter matrix needs to be obtained and converted into N expert matrices, compared with the method of analyzing all target feature vectors in the parameter matrix and then obtaining specific feature vectors and converting them into N expert matrices, the present implementation manner can simplify the calculation steps, reduce the amount of calculation, and thus improve the calculation efficiency.
[0049] As an optional implementation, the N expert matrices are subjected to dimensionality reduction processing by using a distributed domain embedding algorithm to obtain the N first matrices, including:
[0050] Calculating a first similarity between each target expert matrix in the N expert matrices and other expert matrices in the N expert matrices except the target expert matrix, and calculating a second similarity between each second target matrix in a preset target space and other second matrix values in the target space except the second target matrix, wherein the dimension of the matrix included in the target space is smaller than the dimension of any one of the N expert matrices;
[0051] A first matrix corresponding to each target expert matrix in the target space is determined based on the Kullback-Leibler Divergence (KL) divergence, the first similarity and the second similarity, and the first matrix is a second target matrix corresponding to each target expert matrix in the target space.
[0052] Among them, the first similarity and the second similarity can both be called conditional probability similarity, and calculating the first similarity between each target expert matrix in the N expert matrices and other expert matrices in the N expert matrices except the target expert matrix can be understood as: calculating the first similarity between each target expert matrix and other expert matrices, and the target expert matrix can refer to any one of the N expert matrices.
[0053] Similarly, calculating the second similarity between each second target matrix in the preset target space and other second matrix values in the target space except the second target matrix can be understood as: calculating the second similarity between each second target matrix and other second target matrices, and each second target matrix can refer to any matrix in the target space.
[0054] It should be noted that the above-mentioned target space can be a pre-set space, and the dimension of the matrix included in the target space is smaller than the dimension of any expert matrix among the N expert matrices. For example: if the space corresponding to the N expert matrices is a three-dimensional space, then the target space can be a two-dimensional space.
[0055] Among them, determining the first matrix corresponding to each target expert matrix in the target space based on the KL divergence, the first similarity and the second similarity can be understood as: optimizing the low-dimensional embedding by minimizing the KL divergence of the distribution of the first similarities between matrices in the space corresponding to the N expert matrices and the second similarities between matrices in the target space, and finally obtaining the second target matrix corresponding to each target expert matrix in the target space.
[0056] That is, by calculating the KL divergence of the first similarity between matrices in the space corresponding to N expert matrices and the second similarity between matrices in the target space, and taking the minimum value of the KL divergence, when the KL divergence takes the minimum value, the second target matrix corresponding to the target expert matrix in the target space can be determined as the first matrix corresponding to the target expert matrix in the target space.
[0057] In the implementation manner of the present application, since the first matrix corresponding to each target expert matrix in the target space is determined based on the KL divergence, the first similarity and the second similarity, the error of the determination result of the first matrix can be made smaller, thereby improving the accuracy of the determination result of the first matrix.
[0058] As an optional implementation manner, clustering and merging the N first matrices to obtain M second matrices includes:
[0059] Clustering the N first matrices into M types of target matrices, wherein the target matrices include at least one of the first matrices;
[0060] Calculating a mean vector of a first matrix included in the target matrix;
[0061] Calculating the distance between the first matrix included in the target matrix and the mean vector;
[0062] A weighted sum of the first matrix included in the target matrix is calculated to obtain the second matrix, where the weight of the first matrix is the distance between the first matrix and the mean vector.
[0063] The mean vector may be referred to as μ, and the specific type of the distance between the first matrix included in the target matrix and the mean vector is not limited herein. Optionally, the distance may be a Euclidean distance.
[0064] Among them, the specific implementation method of clustering N first matrices into M types of target matrices is not limited here. Optionally, a Gaussian mixture model can be used to cluster the N first matrices into M types of target matrices. In this way, the accuracy and efficiency of the clustering results can be improved.
[0065] Among them, calculating the weighted sum of the first matrices included in the target matrix to obtain the second matrix can be understood as: taking the distance between the first matrix and the mean vector as the weight of the first matrix, and then calculating the weighted average sum of all the first matrices in the target matrix to obtain the second matrix.
[0066] In the implementation manner of the present application, the efficiency and accuracy of the obtained second matrix can be further improved.
[0067] It should be noted that since the artificial intelligence model includes multiple network layers, that is, each network layer can execute the operation steps of the above-mentioned target network layer, this can further reduce the number of parameters of the artificial intelligence model, thereby further improving the processing efficiency of the target data.
[0068] As an optional implementation manner, the processing of target data corresponding to the task to be processed by using the artificial intelligence model of the target network layer including the M second matrices includes:
[0069] Using data corresponding to a target task to adjust an artificial intelligence model of a target network layer including the M second matrices, the target task and the task to be processed both belonging to the same technical field;
[0070] The target data is processed using the adjusted artificial intelligence model.
[0071] Among them, the above-mentioned target task can be called a domain task, and the similarity between the data corresponding to the above-mentioned target task and the data corresponding to the task to be processed can be greater than the preset similarity. In this way, when the target data is processed by the artificial intelligence model adjusted by the target task, the adjusted artificial intelligence model can more accurately identify the target data, thereby making the adjusted artificial intelligence model more efficient in processing the target data.
[0072] Among them, the above-mentioned process of adjusting the artificial intelligence model of the target network layer including M second matrices using the data corresponding to the target task can be referred to as the process of supervised fine-tuning (SFT) of the artificial intelligence model of the target network layer including M second matrices using the data corresponding to the target task.
[0073] In the implementation manner of the present application, when the target data is processed by using an artificial intelligence model adjusted through a target task, the adjusted artificial intelligence model can more accurately identify the target data, thereby making the adjusted artificial intelligence model more efficient in processing the target data.
[0074] In order to more fully illustrate the above embodiment, a specific embodiment is used as an example below. Figure 2 As shown, Figure 2 The MOE-LLM can be understood as the artificial intelligence model in the above embodiment, and the Expert_Gate parameter matrix can be understood as the parameter matrix of the target network layer of the artificial intelligence model, the matrix dimensionality reduction can be understood as the dimensionality reduction processing of the N expert matrices in the parameter matrix to obtain N first matrices, the matrix clustering and the matrix merging based on the clustering structure can be understood as the clustering and merging of the N first matrices to obtain M second matrices, the new model can be understood as the artificial intelligence model of the target network layer including the M second matrices, and the domain task SFT can be understood as adjusting the artificial intelligence model of the target network layer including the M second matrices using the data corresponding to the target task to obtain the adjusted artificial intelligence model, that is, the adjusted artificial model can be understood as Figure 2 The final model in .
[0075] See also Figure 3 , Figure 3 is a structural diagram of a data processing device provided in an embodiment of the present application, such as Figure 3 As shown, the data processing device 300 includes:
[0076] An acquisition module 301 is used to acquire a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, wherein the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds to the expert node one-to-one, and the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold;
[0077] A clustering and merging module 302 is used to cluster and merge the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N;
[0078] The processing module 303 is used to process the target data corresponding to the task to be processed by using the artificial intelligence model of the target network layer including the M second matrices.
[0079] As an optional implementation, the acquisition module 301 includes:
[0080] An analysis submodule, used for performing principal component analysis on the parameter matrix to obtain N expert matrices;
[0081] The dimensionality reduction processing submodule is used to perform dimensionality reduction processing on the N expert matrices by adopting a distribution domain embedding algorithm to obtain the N first matrices.
[0082] As an optional implementation, the analysis submodule includes:
[0083] An analysis unit, configured to perform principal component analysis on the parameter matrix to obtain a target eigenvector in the parameter matrix, wherein the data volume of the target eigenvector is within a target numerical range;
[0084] A conversion unit is used to convert the target feature vector into the N expert matrices.
[0085] As an optional implementation, the dimensionality reduction processing submodule includes:
[0086] A calculation unit, used to calculate a first similarity between each target expert matrix among the N expert matrices and other expert matrices among the N expert matrices except the target expert matrix, and calculate a second similarity between each second target matrix in a preset target space and other second matrix values in the target space except the second target matrix, wherein the dimension of the matrix included in the target space is smaller than the dimension of any one of the N expert matrices;
[0087] A determination unit is used to determine a first matrix corresponding to each target expert matrix in the target space based on the KL divergence, the first similarity and the second similarity, wherein the first matrix is a second target matrix corresponding to each target expert matrix in the target space.
[0088] As an optional implementation, the cluster merging module 302 includes:
[0089] A clustering submodule, used for clustering the N first matrices into M types of target matrices, wherein the target matrices include at least one of the first matrices;
[0090] A first calculation submodule, configured to calculate a mean vector of a first matrix included in the target matrix;
[0091] A second calculation submodule, used for calculating the distance between the first matrix included in the target matrix and the mean vector;
[0092] The third calculation submodule is used to calculate the weighted sum of the first matrix included in the target matrix to obtain the second matrix, where the weight of the first matrix is the distance between the first matrix and the mean vector.
[0093] As an optional implementation, the processing module 303 includes:
[0094] An adjustment submodule, configured to adjust the artificial intelligence model of the target network layer including the M second matrices using data corresponding to a target task, wherein the target task and the task to be processed belong to the same technical field;
[0095] The processing submodule is used to process the target data using the adjusted artificial intelligence model.
[0096] The data processing device 300 can implement the embodiment of the present application Figure 1 The various processes of the illustrated method embodiment achieve the same beneficial effects and will not be described again here to avoid repetition.
[0097] The present application also provides an electronic device. Figure 4 , the electronic device may include a processor 401, a memory 402, and a program 4021 stored in the memory 402 and executable on the processor 401. When the program 4021 is executed by the processor 401, Figure 1 and Figure 2 Any steps in the corresponding method embodiments and achieving the same beneficial effects will not be repeated here.
[0098] A person skilled in the art can understand that all or part of the steps of implementing the above-mentioned embodiment method can be completed by hardware related to program instructions, and the program can be stored in a readable medium. The present application also provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 Any steps in the corresponding method embodiments can achieve the same technical effect and will not be repeated here to avoid repetition.
[0099] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0100] The present application also provides a computer program product, including computer instructions, which can achieve the above-mentioned Figure 1 and Figure 2 Any steps in the corresponding method embodiments can achieve the same technical effect and will not be repeated here to avoid repetition.
[0101] The above is a preferred implementation of the embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A data processing method, characterized in that: include: Obtain a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, wherein the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds to the expert node one-to-one, and the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold; Clustering and merging the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N; The target data corresponding to the task to be processed is processed using the artificial intelligence model of the target network layer including the M second matrices.
2. The method according to claim 1, characterized in that The N expert matrices in the parameter matrix are subjected to dimensionality reduction processing to obtain N first matrices, including: Performing principal component analysis on the parameter matrix to obtain N expert matrices; A distributed domain embedding algorithm is used to perform dimensionality reduction processing on the N expert matrices to obtain the N first matrices.
3. The method according to claim 2, characterized in that The performing principal component analysis on the parameter matrix to obtain N expert matrices includes: Performing principal component analysis on the parameter matrix to obtain a target eigenvector in the parameter matrix, wherein the data amount of the target eigenvector is within a target numerical range; The target feature vector is converted into the N expert matrices.
4. The method according to claim 2, characterized in that: The method of using the distributed domain embedding algorithm to perform dimensionality reduction processing on the N expert matrices to obtain the N first matrices includes: Calculating a first similarity between each target expert matrix in the N expert matrices and other expert matrices in the N expert matrices except the target expert matrix, and calculating a second similarity between each second target matrix in a preset target space and other second matrix values in the target space except the second target matrix, wherein the dimension of the matrix included in the target space is smaller than the dimension of any one of the N expert matrices; A first matrix corresponding to each target expert matrix in the target space is determined based on the Kulbeck-Leibler KL divergence, the first similarity and the second similarity, and the first matrix is a second target matrix corresponding to each target expert matrix in the target space.
5. The method according to claim 1, characterized in that The clustering and merging of the N first matrices to obtain M second matrices includes: Clustering the N first matrices into M types of target matrices, wherein the target matrices include at least one of the first matrices; Calculating a mean vector of a first matrix included in the target matrix; Calculating the distance between the first matrix included in the target matrix and the mean vector; A weighted sum of the first matrix included in the target matrix is calculated to obtain the second matrix, where the weight of the first matrix is the distance between the first matrix and the mean vector.
6. The method according to any one of claims 1 to 5, characterized in that The processing of target data corresponding to the task to be processed by using the artificial intelligence model of the target network layer including the M second matrices includes: Using data corresponding to a target task to adjust an artificial intelligence model of a target network layer including the M second matrices, the target task and the task to be processed both belonging to the same technical field; The target data is processed using the adjusted artificial intelligence model.
7. A data processing device, characterized in that: include: An acquisition module is used to acquire a parameter matrix of a target network layer of an artificial intelligence model, and perform dimensionality reduction processing on N expert matrices in the parameter matrix to obtain N first matrices, wherein the expert matrix is a matrix corresponding to an expert node in the artificial intelligence model, and the expert matrix corresponds to the expert node one-to-one, and the similarity of data processed by at least some of the N expert nodes is greater than a preset threshold; A clustering and merging module, used for clustering and merging the N first matrices to obtain M second matrices, where M and N are both integers greater than 1, and M is less than N; A processing module is used to process target data corresponding to the task to be processed using an artificial intelligence model of a target network layer including the M second matrices.
8. An electronic device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the data processing method as described in any one of claims 1 to 6.
9. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps in the data processing method according to any one of claims 1 to 6.