Information processing method and device, storage medium and computer program product

Abstract

The invention relates to an information processing method and device, a storage medium and a computer program product, and relates to the technical field of computers. The information processing method comprises the steps of performing clustering processing on a plurality of machine learning models according to differences among model parameters of different machine learning models to obtain a plurality of model classifications; performing fusion processing on the machine learning models belonging to the same model classification to obtain a plurality of fusion models; and processing the to-be-processed information by using the plurality of fusion models to obtain a processing result. According to the technical scheme, the computing resource consumption can be reduced, so that the information processing efficiency is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an information processing method, an information processing apparatus, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Machine learning models have demonstrated superior performance in various information processing tasks, such as natural language processing and image processing. The improved performance of machine learning models is attributed to scaling the model parameters, which also leads to higher computational costs.

[0003] In related technologies, in order to control computational costs while scaling model parameters, a hybrid model architecture has been proposed, which routes input tokens to multiple machine learning models for processing. Summary of the Invention

[0004] The inventors of this disclosure have discovered the following problems in the aforementioned related technologies: the computational resource consumption is relatively large, resulting in low information processing efficiency.

[0005] In view of this, this disclosure proposes an information processing technology solution that can reduce computing resource consumption and thus improve the efficiency of information processing.

[0006] According to some embodiments of this disclosure, an information processing method is provided, including: clustering multiple machine learning models based on the differences between model parameters of different machine learning models to obtain multiple model classifications; fusing machine learning models belonging to the same model classification to obtain multiple fused models; and using the multiple fused models to process information to be processed to obtain processing results.

[0007] In some embodiments, the model parameters include an upsampled projection matrix and a gated projection matrix. Clustering multiple machine learning models based on the differences between the model parameters of different machine learning models includes: determining the feature information of each machine learning model based on the upsampled projection matrix and the gated projection matrix of each machine learning model; and clustering multiple machine learning models based on the differences between the feature information of different machine learning models.

[0008] In some embodiments, determining the feature information of each machine learning model based on the upsampled projection matrix and the gated projection matrix of each machine learning model in a plurality of machine learning models includes: fusing the upsampled projection matrix and the gated projection matrix of each machine learning model to determine the feature information of each machine learning model.

[0009] In some embodiments, clustering multiple machine learning models based on the differences between the model parameters of different machine learning models includes: using the most frequently used model among the multiple machine learning models as the cluster center model; and performing clustering based on the differences between the model parameters of other machine learning models and the model parameters of the cluster center model.

[0010] In some embodiments, processing the information to be processed using multiple fusion models to obtain processing results includes: processing the model parameters of multiple fusion models using a dimensionality reduction matrix to obtain multiple dimensionality reduction models; and processing the information to be processed using the multiple dimensionality reduction models to obtain processing results.

[0011] In some embodiments, the dimensionality reduction matrix is ​​determined as follows: based on the difference between the first processing result of the sample information using multiple machine learning models and the second processing result of the sample information using multiple dimensionality reduction models, an objective function is constructed with the candidate dimensionality reduction matrix as a variable, where the multiple dimensionality reduction models are the results of processing multiple machine learning models using the candidate dimensionality reduction matrix; the objective function is solved with the goal of minimizing the difference between the first processing result and the second processing result to determine the dimensionality reduction matrix.

[0012] In some embodiments, the model parameters include a downsampled projection matrix, a gated projection matrix, and an upsampled projection matrix. The dimensionality reduction matrix includes a first dimensionality reduction matrix for dimensionality reduction of the downsampled projection matrix, a second dimensionality reduction matrix for dimensionality reduction of the gated projection matrix, and a third dimensionality reduction matrix for dimensionality reduction of the upsampled projection matrix. The objective function is constructed by: determining a first processing result based on the processing results of the gated projection matrix and the upsampled projection matrix on the sample information; and determining a second processing result based on the processing results of the dimensionality-reduced gated projection matrix, the processing results of the dimensionality-reduced upsampled projection matrix on the sample information, and the first dimensionality reduction matrix.

[0013] In some embodiments, the dimensionality reduction matrix is ​​determined based on the weights of each of the multiple machine learning models, and the weights of each machine learning model are determined based on the usage frequency information of each machine learning model.

[0014] In some embodiments, fusing machine learning models belonging to the same model category to obtain multiple fused models includes: determining the weight of each machine learning model based on the usage frequency information of each machine learning model among the multiple machine learning models; and fusing machine learning models belonging to the same model category according to the weights.

[0015] In some embodiments, determining the weight of each machine learning model based on the usage frequency information of each machine learning model among a plurality of machine learning models includes: determining the expected number of times each machine learning model is used; and determining the weight of each machine learning model based on the sum of the expected number of times machine learning models belonging to the same model category and the expected number of times each machine learning model is used.

[0016] In some embodiments, the information to be processed includes at least one of natural language information and image information.

[0017] According to some other embodiments of this disclosure, an information processing apparatus is provided, comprising: a clustering unit for clustering multiple machine learning models based on differences in model parameters between different machine learning models to obtain multiple model classifications; a fusion unit for fusion processing of machine learning models belonging to the same model classification to obtain multiple fused models; and a processing unit for processing information to be processed using the multiple fused models to obtain processing results.

[0018] In some embodiments, the model parameters include an upsampled projection matrix and a gated projection matrix. The clustering unit determines the feature information of each machine learning model based on the upsampled projection matrix and the gated projection matrix of each machine learning model among multiple machine learning models, and performs clustering processing on multiple machine learning models based on the differences between the feature information of different machine learning models.

[0019] In some embodiments, the clustering unit fuses the upsampled projection matrix and the gated projection matrix of each machine learning model to determine the feature information of each machine learning model.

[0020] In some embodiments, the clustering unit uses the most frequently used model among multiple machine learning models as the cluster center model, and performs clustering based on the differences between the model parameters of other machine learning models and the model parameters of the cluster center model.

[0021] In some embodiments, the processing unit uses a dimensionality reduction matrix to process the model parameters of multiple fusion models to obtain multiple dimensionality reduction models, and uses these multiple dimensionality reduction models to process the information to be processed to obtain processing results.

[0022] In some embodiments, the dimensionality reduction matrix is ​​determined as follows: the processing unit constructs an objective function with the candidate dimensionality reduction matrix as a variable based on the difference between the first processing result of the sample information using multiple machine learning models and the second processing result of the sample information using multiple dimensionality reduction models, wherein the multiple dimensionality reduction models are the results of processing multiple machine learning models using the candidate dimensionality reduction matrix; the processing unit solves the objective function with the objective of minimizing the difference between the first processing result and the second processing result to determine the dimensionality reduction matrix.

[0023] In some embodiments, the model parameters include a downsampling projection matrix, a gated projection matrix, and an upsampling projection matrix. The dimensionality reduction matrix includes a first dimensionality reduction matrix for dimensionality reduction of the downsampling projection matrix, a second dimensionality reduction matrix for dimensionality reduction of the gated projection matrix, and a third dimensionality reduction matrix for dimensionality reduction of the upsampling projection matrix. The processing unit determines a first processing result based on the processing result of the gated projection matrix on the sample information and the processing result of the upsampling projection matrix on the sample information. The processing unit determines a second processing result based on the processing result of the dimensionality-reduced gated projection matrix on the sample information, the processing result of the dimensionality-reduced upsampling projection matrix on the sample information, and the first dimensionality reduction matrix.

[0024] In some embodiments, the dimensionality reduction matrix is ​​determined based on the weights of each of the multiple machine learning models, and the weights of each machine learning model are determined based on the usage frequency information of each machine learning model.

[0025] In some embodiments, the fusion unit determines the weight of each machine learning model based on the usage frequency information of each machine learning model among multiple machine learning models, and performs fusion processing on machine learning models belonging to the same model category according to the weight.

[0026] In some embodiments, the fusion unit determines the expected number of times each machine learning model is used; and determines the weight of each machine learning model based on the sum of the expected number of times machine learning models belonging to the same model category and the expected number of times each machine learning model is used.

[0027] In some embodiments, the information to be processed includes at least one of natural language information and image information.

[0028] According to further embodiments of the present disclosure, an information processing apparatus is provided, including: a memory; and a processor coupled to the memory, the processor being configured to execute the information processing method of any of the above embodiments based on instructions stored in the memory device.

[0029] According to further embodiments of the present disclosure, a computer-readable storage medium is provided having computer instructions stored thereon that, when executed by a processor, implement the information processing method of any of the above embodiments.

[0030] According to further embodiments of this disclosure, a computer program product is also provided, including instructions that, when executed by a processor, cause the processor to perform an information processing method according to any of the foregoing embodiments.

[0031] In the above embodiments, multiple machine learning models are clustered and fused based on the differences between model parameters, which not only ensures the accuracy of information processing, but also reduces the computing resources that need to be scheduled, thereby improving the efficiency of information processing.

[0032] Other features, aspects, and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0033] Embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the drawings described below are merely illustrative of some embodiments of this disclosure and are not intended to limit the scope of this disclosure. In the drawings:

[0034] Figure 1 Flowcharts illustrating some embodiments of the information processing methods of this disclosure;

[0035] Figure 2 Schematic diagrams illustrating some embodiments of clustering and fusion of machine learning models according to this disclosure;

[0036] Figure 3 Block diagrams illustrating some embodiments of the information processing apparatus of this disclosure;

[0037] Figure 4 Block diagrams illustrating other embodiments of the information processing apparatus of this disclosure;

[0038] Figure 5 Block diagrams illustrating further embodiments of the information processing apparatus of this disclosure are shown.

[0039] It should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not necessarily drawn to actual scale. The same or similar reference numerals are used in the various drawings to denote the same or similar parts. Therefore, once an item is defined in one drawing, it may not be discussed further in subsequent drawings. Detailed Implementation

[0040] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. It should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0041] It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of components and steps set forth in these embodiments should be interpreted as merely exemplary and do not limit the scope of this disclosure.

[0042] As used in this disclosure, the term "comprising" and its variations are open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to". The term "based on" means "at least partially based on".

[0043] It should be noted that the concepts of "first," "second," etc., used in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to define the order of functions performed by these devices, modules, or units or their interdependencies. Unless otherwise specified, the concepts of "first," "second," etc., are not intended to imply that the objects described herein must be in a given temporal, spatial, rank, or any other given order.

[0044] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0045] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0046] The embodiments of this disclosure are described in detail below with reference to the accompanying drawings; however, this disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. Furthermore, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be apparent to those skilled in the art from this disclosure.

[0047] It should be understood that this disclosure does not limit how the image to be applied / processed is obtained. In some embodiments of this disclosure, it can be obtained from a storage device, such as internal memory or external storage device. In other embodiments of this disclosure, a camera component can be invoked to capture an image. It should be noted that the acquired image can be a captured image or a frame from a captured video, and is not particularly limited to these.

[0048] In the context of this disclosure, "image" can refer to any of a variety of images, such as color images, grayscale images, etc. It should be noted that the type of image is not specifically limited in the context of this specification. Furthermore, an image can be any suitable image, such as a raw image obtained by a camera device, or an image from which specific processing has been performed, such as preliminary filtering, dealiasing, color adjustment, contrast adjustment, normalization, etc. It should be noted that preprocessing operations may also include other types of preprocessing operations known in the art, which will not be described in detail here.

[0049] As mentioned earlier, routing input labels to multiple machine learning models incurs high memory overhead, leading to significant computational resource consumption. The large number of parameters in machine learning models makes deployment relatively difficult, especially under resource constraints. Model pruning can reduce computational resource consumption. For example, model pruning can be achieved by merging machine learning models.

[0050] To address the aforementioned technical problems, the technical solution disclosed herein aims to perform model pruning on machine learning models. For example, the number of machine learning models can be reduced by clustering them into several model categories and merging machine learning models from the same category.

[0051] In this way, good information processing results can be achieved without the need for time-consuming and computationally resource-intensive distillation processing of the machine learning model, thus reducing computational resource consumption. Moreover, this technical solution is orthogonal to other optimization strategies, such as distillation and quantization.

[0052] For example, the technical solution of this disclosure can be implemented through the following embodiments.

[0053] Figure 1 Flowcharts illustrating some embodiments of the information processing methods of this disclosure are shown.

[0054] like Figure 1 As shown, in step 110, multiple machine learning models are clustered based on the differences in model parameters between different machine learning models to obtain multiple model classifications.

[0055] In some embodiments, multiple machine learning models can be deployed on multiple computing nodes in a computing cluster, such as multiple GPUs (Graphics Processing Units). For example, multiple machine learning models deployed on multiple computing nodes can be retrieved by a server and clustered.

[0056] In step 120, machine learning models belonging to the same model category are fused to obtain multiple fused models.

[0057] In step 130, multiple fusion models are used to process the information to be processed in order to obtain the processing results. For example, multiple fusion models can be deployed on multiple computing nodes, and the information to be processed can be processed using multiple computing nodes.

[0058] In the above embodiments, multiple machine learning models are clustered and fused based on the differences between model parameters. This ensures the accuracy of information processing while reducing the computational resources that need to be scheduled, such as reducing the number of scheduled computing nodes and lowering the consumption of storage and computing hardware resources on the computing nodes. Therefore, it improves the efficiency of the computing cluster in processing information.

[0059] For example, the information to be processed includes at least one of natural language information, image information, audio information, and video information, and the machine learning model is a model with the ability to process the information to be processed. In this way, by clustering and fusing multiple machine learning models based on the differences between model parameters, the processing efficiency of at least one of natural language information, image information, audio information, and video information can be improved.

[0060] The following examples illustrate the method of clustering the machine learning model in step 110.

[0061] For example, E i Let E be the i-th machine learning model among N machine learning models. i It can be implemented using activation functions such as SwiGLU (Swish-Gated Linear Unit), and includes three processing matrices (i.e., model parameters): the downsampling projection matrix D. i The gating projection matrix G is used to project the input from a higher-dimensional space to a lower-dimensional space. i It is used to project the input onto the gate space and generate gate coefficients, etc.; the upsampling projection matrix U i , is used to project the input from a lower-dimensional space to a higher-dimensional space. i It can also include a nonlinear activation function σ(). iThe processing of the information X to be processed can be represented as:

[0062] E i (X)= D i (σ(G i X)⊙(U i X))

[0063] ⊙ represents the Hadamard product, and the above formula is E i The forward processing procedure.

[0064] In some embodiments, model parameters include an up-projection matrix and a gate-projection matrix. Based on the up-projection matrix and gate-projection matrix of each of the multiple machine learning models, feature information for each model is determined. Then, based on the differences in feature information between the different machine learning models, clustering is performed on the multiple machine learning models. For example, the up-projection matrix and gate-projection matrix of each machine learning model are fused to determine the feature information of each model.

[0065] For example, clustering eight machine learning models E1 to E8 yields four model classifications: (E2, E3), (E1, E6), (E5, E7), and (E4, E8).

[0066] In the above embodiments, clustering multiple machine learning models based on the differences in model parameters can accurately group machine learning models with similar forward processing procedures into the same model category. Information processing based on the clustering results ensures accuracy while reducing the computational resources required for scheduling, such as the number of scheduled computing nodes and the consumption of storage and computing hardware resources. This improves the efficiency of the computing cluster in processing information.

[0067] For example, U i and G i Concatenate them into a feature matrix, which serves as E i The feature information of each machine learning model is analyzed; the distance between these models is determined by calculating the differences between their feature matrices; and clustering is then performed based on this distance information. Besides concatenation, other fusion methods such as weighted averages can also be used to analyze U. i and G i The data is processed to obtain feature information.

[0068] In this way, U i and G iClassifying similar machine learning models into the same model category, as the basis for information processing, can reduce the impact of dimensionality reduction on the performance of machine learning models, thereby improving the efficiency of information processing.

[0069] In some embodiments, the most frequently used machine learning models are selected as cluster center models; clustering is performed based on the differences between the model parameters of other machine learning models and the model parameters of the cluster center model. For example, from N machine learning models, the M most frequently used machine learning models can be selected as cluster centers, and clustering is performed based on the distances between the other NM machine learning models and the cluster centers.

[0070] In this way, the most representative machine learning models can be clustered based on their frequency of use, which can improve the efficiency and accuracy of clustering and thus improve the effect of information processing.

[0071] The following examples illustrate, by way of example, how the machine learning models are fused in step 120.

[0072] In some embodiments, the weight of each machine learning model is determined based on the usage frequency information of each machine learning model among multiple machine learning models; and machine learning models belonging to the same model category are fused according to the weight.

[0073] By weighting the machine learning model according to its usage frequency, the influence of the machine learning model in the fusion model can be adaptively adjusted based on its performance, thereby improving the effectiveness of information processing.

[0074] In some embodiments, the expected number of uses for each machine learning model is determined; the weight of each machine learning model is determined based on the sum of the expected number of uses for machine learning models belonging to the same model category and the expected number of uses for each machine learning model. For example, the weight of the model belonging to the i-th model category C can be determined using the following formula. i Machine learning model E j The weights B[i][j] are:

[0075]

[0076] f j For E j Expected number of uses, f k Belongs to C i machine learning models Expected number of uses.

[0077] By using relative usage frequency to weight the machine learning model, we can avoid the adverse effects of excessively large or small absolute usage frequency on the weights of the machine learning model, thereby improving the efficiency of information processing.

[0078] In some embodiments, multiple activation models can be selected from multiple machine learning models based on the information to be processed. For example, the number of multiple machine learning models is N, and the number of activation models for each input token (i.e., the information to be processed) is K; a router can be used to determine the routing weight of each of the N machine learning models based on the current input token; and then the K machine learning models with the largest routing weights can be determined as the activation models for processing the current input token.

[0079] For example, an MLP (Multilayer Perceptron) module contains a router and N machine learning models, where R is a routing weight matrix, including the routing weights of each machine learning model configured by the router based on the input X. For a given input X, the router can calculate the weighted result softmax(RX) of the input X as the activation value; and use the top-K algorithm to select the K machine learning models with the largest activation values ​​as the activated models.

[0080] mask_top_K(softmax(X T R T ))

[0081] In this way, appropriate machine learning models can be adaptively selected for processing based on the current input information, which not only ensures processing accuracy but also improves processing efficiency.

[0082] The following is through Figure 2 The embodiments in the text exemplify the methods of clustering, fusion, and other processing in steps 110 and 120.

[0083] Figure 2 The diagram illustrates some embodiments of clustering and fusion of machine learning models as disclosed in this disclosure.

[0084] like Figure 2 As shown, there are N=8 machine learning models, including E1~E8. For input X, the original routing weights R of the 8 machine learning models are [0 0 0.5 0 0 0.2 0 0], that is, for input X, the activation models are 2 machine learning models with original routing weights not equal to 0.

[0085] exist Figure 2In the clustering process 21, the eight machine learning models can be clustered into M=4 model categories (E2, E3), (E1, E6), (E5, E7), and (E4, E8) using the clustering method in any of the above embodiments. For example, N can be used The M-dimensional clustering matrix A (representing the clustering results) processes the activation values:

[0086] mask_top_K(softmax(X T R T A

[0087]

[0088] In this case, the eight original route weights (0, 0, 0.5, 0, 0, 0, 0.2, 0, 0) can be merged into four merged route weights (0.5, 0.2, 0, 0).

[0089] In the above embodiment, the M most frequently used machine learning models are selected as cluster centers, and the fusion result of U and G is used as the distance metric between other machine learning models and the cluster centers. In this way, information processing based on the clustering results ensures accuracy while reducing the computational resources required for scheduling, thereby improving the efficiency of the computing cluster in processing information.

[0090] exist Figure 2 In the fusion processing 22, M can be used The fusion matrix B fuses the processed activation values:

[0091] mask_top_K(softmax(X T R T )) AB

[0092]

[0093] For example, Figure 2 The original processing results Y of N machine learning models on input X can be represented as E1(X) ~ E8(X). After the above fusion processing, the processing results E1′(X) ~ E4′(X) of M activation models on input X are represented as:

[0094] mask_top_K(softmax(X T R T )) ABY

[0095] From the perspective of integrating the outputs of various machine learning models, E i ′(X) can be represented as:

[0096]

[0097]

[0098]

[0099] w ij These are the weights for each machine learning model.

[0100] By weighting the machine learning model according to its usage frequency, the influence of the machine learning model in the fusion model can be adaptively adjusted based on its historical performance, thereby improving the effectiveness of information processing.

[0101] After clustering, fusion, and other processing, the information to be processed can be processed using multiple fusion models through the following examples.

[0102] In some embodiments, the model parameters of multiple fusion models are processed using a dimensionality reduction matrix to obtain multiple dimensionality reduction models; the information to be processed is then processed using these multiple dimensionality reduction models to obtain processing results. For example, the model parameters include a downsampling projection matrix, a gated projection matrix, and an upsampling projection matrix; the dimensionality reduction matrix includes a first dimensionality reduction matrix T1 for dimensionality reduction of the downsampling projection matrix, a second dimensionality reduction matrix T2 for dimensionality reduction of the gated projection matrix, and a third dimensionality reduction matrix T3 for dimensionality reduction of the upsampling projection matrix.

[0103] For example, dimensionality reduction can be achieved using the following formula:

[0104] E i ′(X)=D′ T1 (σ(T2 G′ X)⊙(T3 U′ X))

[0105] In some embodiments, the dimensionality reduction matrix is ​​determined based on the weights of each machine learning model among multiple machine learning models, and the weights of each machine learning model are determined based on the usage frequency information of each machine learning model. For example, the dimensionality reduction matrix can be configured using the following formula:

[0106] (1)

[0107] w1~ w N Let N be the weights of the machine learning models.

[0108] In the above embodiments, the parameters in the machine learning model were dimensionality reduced, decreasing the computational resources that need to be scheduled, such as reducing the consumption of storage and computing hardware resources on computing nodes. This improves the efficiency of the computing cluster in processing information.

[0109] For example, the method in any of the above embodiments can be used to determine the feature information of each machine learning model based on the U and G of each machine learning model in a plurality of machine learning models; and to perform clustering processing on the plurality of machine learning models based on the differences between the feature information of different machine learning models.

[0110] In this way, by merging similar machine learning models U and G, the impact of dimensionality reduction matrices T2 and T3 on the processing performance of the machine learning models can be reduced. This ensures both the accuracy of information processing and reduces the consumption of computing resources, thereby improving the efficiency of information processing.

[0111] The following examples illustrate how to optimize the dimensionality reduction matrix.

[0112] In some embodiments, an objective function is constructed using a candidate dimensionality reduction matrix as a variable, based on the difference between a first processing result of sample information using multiple machine learning models and a second processing result of sample information using multiple dimensionality reduction models. The multiple dimensionality reduction models are the results of processing multiple machine learning models using the candidate dimensionality reduction matrix. The objective function is solved to minimize the difference between the first and second processing results to determine the dimensionality reduction matrix. For example, the first processing result is determined based on the processing result of the sample information using a gated projection matrix and an upsampled projection matrix; the second processing result is determined based on the processing result of the dimensionality-reduced gated projection matrix, the processing result of the dimensionality-reduced upsampled projection matrix, and the first dimensionality reduction matrix.

[0113] For example, when using T2 and T3 in formula (1), T1 can be optimized in the following way. For the input sample S, the first processing result before dimensionality reduction is T1 (σ(T2 G′ S)⊙(T3 U′ S)), and the second processing result after dimensionality reduction using the candidate dimensionality reduction matrix is ​​σ(G′ S)⊙(U′ S). Taking minimizing the difference between the first and second processing results as the objective function, the optimal T1 can be solved using the least squares method.

[0114] In this way, solving for the dimensionality reduction matrix is ​​transformed into an optimization problem of minimizing the difference before and after dimensionality reduction, in order to determine the optimal dimensionality reduction matrix. This ensures both the accuracy of information processing and reduces computational resource consumption, thereby improving the efficiency of information processing.

[0115] Furthermore, combining the above embodiments, by merging similar machine learning models U and G, the impact of dimensionality reduction matrices T2 and T3 on the processing performance of the machine learning models can be reduced. With T2 and T3 fixed, optimizing T1 can minimize the impact of dimensionality reduction on the performance of the machine learning models. Thus, the accuracy of information processing is ensured while reducing the consumption of computing resources and improving the efficiency of information processing.

[0116] In the above embodiments, based on clustering, the machine learning models are fused according to their relative usage frequency as weights, and the machine learning models are reduced in dimensionality using an optimized dimensionality reduction matrix. This ensures the accuracy of information processing, reduces the consumption of computing resources, and improves the efficiency of information processing.

[0117] Figure 3 Block diagrams illustrating some embodiments of the information processing apparatus of this disclosure are shown.

[0118] like Figure 3 As shown, the information processing device 3 includes: a clustering unit 31, used to cluster multiple machine learning models according to the differences between the model parameters of different machine learning models, so as to obtain multiple model classifications; a fusion unit 32, used to fuse machine learning models belonging to the same model classification, so as to obtain multiple fused models; and a processing unit 33, used to process the information to be processed using the multiple fused models, so as to obtain processing results.

[0119] In some embodiments, the model parameters include an upsampled projection matrix and a gated projection matrix. The clustering unit 31 determines the feature information of each machine learning model based on the upsampled projection matrix and the gated projection matrix of each machine learning model among the multiple machine learning models, and performs clustering processing on the multiple machine learning models based on the differences between the feature information of different machine learning models.

[0120] In some embodiments, clustering unit 31 performs fusion processing on the upsampled projection matrix and gated projection matrix of each machine learning model to determine the feature information of each machine learning model.

[0121] In some embodiments, clustering unit 31 uses the most frequently used model among multiple machine learning models as the cluster center model, and performs clustering processing based on the differences between the model parameters of other machine learning models and the model parameters of the cluster center model.

[0122] In some embodiments, the processing unit 33 uses a dimensionality reduction matrix to process the model parameters of multiple fusion models to obtain multiple dimensionality reduction models, and uses the multiple dimensionality reduction models to process the information to be processed to obtain processing results.

[0123] In some embodiments, the dimensionality reduction matrix is ​​determined as follows: the processing unit 33 constructs an objective function with the candidate dimensionality reduction matrix as a variable based on the difference between the first processing result of the sample information using multiple machine learning models and the second processing result of the sample information using multiple dimensionality reduction models, wherein the multiple dimensionality reduction models are the results of processing multiple machine learning models using the candidate dimensionality reduction matrix; the processing unit 33 solves the objective function with the goal of minimizing the difference between the first processing result and the second processing result to determine the dimensionality reduction matrix.

[0124] In some embodiments, the model parameters include a downsampled projection matrix, a gated projection matrix, and an upsampled projection matrix. The dimensionality reduction matrix includes a first dimensionality reduction matrix for dimensionality reduction of the downsampled projection matrix, a second dimensionality reduction matrix for dimensionality reduction of the gated projection matrix, and a third dimensionality reduction matrix for dimensionality reduction of the upsampled projection matrix. The processing unit 33 determines a first processing result based on the processing result of the gated projection matrix on the sample information and the processing result of the upsampled projection matrix on the sample information, and determines a second processing result based on the processing result of the dimensionality-reduced gated projection matrix on the sample information, the processing result of the dimensionality-reduced upsampled projection matrix on the sample information, and the first dimensionality reduction matrix.

[0125] In some embodiments, the dimensionality reduction matrix is ​​determined based on the weights of each of the multiple machine learning models, and the weights of each machine learning model are determined based on the usage frequency information of each machine learning model.

[0126] In some embodiments, the fusion unit 32 determines the weight of each machine learning model based on the usage frequency information of each machine learning model among multiple machine learning models, and performs fusion processing on machine learning models belonging to the same model category based on the weight.

[0127] In some embodiments, the fusion unit 32 determines the expected number of times each machine learning model is used; and determines the weight of each machine learning model based on the sum of the expected number of times machine learning models belonging to the same model category and the expected number of times each machine learning model is used.

[0128] In some embodiments, the information to be processed includes at least one of natural language information and image information.

[0129] In the above embodiments, multiple machine learning models are clustered and fused based on the differences between model parameters, which not only ensures the accuracy of information processing, but also reduces the computing resources that need to be scheduled, thereby improving the efficiency of information processing.

[0130] Figure 4 Block diagrams illustrating other embodiments of the information processing apparatus of this disclosure are shown.

[0131] Memory 41 is used to store one or more computer-readable instructions. Memory 41 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory. Memory 41 may, for example, store operating systems, application programs, bootloaders, databases, and other programs, as well as various application programs and various data.

[0132] The processor 42 is configured to execute computer-readable instructions to implement the information processing method in any of the foregoing embodiments. Specific implementations of each step of the information processing method can be found in the above embodiments; repeated details will not be elaborated here.

[0133] The processor 42 can be configured to execute the steps of the information processing method in any of the above embodiments. For example, the information processing method includes: clustering multiple machine learning models based on the differences between the model parameters of different machine learning models to obtain multiple model classifications; fusing machine learning models belonging to the same model classification to obtain multiple fused models; and using the multiple fused models to process the information to be processed to obtain processing results. The processor 42 can be embodied in various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be an x86 or ARM architecture, etc.

[0134] The processor 42 and the memory 41 can communicate with each other directly or indirectly. For example, the processor 42 and the memory 41 can communicate via a network. The network can include a wireless network, a wired network, and / or any combination of wireless and wired networks. The processor 42 and the memory 41 can also communicate with each other via a system bus, which is not limited in this disclosure.

[0135] It should be noted that Figure 4 The components of the information processing apparatus 41 shown are merely exemplary and not limiting. The information processing apparatus 41 may have other components depending on the specific application requirements. The processor 42 can control other components in the information processing apparatus 4 to perform desired functions.

[0136] The information processing device 4 can be implemented by software, firmware and / or hardware, and can be integrated into a device with the relevant application installed.

[0137] In the above embodiments, multiple machine learning models are clustered and fused based on the differences between model parameters, which not only ensures the accuracy of information processing, but also reduces the computing resources that need to be scheduled, thereby improving the efficiency of information processing.

[0138] Figure 5 Block diagrams illustrating further embodiments of the information processing apparatus of this disclosure are shown.

[0139] Figure 5 The information processing device 5 shown can be a computer system with a dedicated hardware structure, which can perform corresponding functions when relevant applications are installed.

[0140] Information processing device 5 includes, but is not limited to, mobile terminals such as smartphones, laptops, personal digital assistants (PDAs), tablet PCs, PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), wearable devices, and fixed terminals such as digital televisions and desktop computers.

[0141] like Figure 5 As shown, the Central Processing Unit (CPU) 51 performs various processes based on a program stored in the Read-Only Memory (ROM) 52 or a program loaded from the storage section 58 into the Random Access Memory (RAM) 53. The RAM 53 stores data required as needed when the CPU 51 performs various processes, etc. The CPU is merely exemplary; it could also be other types of processors, such as the various processors described above. The ROM 52, RAM 53, and storage section 58 can be various forms of computer-readable storage media. It should be noted that although... Figure 5 The diagram shows ROM 52, RAM 53 and storage section 58, but one or more of them may be combined or located in the same or different memory or storage modules.

[0142] CPU 51, ROM 52 and RAM 53 are interconnected via bus 54. Input / output interface 55 is also connected to bus 54.

[0143] The following components are connected to the input / output interface 55: input section 56, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output section 57, including displays such as cathode ray tube (CRT), liquid crystal display (LCD), speakers, vibrators, etc.; storage section 58, including hard disks, magnetic tapes, etc.; and communication section 59, including network interface cards such as LAN cards, modems, etc. The communication section 59 allows communication processing via a network such as the Internet. It is easy to understand that, although... Figure 5 The parts of the information processing device 5 shown communicate via bus 54, but they can also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of wireless and wired networks.

[0144] As needed, drive 510 is also connected to input / output interface 55. Removable media 511, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on drive 510 as needed, so that computer programs read from them can be installed into storage section 58 as needed.

[0145] When the above series of processes are implemented through software, the program constituting the software can be installed from a network such as the Internet or a storage medium such as a removable medium 511.

[0146] According to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product that, when run on a computer, causes the computer to implement the information processing method of any of the foregoing embodiments. For example, the information processing method includes: clustering multiple machine learning models based on differences in model parameters between different machine learning models to obtain multiple model classifications; fusing machine learning models belonging to the same model classification to obtain multiple fused models; and processing information to be processed using the multiple fused models to obtain processing results. The computer program product includes computer instructions carried on a computer-readable medium, containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer instructions can be downloaded and installed from a network via communication section 59, or installed from storage section 58, or installed from ROM 52. When the computer program is executed by CPU 51, the information processing method of any embodiment of this disclosure is performed.

[0147] It should be noted that, in the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0148] A computer-readable medium may be a computer-readable storage medium, a computer-readable signal medium, or any combination thereof.

[0149] Computer-readable storage media include, but are not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Computer instructions are stored on the computer-readable storage medium that, when executed by a processor, implement the information processing methods of any of the foregoing embodiments.

[0150] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0151] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0152] In some embodiments, a computer program is also provided, comprising: instructions that, when executed by a processor, cause the processor to perform the information processing method of any of the foregoing embodiments. For example, the instructions may be embodied in computer program code.

[0153] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0155] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0156] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. An information processing method, comprising: Based on the differences in model parameters between different machine learning models, multiple machine learning models are clustered to obtain multiple model classifications; Machine learning models belonging to the same model category are fused to obtain multiple fused models; The multiple fusion models are used to process the information to be processed in order to obtain the processing results.

2. The information processing method according to claim 1, wherein, The model parameters include the upsampled projection matrix and the gated projection matrix. The clustering process for multiple machine learning models based on the differences in model parameters between different machine learning models includes: The feature information of each machine learning model is determined based on the upsampled projection matrix and the gated projection matrix of each of the plurality of machine learning models; Based on the differences in feature information between different machine learning models, clustering is performed on the multiple machine learning models.

3. The information processing method according to claim 2, wherein, The step of determining the feature information of each machine learning model based on the upsampled projection matrix and the gated projection matrix of each of the plurality of machine learning models includes: The upsampled projection matrix and the gated projection matrix of each machine learning model are fused to determine the feature information of each machine learning model.

4. The information processing method according to claim 1, wherein, The clustering process for multiple machine learning models based on the differences in model parameters between different machine learning models includes: The model most frequently used among the multiple machine learning models will be used as the cluster center model. The clustering process is performed based on the differences between the model parameters of other machine learning models and the model parameters of the cluster center model.

5. The information processing method according to claim 1, wherein, The process of using the multiple fusion models to process the information to be processed to obtain the processing result includes: The model parameters of the multiple fusion models are processed using a dimensionality reduction matrix to obtain multiple dimensionality reduction models; The information to be processed is processed using the multiple dimensionality reduction models to obtain the processing results.

6. The information processing method according to claim 5, wherein, The dimensionality reduction matrix is ​​determined in the following way: Based on the difference between the first processing result of the sample information using the multiple machine learning models and the second processing result of the sample information using the multiple dimensionality reduction models, an objective function is constructed with the candidate dimensionality reduction matrix as a variable. The multiple dimensionality reduction models are the results of processing the multiple machine learning models using the candidate dimensionality reduction matrix. The objective function is solved to determine the dimensionality reduction matrix by minimizing the difference between the first processing result and the second processing result.

7. The information processing method according to claim 6, wherein, The model parameters include the downsampled projection matrix, the gated projection matrix, and the upsampled projection matrix. The dimensionality reduction matrix includes a first dimensionality reduction matrix for reducing the dimensionality of the downsampled projection matrix, a second dimensionality reduction matrix for reducing the dimensionality of the gated projection matrix, and a third dimensionality reduction matrix for reducing the dimensionality of the upsampled projection matrix. The objective function for construction includes: The first processing result is determined based on the processing result of the sample information by the gated projection matrix and the processing result of the sample information by the upsampled projection matrix; The second processing result is determined based on the processing results of the gated projection matrix after dimensionality reduction, the processing results of the upsampled projection matrix after dimensionality reduction, and the first dimensionality reduction matrix.

8. The information processing method according to claim 5, wherein, The dimensionality reduction matrix is ​​determined based on the weight of each of the plurality of machine learning models, and the weight of each machine learning model is determined based on the usage frequency information of each machine learning model.

9. The information processing method according to any one of claims 1-8, wherein, The process of fusing machine learning models belonging to the same model category to obtain multiple fused models includes: The weight of each machine learning model is determined based on the usage frequency information of each of the multiple machine learning models. Based on the weights, machine learning models belonging to the same model category are fused together.

10. The information processing method according to claim 9, wherein, The step of determining the weight of each machine learning model based on the usage frequency information of each of the plurality of machine learning models includes: Determine the expected number of times each machine learning model will be used; The weight of each machine learning model is determined based on the sum of the expected usage counts of machine learning models belonging to the same model category and the expected usage count of each machine learning model.

11. The information processing method according to any one of claims 1-8, wherein, The information to be processed includes at least one of natural language information and image information.

12. An information processing apparatus, comprising: Clustering units are used to cluster multiple machine learning models based on the differences in model parameters between different machine learning models in order to obtain multiple model classifications. The fusion unit is used to fuse machine learning models belonging to the same model category to obtain multiple fused models; The processing unit is used to process the information to be processed using the multiple fusion models to obtain the processing result.

13. An information processing apparatus, comprising: Memory; and A processor coupled to the memory, the processor being configured to perform the information processing method of any one of claims 1-11 based on instructions stored in the memory.

14. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the information processing method according to any one of claims 1-11.

15. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the information processing method according to any one of claims 1-11.

Images