Information processing method and device, equipment, medium and product

Abstract

The invention discloses an information processing method and device, equipment, a medium and a product, and relates to the technical field of artificial intelligence. The information processing method comprises the steps of obtaining first information; wherein the first information comprises user request information and first attribute information of a first GPU (Graphic Processing Unit); the first GPU is any one GPU in a plurality of GPUs running the first large model; inputting the first information into a second large model running on the first GPU to obtain a first reasoning result which is output by the second large model and corresponds to the user request information; wherein the second large model comprises a GPU fitting device and the first large model; and the GPU fitting device is used for fitting the calculation precision difference of the first large model on the plurality of GPUs through the first attribute information, so that the first large model running on the plurality of GPUs outputs the same reasoning result for the user request information. According to the scheme disclosed by the invention, the reasoning results of the large model on different GPUs can be kept the same.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to an information processing method and device, equipment, medium and product. BACKGROUND

[0002] Large language models (LLMs) refer to deep learning models with huge parameters and large training data, which perform outstandingly in the field of natural language processing (NLP). Such models can complete various tasks such as text generation, translation, question answering and code writing through massive data and powerful computing power.

[0003] Due to some factors, such as graphics processing unit (GPU) architecture, GPU operator version, GPU computing precision, etc., the inference results of the same large model on different GPUs are different. SUMMARY

[0004] Embodiments of the application provide an information processing method, device, equipment, medium and product, which can solve the problem that the inference results of the same large model on different GPUs are different.

[0005] In a first aspect, embodiments of the application provide an information processing method, comprising: obtaining first information; wherein the first information comprises user request information and first attribute information of a first GPU; the first GPU is any one of a plurality of GPUs running a first large model; inputting the first information into a second large model running on the first GPU to obtain a first inference result corresponding to the user request information output by the second large model; wherein the second large model comprises a GPU fitter and the first large model; the GPU fitter is used to fit the computing precision difference of the first large model on the plurality of GPUs through the first attribute information, so that the first large model running on the plurality of GPUs outputs the same inference result for the user request information.

[0006] In a second aspect, embodiments of the application provide an information processing device, comprising: a first obtaining module configured to obtain first information; wherein the first information comprises user request information and first attribute information of a first GPU; the first GPU is any one of a plurality of GPUs running a first large model; The inference module is configured to input the first information into a second large model running on the first GPU to obtain a first inference result corresponding to the user request information and output by the second large model; the second large model comprises a GPU fitter and the first large model; and the GPU fitter is configured to fit the calculation accuracy difference of the first large model on the plurality of GPUs by using the first attribute information, so that the first large model running on the plurality of GPUs outputs the same inference result for the user request information.

[0007] In a third aspect, an electronic device is provided, and the electronic device includes a processor and a memory storing computer program instructions; and the processor implements the information processing method provided in the embodiments of the present application when executing the computer program instructions.

[0008] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer program instructions; and the computer program instructions are executed by a processor to implement the information processing method provided in the embodiments of the present application.

[0009] In a fifth aspect, a computer program product is provided, and instructions in the computer program product are executed by a processor of an electronic device to enable the electronic device to perform the information processing method provided in the embodiments of the present application.

[0010] In the embodiments of the present application, the first information is obtained; the first information includes user request information and first attribute information of a first graphics processing unit (GPU); the first GPU is any one of a plurality of GPUs running a first large model; the first information is input into a second large model running on the first GPU to obtain a first inference result corresponding to the user request information and output by the second large model; the second large model includes a GPU fitter and the first large model; and the GPU fitter is configured to fit the calculation accuracy difference of the first large model on the plurality of GPUs by using the first attribute information, so that the first large model running on the plurality of GPUs outputs the same inference result for the user request information. In this way, the GPU fitter can fit the calculation accuracy difference of the large model on the plurality of GPUs, so that the same large model running on the plurality of GPUs can output the same inference result for the same user request information. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced. Those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0012] Figure 1 is a flowchart of the information processing method provided in the embodiments of the present application; Figure 2 is a schematic diagram of information processing provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of an information processing device provided by an embodiment of the present application; Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0013] The features and exemplary embodiments of various aspects of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are intended to explain the present application, not to limit the present application. The present application can be implemented without some of the specific details described below. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.

[0014] It should be noted that, in this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0015] The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. The terms related to the embodiments of the present application are explained below.

[0016] Large Language Models (LLMs) are artificial intelligence models with a large number of parameters and a large amount of training data.

[0017] Operator version is an optimized implementation of an algorithm or data flow model in computer science.

[0018] Fitter is a core concept in mathematics, statistics and machine learning, which refers to a tool for approximating the law of real data through algorithms or models, aiming to extract patterns from observed data and use them for prediction or explanation.

[0019] Mixture of Experts (MoE) is a neural network architecture that dynamically selects a small number of sub-networks, called "experts", for each input to compute through a gating network, in a sparse activation manner to improve model capacity and computational efficiency.

[0020] Experts are a set of neural networks that share the same structure but have independent parameters. In the Transformer architecture, they are represented as multiple parallel feed-forward neural network layers. Each expert network can be viewed as a model that has expertise in a certain domain or data pattern.

[0021] Gating Network is a lightweight routing or scheduling network that analyzes the input and decides which expert networks to send it to for processing. The gating network generates a weight or probability for each expert network, and the expert network with the highest weight or probability will be activated.

[0022] Token is the basic unit for artificial intelligence models to understand and generate text.

[0023] Learning rate is a hyperparameter that controls the step size of parameter updates when training a neural network, directly affecting the convergence speed and performance of the model.

[0024] The information processing method, device, equipment, medium and product provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios.

[0025] Figure 1 is a flowchart of the information processing method provided by the embodiments of the present application. As shown in Figure 1 , the information processing method can include: Step 101: obtaining first information; wherein the first information includes user request information and first attribute information of a first GPU; the first GPU is any one of a plurality of GPUs running a first large model; In some embodiments of the present application, the first attribute information in the embodiments of the present application can include at least one of the following: architecture information, model information, operator version information, and calculation precision information.

[0026] Step 102: inputting the first information into a second large model running on the first GPU to obtain a first inference result output by the second large model and corresponding to the user request information; wherein the second large model comprises a GPU fitter and a first large model; the GPU fitter is configured to fit a calculation precision difference of the first large model on multiple GPUs through the first attribute information, so that the first large model running on the multiple GPUs outputs the same inference result for the user request information. In some embodiments of the present application, in order to enable the fitter to flexibly adapt to different GPUs, it is necessary to comprehensively consider the influence of the GPU on the inference result of the large model from the aspects of hardware, software, application, etc. At the hardware level, the fitter needs to take into account GPU hardware-related parameters, such as GPU architecture, GPU model, and GPU calculation precision, etc.; at the software level, the fitter needs to take into account software-related parameters, such as GPU operator version; wherein the implementation of the operator is different for different operator versions. In some cases, when the large model is applied, different deployment precisions can also be selected according to different scenarios, at which time the fitter needs to take into account the influence of model application-related parameters.

[0027] In some embodiments of the present application, after obtaining the first attribute information of the first GPU and the user request information, the first attribute information of the first GPU and the user request information can be spliced, and the spliced information can be input into the second large model. It can be understood that the second large model in the embodiments of the present application is a large model with a fitter.

[0028] In some embodiments of the present application, step 102 can comprise: inputting the first information into the GPU fitter to obtain a second inference result output by the GPU fitter and corresponding to the user request information; and inputting the second inference result and the user request information into the first large model to obtain the first inference result output by the first large model.

[0029] In some embodiments of the present application, the GPU fitter can comprise a plurality of expert networks; accordingly, inputting the first information into the GPU fitter to obtain a second inference result output by the GPU fitter and corresponding to the user request information can comprise: determining a target expert network from the plurality of expert networks according to the first attribute information; and inputting the user request information into the target expert network to obtain a first inference result output by the target expert network.

[0030] In some embodiments of the present application, the GPU fitter can comprise an expert network judgment module, and the expert network judgment module comprises a feedforward network layer and a probability output (softmax) layer. When the spliced information is input into the expert network judgment module, the softmax layer outputs a probability corresponding to each expert network, and then the expert network with the largest probability is taken as the target expert network.

[0031] The selection of the target expert network is determined by GPU architecture information, GPU model information, GPU operator version information, and GPU computing precision information.

[0032] In some embodiments of the present application, the target expert network includes an N-layer low-rank matrix network; where N is the number of network layers of the large model. Inputting the user request information into the target expert network to obtain the second inference result output by the target expert network can include: inputting the user request information into the target expert network to obtain the inference result output by each layer of the low-rank matrix network; where the second inference result includes the inference result output by the N-layer low-rank matrix network.

[0033] In some embodiments of the present application, the GPU fitter has a hybrid expert model, and the hybrid expert model includes a plurality of expert networks, each of which includes an N-layer low-rank matrix network, where N is the number of network layers of the large model. Each layer of the low-rank matrix network includes a d*n-dimensional sub-neural network, an n*d-dimensional sub-neural network, and an n*n feedforward neural network; where d is the dimension of the embedding vector corresponding to the user request information, n is the dimension after dimension reduction, the d*n-dimensional sub-neural network is used to map the input to a low-dimensional space, the n*d-dimensional sub-neural network is used to map the representation in the low-dimensional space back to the original space, and the feedforward neural network allows the inference result to be finely adjusted in the low-dimensional space.

[0034] In embodiments of the present application, by combining multiple expert networks and multiple layers of low-rank matrix networks, the fitter can flexibly take into account many influencing factors such as computing precision, and can reduce the amount of activation parameters of the low-rank matrix by activating part of the experts, reduce the amount of computing parameters of the fitter in a single inference, and reduce the additional computing resource demand brought by the fitter, so that the fitter can flexibly and at low cost compensate for the difference in computing precision.

[0035] In some embodiments of the present application, inputting the second inference result and the user request information into the first large model to obtain the first inference result output by the first large model can include: fusing the inference result output by the i-th layer of the low-rank matrix network and the inference result output by the i-th network layer of the first large model to obtain the input of the i+1-th network layer of the first large model; where i is a positive integer greater than or equal to 1 and less than or equal to N, and the N-th network layer of the first large model outputs the first inference result.

[0036] Embodiments of the present application do not limit the way in which the inference result output by the i-th layer of the low-rank matrix network and the inference result output by the i-th network layer of the first large model are fused, and any available way can be applied in embodiments of the present application, for example, splicing the inference result output by the i-th network layer of the first large model and the inference result output by the i-th layer of the low-rank matrix network.

[0037] In some embodiments of this application, after the user request information and the second inference result output by the GPU fitter are input into the first large model, each network layer of the first large model outputs the corresponding inference result, and the output layer of the first large model outputs the first inference result.

[0038] In some embodiments of this application, the inference result output by the i-th low-rank matrix network and the inference result output by the i-th network layer of the first large model can be fused to obtain the input of the (i+1)-th network layer of the first large model, and the input of the (i+1)-th network layer of the first large model can be input into the (i+1)-th network layer of the first large model.

[0039] It should be noted that when i is N, the inference result output by the Nth layer low-rank matrix network and the inference result output by the Nth layer network of the first large model are fused to obtain the input of the output layer of the first large model.

[0040] In some embodiments of this application, step 103 may include: fusing the second inference result output by the target expert network output layer and the inference result output by the i-th layer of the first large model to obtain the input of the (i+1)-th layer of the first large model; where i is a positive integer greater than or equal to 1 and less than or equal to N, and the N-th layer of the first large model outputs the first inference result.

[0041] In some embodiments of this application, the second inference result output by the target expert network output layer can be scaled using a gating signal, and the scaled inference result can be fused with the inference results output by each layer of the first large model. The gating signal is used to control the magnitude of the fitter's output value.

[0042] In the embodiments of this application, the gating signal can prevent the fitter output from being too large or too small, thus affecting the original output result of the large model.

[0043] In some embodiments of this application, the information processing method provided in this application may further include: acquiring multiple training samples; determining candidate training samples from the multiple training samples based on the second attribute information of the training samples at preset intervals; and training a GPU fitter using the candidate training samples.

[0044] In some embodiments of this application, the second attribute information includes at least one of the following: Application functions, task categories, sentence structures, and learning difficulty.

[0045] In some embodiments of this application, application functions, task categories, sentence structures, and learning difficulty are used to select representative fitter training data from multiple dimensions and in a comprehensive manner to achieve good fit training.

[0046] In some embodiments of this application, application functions represent common functional requirements in large-scale model application scenarios, such as language understanding ability, language generation ability, knowledge question answering ability, text processing ability, and vertical scenario ability; task categories represent related large-scale model task types included under the application function category, such as semantic understanding, contextual understanding, sentiment understanding, long text understanding, text generation, code generation, creative writing, factual knowledge answering, complex question reasoning answering, code logic verification, information summarization, text classification, text rewriting, language translation, healthcare, financial risk, investment analysis, education and learning, and academic research. The correspondence between application functions and task categories is shown in Table 1. Table 1 In some embodiments of this application, the expression sentence type is the sentence type of the input text under a specific task. The correspondence between the classification dimensions of the expression sentence type and the sentence type is shown in Table 2: Table 2 In some embodiments of this application, the sentence type of the input text to the large model is determined by adding the information from Table 2 above to a prompt. For example, the prompt might be: "You are a linguistics expert, you are good at classifying Chinese sentence types, existing sentence types include subject-verb structure, subject-verb-object structure... Please determine which sentence type the following sentence belongs to: {input text}". After classifying the sentence type of all training data, for missing or insufficient sentence types, the large model is used in conjunction with the prompt containing sentence type requirements to rewrite the data and supplement the relevant type data.

[0047] In some embodiments of this application, determining candidate training samples from multiple training samples based on the second attribute information of the training samples may include: classifying multiple training samples according to application functions and task categories to obtain categories of multiple training samples; determining the expression sentence patterns of multiple training samples using a first large model; determining the learning difficulty of each training sample based on the difference between the calculation results of the second large model and the calculation results of the first large model on the word elements in the training samples, and the number of word elements included in the training samples, and classifying multiple training samples into levels according to the learning difficulty; selecting the same number of training samples from training samples with the same category, the same expression sentence patterns, and different levels; and using the selected training samples as candidate training samples.

[0048] In some embodiments of this application, when considering the difference between the calculation results of the second largest model on the word units in the training samples and the calculation results of the first largest model on the word units, as well as the number of word units included in the training samples, the learning difficulty of each training sample can be determined by the following formula (1): (1) In formula (1), avg_value represents the learning difficulty of the training sample, L represents the number of tokens in the training sample, l represents the index of the token in the training sample, i represents the i-th feature of the token in the training sample, and d represents the total number of feature dimensions. This is the result of the second largest model's calculation of the token. This is the result of the first major model's calculation of the token.

[0049] In some embodiments of this application, after the learning difficulty of each training sample is determined by the above formula (1), the learning difficulty of each training sample can be normalized to obtain the normalized value of the learning difficulty of each training sample.

[0050] In some embodiments of this application, the learning difficulty of each training sample can be normalized using the following formula (2): (2) In formula (2), This is a normalized value representing the learning difficulty of the training samples. To adjust the learning difficulty of the training samples, This represents the maximum learning difficulty among all training samples.

[0051] In some embodiments of this application, after obtaining the normalized value of the learning difficulty of each training sample, the training samples can be classified into different levels based on the normalized value of the learning difficulty. For example, training samples with normalized values ​​in the interval [0.7, 1] are classified as high-level training samples, training samples with normalized values ​​in the interval [0.3, 0.7) are classified as medium-level training samples, and training samples with normalized values ​​in the interval [0, 0.3] are classified as low-level training samples.

[0052] When determining candidate training samples, firstly, multiple training samples in the first training dataset are classified according to application function and task type. Secondly, the expression sentence structure is judged by a large model. Then, the learning difficulty of each training sample is calculated according to the learning difficulty calculation method mentioned above, and a level is assigned. For the permutation and combination of the three-level categories of application function, task type, and expression sentence structure, the same number of training samples are evenly sampled from the training samples at the low, medium, and high difficulty levels. These sampled training samples are used as the initial training samples, and the fitter is trained using the initial training samples.

[0053] In this embodiment of the application, uniform sampling in training samples of different levels can make the learning difficulty of the training samples evenly distributed, thereby reducing the learning difficulty of the fitter.

[0054] In some embodiments of this application, during the training process, the learning difficulty of each training sample can be recalculated using a second large model with a trained fitter at regular intervals, and the training samples can be resampled. The fitter can then be trained again using the resampled training samples to achieve adaptive adjustment of the training samples.

[0055] In the embodiments of this application, by selecting initial training samples and resampling the training samples, the fitter can learn and improve smoothly. By adapting the learning difficulty, it is possible to select data with large differences in model inference result values, efficiently expand the model learning coverage, improve training efficiency and model generalization, and reduce the time cost of adaptation.

[0056] In this embodiment, a large-scale model task structure is constructed, divided into four layers: large-scale model application function, task type, expression sentence structure, and learning difficulty. This structure is used to select representative fitting training data from multiple dimensions and in a comprehensive manner. The division and supplementation of application function, task type, and expression sentence structure are achieved through large-scale model prompting engineering. In this application, the learning difficulty is determined by the difference between the calculated value of the large model on the new GPU and the calculated value of the large model on the original GPU, as well as the number of words included in the training sample. After normalizing the learning difficulty, it is divided into three levels: high, medium, and low. During the training process, the level evaluation results are gradually adjusted, and new training samples are automatically selected to achieve adaptive adjustment and improvement of training samples.

[0057] In some embodiments of this application, training a GPU fitter using alternative training samples may include: training the GPU fitter using alternative training samples and employing a hierarchical, phased training method with a learning rate.

[0058] In some embodiments of this application, training the GPU fitter using alternative training samples and a hierarchical, phased training method with a learning rate may include: determining the learning rate according to the following formula (3) during the training of the GPU fitter: (3) in, Let be the learning rate of the nth layer of the GPU fitter's neural network at training time t. The initial learning rate, It is a constant; , Let represent the exponential decay rate of the second-order moment estimate, and t-1 be the previous training time point t. , For differentiation operators, This represents the trainable parameters of the nth layer of the neural network. Let be the loss function of the nth layer neural network at training time t; , This represents the inference results output by the first large model on the training samples. This represents the inference results output by the second largest model on the training samples. For gating signals, This represents the L2 norm constraint.

[0059] In some embodiments of this application, the learning rate can be dynamically adjusted from bottom to top according to the number of neural network layers n in the above formula (3), and the learning rate can be dynamically adjusted according to time by the training time point t in the above formula (3).

[0060] In some embodiments of this application, both the fitter and the large model are N-layer neural networks. Since the input to the upper layers of the fitter depends on the output of the lower layers, it varies more significantly during training. The lower layers typically complete the fitting learning process first. Therefore, this application incorporates a layered and phased training strategy tailored to the model structure, allowing the learning rate to be dynamically and independently adjusted across network layers and over time within the model structure space.

[0061] In some embodiments of this application, a gate-based signal is introduced into the loss function. The L2 norm constraint can prevent the fitter output from having too much influence on the output of the large model, thus reducing the generalization of the large model after transfer.

[0062] In some embodiments of this application, the neural network operates from bottom to top, with the learning rate decreasing at a progressively slower rate layer by layer, thereby achieving better fitter training results. Simultaneously, to accelerate the fitter learning process, this application constructs the loss function value of the hierarchical neural network based on the difference between the calculated values ​​of the original large model and the calculated values ​​of the large model with the fitter, enabling refined introduction of training supervision signals and reducing the learning difficulty of the fitter.

[0063] In some embodiments of this application, the fitter is trained by constructing a hierarchical, phased training method that integrates gating signal control. During the training of the fitter, a spatially hierarchical and temporally dynamic learning adjustment scheme is adopted. The network structure progresses from bottom to top, and the learning rate decreases at a progressively slower rate, enabling dynamic adjustments over time. Simultaneously, a loss function value is constructed for the hierarchical neural network to achieve refined introduction of training supervision signals, reducing the learning difficulty of the fitter and enabling spatially hierarchical adjustments.

[0064] Figure 2 This is an overall schematic diagram of the information processing provided in the embodiments of this application.

[0065] exist Figure 2 In this process, based on GPU hardware and corresponding software, the learning difficulty of each training sample in the overall training sample set is determined by the difference between the calculation results of the large model under the new GPU and the large model under the original GPU and the number of words included in the training sample. Based on the application function, task type, expression sentence structure and learning difficulty, candidate training samples are determined from the overall training sample set. The fitter is trained using a hierarchical and phased training method using the candidate training samples. During the training process, the learning difficulty of each training sample is recalculated using the large model with the trained fitter, and the training samples are resampled to determine candidate training samples. The fitter is then trained again using the resampled training samples.

[0066] When using a large model for inference, the user input text is acquired and converted into an embedding vector. The embedding vector, GPU architecture information, GPU model information, GPU operator version information, and GPU computational precision information are concatenated and input into the expert network judgment module of the fitter. The expert network judgment module determines the target expert network from multiple expert networks. The embedding vector is input into the large model and the target expert network. The inference results output by the target expert network are scaled using a gating signal. The scaled inference results are fused with the inference results output by each layer of the large model. The multiple inference results output by the last layer of the large model are input into the softmax layer. The softmax layer outputs the probability of each inference result, and then outputs the inference result corresponding to the highest probability.

[0067] This application also provides an information processing device, such as... Figure 3As shown. Figure 3 This is a schematic diagram of the structure of the information processing device provided in the embodiments of this application. The information processing device 300 may include: The first acquisition module 301 is used to acquire first information; wherein, the first information includes user request information and first attribute information of the first graphics processor (GPU); the first GPU is any one of the multiple GPUs running the first large model; The inference module 302 is used to input the first information into the second large model running on the first GPU, and obtain the first inference result corresponding to the user request information output by the second large model; wherein, the second large model includes a GPU fitter and the first large model; the GPU fitter is used to fit the computational accuracy difference of the first large model on multiple GPUs through the first attribute information, so that the first large model running on multiple GPUs outputs the same inference result for the user request information.

[0068] In some embodiments of this application, the inference module 302 includes: The first inference submodule is used to input the first information into the GPU fitter and obtain the second inference result output by the GPU fitter corresponding to the user request information. The second inference submodule is used to input the second inference result and the user request information into the first main model to obtain the first inference result output by the first main model.

[0069] In some embodiments of this application, the GPU fitter includes multiple expert networks; Accordingly, the first reasoning submodule includes: The determining unit is used to determine the target expert network from multiple expert networks based on the first attribute information; The inference unit is used to input user request information into the target expert network and obtain the second inference result output by the target expert network.

[0070] In some embodiments of this application, the target expert network includes an N-layer low-rank matrix network; where N is the number of layers in the large model's network. Accordingly, the reasoning unit is specifically used for: The user request information is input into the target expert network to obtain the inference results output by each layer of the low-rank matrix network; wherein, the second inference result includes the inference results output by the N layers of low-rank matrix networks.

[0071] In some embodiments of this application, the second inference submodule is specifically used for: The inference result output by the i-th low-rank matrix network is fused with the inference result output by the i-th network layer of the first large model to obtain the input of the (i+1)-th network layer of the first large model; where i is a positive integer greater than or equal to 1 and less than or equal to N; the N-th network layer of the first large model outputs the first inference result.

[0072] In some embodiments of this application, the first attribute information includes at least one of the following: Architecture information, model information, operator version information, and calculation accuracy information.

[0073] In some embodiments of this application, the information processing apparatus provided in this application further includes: The second acquisition module is used to acquire multiple training samples; The determination module is used to determine candidate training samples from multiple training samples at preset intervals based on the second attribute information of the training samples. The training module is used to train the GPU fitter using alternative training samples.

[0074] In some embodiments of this application, the second attribute information includes at least one of the following: Application functions, task categories, sentence structures, and learning difficulty.

[0075] In some embodiments of this application, the determining module is specifically used for: Multiple training samples are classified according to application function and task category to obtain the categories of multiple training samples; The first major model is used to determine the expression patterns of multiple training samples; Based on the difference between the calculation results of the second model and the calculation results of the first model for the word units in the training samples, and the number of word units included in the training samples, the learning difficulty of each training sample is determined, and multiple training samples are classified into levels according to the learning difficulty. Select the same number of training samples from training samples of the same category, the same sentence structure, but different levels; The selected training samples will be used as candidate training samples.

[0076] In some embodiments of this application, the determining module is specifically used for: The learning difficulty of the training samples is determined by the above formula (1).

[0077] In some embodiments of this application, the training module is specifically used for: Using alternative training samples, a tiered and phased training approach with varying learning rates is employed to train the GPU fitter.

[0078] In some embodiments of this application, the training module is specifically used for: During the training of the GPU fitter, the learning rate is determined according to the above formula (3).

[0079] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0080] Electronic device 400 may include processor 401 and memory 402 storing computer program instructions.

[0081] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0082] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where suitable, memory 402 may include removable or non-removable (or fixed) media. Where suitable, memory 402 may be internal or external to an electronic device. In some specific embodiments, memory 402 is a non-volatile solid-state memory.

[0083] In some specific embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the information processing methods according to this application.

[0084] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement the information processing method provided in the embodiments of this application.

[0085] In one example, the electronic device may also include a communication interface 403 and a bus 410. Wherein, as... Figure 4As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0086] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0087] Bus 410 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0088] The electronic device can execute the information processing method provided in the embodiments of this application, thereby achieving the corresponding technical effects of the information processing method provided in the embodiments of this application.

[0089] In addition, in conjunction with the information processing methods in the above embodiments, this application also provides a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the information processing method provided in this application. Examples of computer-readable storage media include non-transitory computer-readable media, such as ROM, RAM, magnetic disks, or optical disks.

[0090] This application provides a computer program product. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the information processing method provided in this application and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0091] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0092] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0093] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0094] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0095] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An information processing method, characterized in that, The method includes: Obtain first information; wherein, the first information includes user request information and first attribute information of the first graphics processor (GPU); the first GPU is any one of the multiple GPUs running the first large model; The first information is input into a second large model running on the first GPU to obtain a first inference result output by the second large model corresponding to the user request information; wherein, the second large model includes a GPU fitter and the first large model; the GPU fitter is used to fit the computational accuracy difference of the first large model on the multiple GPUs through the first attribute information, so that the first large model running on the multiple GPUs outputs the same inference result for the user request information.

2. The method according to claim 1, characterized in that, The step of inputting the first information into a second large model running on the first GPU to obtain a first inference result output by the second large model corresponding to the user request information includes: The first information is input into the GPU fitter to obtain the second inference result output by the GPU fitter corresponding to the user request information; The second inference result and the user request information are input into the first large model to obtain the first inference result output by the first large model.

3. The method according to claim 2, characterized in that, The GPU fitter includes multiple expert networks; The step of inputting the first information into the GPU fitter to obtain the second inference result output by the GPU fitter corresponding to the user request information includes: Based on the first attribute information, a target expert network is determined from the plurality of expert networks; The user request information is input into the target expert network to obtain the second inference result output by the target expert network.

4. The method according to claim 3, characterized in that, The target expert network comprises an N-layer low-rank matrix network; where N is the number of layers in the large model's network. The step of inputting the user request information into the target expert network and obtaining the second inference result output by the target expert network includes: The user request information is input into the target expert network to obtain the inference results output by each layer of the low-rank matrix network; wherein, the second inference result includes the inference results output by the N layers of the low-rank matrix network.

5. The method according to claim 4, characterized in that, The step of inputting the second inference result and the second inference result into the first large model to obtain the first inference result output by the first large model includes: The inference result output by the i-th low-rank matrix network is fused with the inference result output by the i-th network layer of the first large model to obtain the input of the (i+1)-th network layer of the first large model; where i is a positive integer greater than or equal to 1 and less than or equal to N, and the N-th network layer of the first large model outputs the first inference result.

6. The method according to any one of claims 1 to 5, characterized in that, The first attribute information includes at least one of the following: Architecture information, model information, operator version information, and calculation accuracy information.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain multiple training samples; At preset time intervals, candidate training samples are determined from the plurality of training samples based on the second attribute information of the training samples; The GPU fitter is trained using the alternative training samples.

8. The method according to claim 7, characterized in that, The second attribute information includes at least one of the following: Application functions, task categories, sentence structures, and learning difficulty.

9. The method according to claim 8, characterized in that, The step of determining candidate training samples from the plurality of training samples based on the second attribute information of the training samples includes: The multiple training samples are classified according to the application function and the task category to obtain the category of the multiple training samples; The first large model is used to determine the expression patterns of the multiple training samples; Based on the difference between the calculation results of the second large model on the word units in the training samples and the calculation results of the first large model on the word units, as well as the number of word units included in the training samples, the learning difficulty of each training sample is determined, and the multiple training samples are classified into levels according to the learning difficulty. Select the same number of training samples from training samples of the same category, the same sentence structure, but different levels; The selected training samples are used as the alternative training samples.

10. The method according to claim 9, characterized in that, The step of determining the learning difficulty of each training sample based on the difference between the calculation results of the second large model and the calculation results of the first large model on the word units in the training sample, and the number of word units included in the training sample, includes: The learning difficulty of the training samples is determined by the following formula: Where avg_value represents the learning difficulty of the training sample, L represents the number of words in the training sample, l represents the index of the word in the training sample, i represents the i-th feature of the word in the training sample, and d represents the total number of feature dimensions. This represents the calculation result of the i-th dimension feature of the word in the second largest model. This represents the calculation result of the i-th dimension feature of the word in the first large model.

11. The method according to claim 7, characterized in that, The step of training the GPU fitter using the candidate training samples includes: Using the candidate training samples, the GPU fitter is trained using a hierarchical and phased training method with a learning rate.

12. The method according to claim 11, characterized in that, The step of training the GPU fitter using the candidate training samples and employing a hierarchical, phased training approach with a learning rate includes: During the training of the GPU fitter, the learning rate is determined according to the following formula: in, Let be the learning rate of the nth layer of the GPU fitter's neural network at training time t. The initial learning rate, It is a constant; , Let represent the exponential decay rate of the second-order moment estimate, and t-1 be the previous training time point t. , For differentiation operators, This represents the trainable parameters of the nth layer of the neural network. Let be the loss function of the nth layer neural network at training time t; , This represents the inference results output by the first large model on the training samples. This represents the inference results output by the second largest model on the training samples. For gating signals, This represents the L2 norm constraint.

13. An information processing device, characterized in that, The device includes: A first acquisition module is used to acquire first information; wherein, the first information includes user request information and first attribute information of a first GPU; the first GPU is any one of multiple GPUs running the first large model; An inference module is used to input the first information into a second large model running on the first GPU to obtain a first inference result output by the second large model corresponding to the user request information; wherein, the second large model includes a GPU fitter and the first large model; the GPU fitter is used to fit the computational accuracy difference of the first large model on the multiple GPUs through the first attribute information, so that the first large model running on the multiple GPUs outputs the same inference result for the user request information.

14. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the information processing method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the information processing method as described in any one of claims 1-12.

16. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the information processing method as described in any one of claims 1-12.

Images