Weight conversion method and device of large language model, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for weight conversion of a large language model. The method includes: determining a model graph and a weight dictionary of the large language model, wherein the weight dictionary is in key-value pair form, the key in the key-value pair is the name of the constituent module of the large language model, and the value in the key-value pair is the weight tensor of the large language model; performing a weight self-processing operation on the weight tensor; updating the model graph corresponding to the weight dictionary after the weight self-processing operation; and determining the weight dictionary corresponding to the updated model graph as the first format model weights; and performing weight conversion on the first format model weights based on the mapping relationship between the first format weight file and the second format weight file to obtain the second format model weights. This method performs weight conversion on the first format model weights based on the mapping relationship, completing the weight conversion operation without developing additional code, reducing the overhead and time of weight conversion for large language models.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for weight transformation of a large language model. Background Technology

[0002] In existing technologies, common custom processing and transformation operations include weight conversion between frameworks, sparse expert modification of models, and adding bias layers.

[0003] However, due to the diversity of R&D needs, development frameworks and model structures, developers often face problems such as long development time, large development workload and difficulty in reuse when developing their own code for custom weight processing and transformation operations. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for weight transformation of large language models, in order to solve the shortcomings of existing technologies where, due to the diversity of research and development needs, development frameworks, and model structures, developers often have to develop custom weight processing and transformation code, which is time-consuming, requires a large amount of development effort, and is difficult to reuse.

[0005] This invention provides a weight transformation method for a large language model, comprising the following steps: Determine the model graph and weight dictionary of the large language model. The weight dictionary is in the form of key-value pairs, where the key in the key-value pair is the name of the component module of the large language model, and the value in the key-value pair is the weight tensor of the large language model. Perform a weight self-processing operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. Based on the mapping relationship between the first format weight file and the second format weight file, the weights of the first format model are converted to obtain the weights of the second format model.

[0006] According to the present invention, a weight transformation method for a large language model is provided, wherein the weight self-processing operation includes at least one of weight addition operation, weight deletion operation, and weight modification operation; The step of performing a weight self-processing operation on the weight tensor and updating the model graph corresponding to the weight dictionary after the weight self-processing operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights, includes at least one of the following: Perform the weight increment operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight increment operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. Perform the weight deletion operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight deletion operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

[0007] According to the present invention, a weight transformation method for a large language model includes at least one of the following: weight modification operation, tensor dimension modification of the weights of the organization modules in the large language model, and hybrid expert modeling. The step of performing the weight modification operation on the weight tensor, updating the model graph corresponding to the weight dictionary after the weight modification operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes at least one of the following: The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight. Modify the tensor dimension of the organization module weights in the weight tensor, update the model graph corresponding to the weight dictionary after the tensor dimension of the organization module weights is modified, and determine the weight dictionary corresponding to the updated model graph as the first format model weights. The weight tensor is subjected to the hybrid expert modeling, and the model graph corresponding to the weight dictionary after hybrid expert modeling is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

[0008] According to a method for weight transformation of a large language model provided by the present invention, the step of performing hybrid expert modeling on the weight tensor, updating the model graph corresponding to the weight dictionary after hybrid expert modeling, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes: The original model weights of the multilayer perceptron in the weight tensor are uniformly divided into a preset number of expert head sets, or the original model weights are copied to obtain the preset number of expert head sets. The weight dictionary after uniformly dividing or copying the original model weights is determined as the updated weight dictionary, and the multilayer perceptron in the model graph corresponding to the updated weight dictionary is replaced with a hybrid expert module. The weight dictionary corresponding to the updated model graph is determined as the expert weights. The routing header of the gated network in the hybrid expert model is generated by random initialization, and the weights of the first format model are determined based on the expert weights and the routing header.

[0009] According to a weight transformation method for a large language model provided by the present invention, the method includes modifying the tensor dimension of the organization module weights on the weight tensor, updating the model graph corresponding to the weight dictionary after the tensor dimension modification of the organization module weights, and determining the weight dictionary corresponding to the updated model graph as the first format model weights, comprising: A new tensor for determining the weights of the organizational modules is obtained by assigning values ​​to the new tensor to obtain the target weight tensor. Remove the weights of the organization modules and point the target weight tensor to the weight tensor of the organization modules to obtain the target weight dictionary. Update the model graph corresponding to the target weight dictionary and determine the weight dictionary corresponding to the updated model graph as the first format model weights.

[0010] According to the present invention, a method for weight conversion of a large language model, wherein the weight conversion of the first format model weights based on the mapping relationship between a first format weight file and a second format weight file to obtain the second format model weights includes: Perform the following steps for all model components in the first format weight file: Obtain the model components of the first format weight file and the first weight of the model components; based on the mapping relationship, determine the second weight in the second format weight file corresponding to the first weight; if the tensor dimensions of the first weight and the second weight are consistent, determine the second weight based on the first weight and store the second weight; if the tensor dimensions of the first weight and the second weight are inconsistent, return an exception message. Export all the second weights as the second format model weights.

[0011] The present invention also provides a weight transformation device for a large language model, comprising the following units: A determining unit is used to determine the model graph and weight dictionary of a large language model. The weight dictionary is in the form of key-value pairs, where the key in each key-value pair is the name of the component module of the large language model, and the value in each key-value pair is the weight tensor of the large language model. The weight self-processing unit is used to perform weight self-processing operations on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight conversion unit is used to convert the weights of the first format model based on the mapping relationship between the first format weight file and the second format weight file to obtain the weights of the second format model.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the weight transformation method of any of the large language models described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the weight transformation method for a large language model as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a weight transformation method for a large language model as described above.

[0015] The present invention provides a method, apparatus, electronic device, and storage medium for weight conversion of large language models. On the one hand, based on the concept of tensor maintenance and reconstruction graph, this method highly integrates many commonly used custom weight processing operations, thereby broadening the application scope of weights in different scenarios and effectively solving key problems such as the overly singular application scenarios and difficulty in reuse of existing algorithms. On the other hand, based on the mapping relationship between a first-format weight file and a second-format weight file, this method performs weight conversion on the first-format model weights to obtain the second-format model weights. The weight conversion operation is completed without additional code development, reducing the overhead of weight conversion for large language models, reducing the weight conversion time, and greatly reducing the difficulty of use for users. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the weight transformation method for a large language model provided by the present invention.

[0018] Figure 2 This is a flowchart illustrating step 130 in the large language model weight conversion method provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the weight self-processing process based on the ideas of dimensional tensors and reconstructed graphs provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the structure of the weight conversion device for the large language model provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] With the booming development of the open-source community, Large Language Model (LLM) technologies have entered a phase of rapid development. This trend has also spurred increasing demand for innovation in LLM technology from enterprises and research institutions. Considering that most teams cannot afford the high costs of pre-training large models from scratch, new structural designs or improvements to inference / training processes based on open-source model weights are currently the mainstream innovation model for large language models. Among these, HF (Hugging Face) format weights have become the preferred choice for large language model developers due to their strong versatility and good standardization.

[0024] Whether it's structural design or process improvement, developers inevitably have to customize and transform model weights in most cases. Here, customizing and transforming weights refers to modifying the original open-source model weights according to the developer's designed algorithm, exporting or converting them into new weights that conform to the application format of a certain development framework for use in new training or inference processes, ultimately meeting specific development needs.

[0025] Common custom processing and transformation operations include weight conversion between frameworks, sparse expert modification of models, and adding bias layers. However, due to the diversity of R&D needs, development frameworks, and model structures, developers often face problems such as long development time, large development workload, and difficulty in reusability when developing custom weight processing and transformation operations themselves.

[0026] Currently, existing methods for customizing and transforming weights in large language models are typically limited to developing specific functionalities and lack a unified design and implementation for most needs. They can be mainly categorized into the following two types: (1) Weight transformation method based on a specific framework These methods typically convert the weights of large language models from HF format to a specific format to enable the use of a particular AI development framework for model training or inference tasks. The need for format conversion often stems from considerations of both universality (large models from different sources have similar structures but different names) and efficiency (accelerating training / inference).

[0027] This type of method is commonly used in various distributed training frameworks for large models, including mainstream frameworks such as Megatron (Megatron-LM), Nemo (NeMo-Megatron), MindFormers (MindSporeTransformers), and ModelLink. The conversion operation in these frameworks mainly involves three steps: 1. Unifying the names of the HF format large model weights to names consistent with the framework's model definition; 2. Splitting the weights according to a parallel acceleration strategy (optional); 3. Converting to the specified weight format and exporting. This type of method is usually limited to scenarios using a specific framework and generally does not support direct conversion between frameworks (which requires indirect conversion through HF format weights) or other self-processing operations on weights. When a new model needs to be used, developers often need to spend a significant amount of time learning and understanding the new weight standard and manually developing the conversion code.

[0028] (2) Methods that specifically support certain weight processing operations These methods are primarily designed and implemented independently for certain special and relatively complex weight processing operations. The LLaMA-MoE open-source library focuses on the currently popular Mixture of Experts (MoE) architecture, designing and implementing various methods to convert LLaMA model weights into MoE version weights for related research. Its weight processing module mainly includes two core operations: dividing the original MLP (Multilayer Perceptron) layer weights into multiple expert units according to specific criteria, and adding routing units and initializing their weights.

[0029] Because they only consider a limited number of weight processing operations, these methods typically offer relatively complete functionality to support developers in quickly starting training or inference processes. For example, the LLaMA-MoE open-source library mentioned above provides not only the weight processing unit but also concrete model implementation code and complete model training capabilities. However, precisely because they need to consider too many factors, these methods struggle to integrate more types of weight processing operations and cannot support diverse user needs for weight self-processing and transformation.

[0030] To address the shortcomings of existing solutions, this invention proposes a novel method for self-processing and transforming the weights of large language models, which mainly solves the following two technical problems: (1) How to improve the versatility of custom weight processing methods to meet diverse R&D scenarios and reduce the difficulty of using the algorithm. This invention aims to design and implement an algorithm capable of performing numerous commonly used custom weight processing operations on a wider range of open-source weights for large language models. This ensures the algorithm can be applied to the vast majority of research and development scenarios (with virtually no code modification required), thereby addressing the limitation of existing methods in their limited application scenarios. The goal is to reduce the cost for developers and improve the efficiency of researching new structures for large language models. To ensure the feasibility of the method and control the difficulty of its design and implementation, achieving the above objectives mainly requires solving the following two sub-problems: first, how to enable the algorithm to load and process the weights of diverse, mainstream large language models; and second, how to avoid differences in weight processing operations and achieve efficient integration and unification of commonly used operations.

[0031] (2) How to reduce the weight transformation overhead and learning cost brought about by using specific training / inference frameworks Weights obtained through custom weight processing often only demonstrate their true value when used within a specific training / inference framework. However, some frameworks, especially those related to training, require an additional overhead of converting the weight format to a framework-supported format before the processed weights can be successfully loaded and used. Existing solutions often lack versatility, requiring developers to modify the algorithms before they can be applied to diverse real-world scenarios. This undoubtedly increases the overhead of weight conversion and the learning curve for developers. Therefore, to further reduce development costs and improve efficiency, it is a worthwhile research and solution to ensure that conversion algorithms can be applied to a sufficient number of development scenarios while minimizing the additional overhead of weight conversion.

[0032] To address the aforementioned problems, this invention provides a weight transformation method for large language models. Figure 1 This is a flowchart illustrating the weight transformation method for a large language model provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110, 120 and 130.

[0033] Step 110: Determine the model graph and weight dictionary of the large language model. The weight dictionary is in the form of key-value pairs. The key in the key-value pair is the name of the component module of the large language model, and the value in the key-value pair is the weight tensor of the large language model.

[0034] Specifically, firstly, the model graph and weight dictionary of the large language model are determined. Considering that commonly used large language model inference and training frameworks support loading and using model weights in HF format, this embodiment of the invention, consistent with most existing algorithms, also chooses to design and implement the corresponding algorithm only for HF format weights. This will enable this embodiment of the invention to significantly reduce the complexity of related algorithm design and implementation while ensuring high versatility.

[0035] The weight dictionary is in the form of key-value pairs. The key in the key-value pair is the name of the component module of the large language model, and the value in the key-value pair is the weight tensor of the large language model. The component module name may include transformer layer, lm_head layer and position encoding layer, etc., but this embodiment of the invention does not make specific limitations on this.

[0036] To improve the versatility of the model, this embodiment of the invention first needs to unify the structural definitions of different open-source models. This unification process has the following two major advantages: First, by defining a unified model structure, the algorithm can clearly identify the differences between the various component modules. This allows for more efficient and uniform operations on specific types of modules when performing certain weight processing operations. For example, when performing a segmentation operation on all Multilayer Perceptron (MLP) modules, users do not need to provide detailed naming information for all MLP modules, thus simplifying the operation process and improving processing efficiency. Second, a unified model structure definition facilitates the verification of the rationality of the generated weights after the weight processing operations are completed. Specifically, it ensures that the processed weights conform to the basic operating rules of large language models. This verification mechanism is necessary because only weights that conform to the rules can guarantee the stability and effectiveness of the model in subsequent operations.

[0037] In contrast, existing weight processing algorithms typically perform specific processing operations only for specific models, without considering the uniformity of model structure definitions. However, this targeted approach often lacks versatility and flexibility when dealing with a variety of different models. Therefore, this invention aims to solve this problem by unifying model structure definitions, thereby improving the versatility and efficiency of model processing.

[0038] Through detailed analysis, the current structure of large language models can be mainly divided into two categories: LLaMA-like and Mixtral-like. The LLaMA-like structure mainly includes four components: embedded_token, attention, MLP, and lm_head. The Mixtral-like structure, based on the above, adds two new components, router and expert, to the MLP structure. The unified model composition definition is shown in Table 1. To ensure the algorithm can correctly establish a one-to-one mapping between the weight modules input by the user and the aforementioned unified weight composition, the user needs to provide corresponding auxiliary information.

[0039] Table 1. Unified definition of model components Step 120: Perform a weight self-processing operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight.

[0040] Specifically, a weight self-processing operation can be performed on the weight tensors in the weight dictionary, and the model graph corresponding to the weight dictionary after the weight self-processing operation can be updated. The weight dictionary corresponding to the updated model graph can then be determined as the first format model weights.

[0041] Specifically, existing algorithms are designed from the outset to focus on performing pre-defined processing operations on the weights and specific components of a particular model. Therefore, they inherently lack the ability to autonomously identify model structure and weights, resulting in poor versatility. Once the structure of model weights, components, or processing operations change, these algorithms often struggle to adapt or even fail to function properly. Therefore, to significantly improve the versatility of algorithms and enable them to adapt to different models and diverse processing needs, additional information support must be introduced. This additional information will endow the algorithms with the ability to identify specific components and adapt to different weight processing operations, thereby enhancing their flexibility and adaptability when facing different models and tasks.

[0042] To enable the identification of component modules, this embodiment of the invention designed and implemented the unified model component definition shown in Table 1. Furthermore, to enable the identification of weight processing operations, this embodiment of the invention designed the following weight self-processing operation method based on the principles of maintaining tensors and reconstructing graphs.

[0043] Based on the preceding content, considering that various weight processing operations essentially change the values ​​and dimensions of specific tensors, the algorithm, after ensuring that the specified tensor can be identified through a unified model structure definition method, only requires the user to provide the modified tensor values ​​or tensor dimension information to complete the corresponding weight processing operation. Furthermore, considering that weight processing operations such as adding / deleting a layer of network structure modify the composition and order of the model's network structure, this embodiment of the invention also introduces the concept of graph reconstruction to ensure that weight processing can be executed correctly.

[0044] The concept of reconstructing the network graph essentially involves maintaining a network structure graph (model graph) throughout the entire process. This network structure graph contains not only the names and dimensions of all specific components, but also the order in which these components occur during runtime. Changing tensor values ​​and dimensions requires simultaneous modifications to this network structure graph. By maintaining the tensors and the model graph, most weight processing operations can be easily implemented.

[0045] The weight self-processing operation includes at least one of weight addition, weight deletion and weight modification operations, and the embodiments of the present invention do not specifically limit this.

[0046] It should be noted that existing weight self-processing algorithms are tightly bound to specific network structures and processing operations. Therefore, once the operation is successful, they can be used without considering the verification process of weight validity. However, the algorithm proposed in this embodiment of the invention introduces a greater degree of freedom in weight processing due to its improved versatility. Therefore, there is a higher probability that the processed weights will be unusable. To address this issue, this embodiment of the invention designs an input verification-based method to improve the usability of the processed weights. The core of this method lies in leveraging the regularity of the computational structure of a large language model and simulating the dimensionality changes of the input tensor during the computation process using the model graph and weight dimensions. Once it is found that the dimensionality derivation cannot proceed normally, it indicates that there is an unreasonable weight self-processing operation.

[0047] After verifying the validity of the model weights, this embodiment of the invention will save the dictionary structure of the stored weights as the first format model weights (HF format model weights) for use in subsequent model training or inference development scenarios.

[0048] Step 130: Based on the mapping relationship between the first format weight file and the second format weight file, perform weight transformation on the first format model weights to obtain the second format model weights.

[0049] Specifically, the weights of the first format model can be converted based on the mapping relationship between the first format weight file and the second format weight file to obtain the weights of the second format model, wherein the weights of the second format model are in zarr format.

[0050] Unlike common large language model inference frameworks that typically support directly loading and using HF format weights, some mainstream training frameworks (such as Megatron and Nemo-Megatron), in order to improve training efficiency and reduce implementation difficulty, usually only allow loading weight files in a specific format. This means they cannot directly use the HF format weights derived by the aforementioned weight self-processing algorithms. Existing algorithms generally address this issue by developing scripts to convert HF format to the target format for specific model structures. However, this approach easily leads to two problems, resulting in a surge in conversion costs.

[0051] (1) Low universality. Due to the differences in weights among open-source models, users may need to modify the conversion script according to the model structure they are using.

[0052] (2) High learning cost. Users not only need to understand in detail the specific model storage format supported by the training framework, but also need to master the definition of the model structure by the training framework and how the conversion script is implemented.

[0053] To further alleviate the aforementioned problems and reduce the overhead of weight conversion operations, this invention proposes a novel weight conversion method based on the Zarr format. First, to address the issue of low universality, this algorithm shifts the focus from the model weight format to the training framework; that is, it only considers converting HF format weights for various model structures to Zarr format model weights. Here, Zarr format is a weight format supported by the Megatron-LM framework. This decision is made primarily because Megatron-LM is currently the most mainstream training framework for large language models, possessing comprehensive functionality, extremely high training efficiency, and applicability to most training scenarios. Making this framework more readily usable with HF format weights from various model structures significantly reduces the design and implementation complexity of the algorithm while ensuring its high universality.

[0054] Secondly, to address the high learning curve for users, unlike existing algorithms that rely on developing code based on model structure definitions, this algorithm directly processes the model weights. Users only need to provide the mapping relationship between the weight components in two different formats to complete the weight conversion without developing additional code. This advantage stems from the fact that both the zarr and HF formats store weights using components as the basic unit. Compared to the former, users only need to understand the model definition within the Megatron-LM framework, without needing to learn the implementation logic of the conversion script. This approach, by avoiding the development of additional code, further reduces the overhead of converting weight formats.

[0055] The method provided in this invention, on the one hand, based on the concept of tensor maintenance and graph reconstruction, highly integrates many commonly used custom weight processing operations, thereby broadening the application scope of weights in different scenarios and effectively solving key problems such as the overly singular application scenarios and difficulty in reuse of existing algorithms; on the other hand, based on the mapping relationship between the first format weight file and the second format weight file, the method performs weight conversion on the first format model weights to obtain the second format model weights, completing the weight conversion operation without additional code development, reducing the overhead of weight conversion for large language models, reducing the weight conversion time, and greatly reducing the difficulty of use for users.

[0056] In related technologies, existing solutions suffer from the drawback of limited functionality: due to the need to consider additional factors such as specific model implementation and training / inference process design, the weight self-processing and transformation functions involved are generally relatively simple, and often cannot meet the diverse weight processing needs of large model developers at once.

[0057] Based on the above problems, the weight self-processing operation includes at least one of the following: weight addition operation, weight deletion operation, and weight modification operation; Step 120 includes at least one of the following: Perform the weight increment operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight increment operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. Perform the weight deletion operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight deletion operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

[0058] Specifically, the weight self-processing operation includes at least one of the following: weight addition operation, weight deletion operation, and weight modification operation.

[0059] Accordingly, weight increment operations can be performed on the weight tensors, and the model graph corresponding to the weight dictionary after the weight increment operation is updated. The weight dictionary corresponding to the updated model graph is then determined as the first-format model weights. Specifically, in the large language model, weight increment operations mainly include adding transformer layers and lm_head layers. For adding transformer layers, regarding the reconstruction graph (model graph), after the user provides the insertion position information (i.e., the layer number), the algorithm automatically inserts the new layer node in the model graph and modifies the subsequent layer numbers. Tensor maintenance only needs to consider the source of the values. Since users are unlikely to manually input values ​​in most cases, the algorithm only provides two methods: copying the weight values ​​of a layer with a specified index and randomly initializing the values. For adding the lm_head layer, considering the special nature of this layer, tensor value modification provides two methods: copying the embedded_tokens weight values ​​and randomly initializing the values. Regarding the reconstruction graph, since this layer can only be located at the last layer of the large language model, there is no need to provide position information; the algorithm can directly add a layer at the end of the model graph.

[0060] The algorithm performs a weight deletion operation on the weight tensors and updates the model graph corresponding to the weight dictionary after the operation. The updated model graph's weight dictionary is then used as the first-format model weights. It's important to note that the weight deletion operation is essentially the same as the weight addition operation, primarily involving the removal of transformer layers and lm_head layers (the embedded_tokens layer is mandatory in large language models). Similarly, for transformer layer removal, the algorithm automatically deletes the corresponding layer from the model graph after the user provides the deletion location information and automatically adjusts the numbering of subsequent layers. For tensor dimensions, the process is simpler; all tensors of that layer are extracted directly. Removing lm_head layers follows the same process, except that adjusting the numbering of subsequent layers is not required.

[0061] A weight modification operation is performed on the weight tensor, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is then determined as the first format model weights. Here, the weight modification operation includes at least one of the following: weight modification operation, tensor dimension modification of the weights of the organization modules in the large language model, and hybrid expert modeling.

[0062] The method provided in this embodiment of the invention includes at least one of weight addition, weight deletion and weight modification operations. The weight self-processing operations involved are of various types, thereby meeting the diverse weight processing needs of large model developers in one go.

[0063] Based on the above embodiments, the weight modification operation includes at least one of the following: weight modification operation, tensor dimension modification of the weights of the organization modules in the large language model, and hybrid expert modeling. The step of performing the weight modification operation on the weight tensor, updating the model graph corresponding to the weight dictionary after the weight modification operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes at least one of the following: The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight. Modify the tensor dimension of the organization module weights in the weight tensor, update the model graph corresponding to the weight dictionary after the tensor dimension of the organization module weights is modified, and determine the weight dictionary corresponding to the updated model graph as the first format model weights. The weight tensor is subjected to the hybrid expert modeling, and the model graph corresponding to the weight dictionary after hybrid expert modeling is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

[0064] Specifically, the weight modification operation includes at least one of the following: weight modification operation, tensor dimension modification of the weights of organizational modules in a large language model, and hybrid expert modeling.

[0065] Accordingly, the weight tensor is modified, and the model graph corresponding to the weight dictionary after the modification is updated. The weight dictionary corresponding to the updated model graph is then determined as the first format model weight.

[0066] Specifically, the weight modification operation is relatively simple, involving two operations: locating the tensor and modifying the weights. Tensor location falls into two categories: first, modifying the weights of identical components across all transformer layers; and second, modifying the weights of specific layers and components. For the former, the user only needs to provide a unified component name, and the algorithm can use the model graph to perform the location operation. For the latter, in addition to providing the component name, the specific layer number is also required for the algorithm to automatically locate the specific tensor. Regarding the weight modification operation, considering the impracticality of users manually outputting each weight, this algorithm only provides two assignment mechanisms: randomly generating weights and copying the weights of identical components from other layers. For the latter, the user also needs to provide the specific layer number of the component being copied.

[0067] Accordingly, the tensor dimensions of the organization module weights can be modified, and the model graph corresponding to the modified weight dictionary can be updated. The weight dictionary corresponding to the updated model graph is then determined as the first-format model weights. This operation is mainly used to modify the tensor dimensions of certain specific organization module weights, and it also includes the operations of locating and modifying tensors.

[0068] Since large language models with a MoE structure possess certain performance advantages in model inference, weight self-processing algorithms should be capable of modifying non-MoE structure model weights into MoE structure model weights. Hybrid expert modeling (MoE) can be performed on the weight tensor, and the model graph corresponding to the MoE weight dictionary can be updated. The weight dictionary corresponding to the updated model graph is then determined as the first-format model weights. To achieve this functionality, MoE can be decomposed into two operations: expert weight generation and route header generation.

[0069] Based on the above embodiments, the step of performing hybrid expert modeling on the weight tensor, updating the model graph corresponding to the weight dictionary after hybrid expert modeling, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes: Step 210: The original model weights of the multilayer perceptron in the weight tensor are uniformly divided into a preset number of expert head sets, or the original model weights are copied to obtain the preset number of expert head sets. Step 220: The weight dictionary after uniformly dividing or copying the original model weights is determined as the updated weight dictionary, and the multilayer perceptron in the model graph corresponding to the updated weight dictionary is replaced with a hybrid expert module, and the weight dictionary corresponding to the updated model graph is determined as the expert weights. Step 230: Generate the routing header of the gated network in the hybrid expert model by random initialization, and determine the first format model weights based on the expert weights and the routing headers.

[0070] Specifically, regarding the generation of expert weights, considering that reusing the original weights is more effective than randomly generating them, and that the tensor dimensions of each expert head are usually consistent, this embodiment of the invention achieves this by splitting or copying the original MLP structure. For the former, this embodiment of the invention directly and evenly divides the original model weights of the multilayer perceptron in the weight tensor into a specified number of expert head sets; for the latter, this embodiment of the invention generates a specified number of expert head sets by copying the original model weights.

[0071] Furthermore, the weight dictionary after uniformly dividing or copying the original model weights is determined as the updated weight dictionary, and the multilayer perceptron in the model graph corresponding to the updated weight dictionary is replaced with a hybrid expert module, and the weight dictionary corresponding to the updated model graph is determined as the expert weights.

[0072] Finally, besides generating expert weights, the non-MoE structure model weights also lack the weights of the crucial component, the routing header. Since routing header weights can usually only be obtained through post-training, this embodiment of the invention directly generates the routing header of the gated network in the hybrid expert model by random initialization when processing the tensor of the structure weights. Based on the expert weights and the routing header, the first format model weights are determined.

[0073] Based on the above embodiments, the step of modifying the tensor dimension of the organization module weights in the weight tensor, updating the model graph corresponding to the weight dictionary after the tensor dimension modification of the organization module weights, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes: Step 310: Determine a new tensor for the weights of the organization module, and assign values ​​to the new tensor to obtain the target weight tensor; Step 320: Remove the weights of the organization module and point the target weight tensor to the weight tensor of the organization module to obtain the target weight dictionary. Update the model graph corresponding to the target weight dictionary and determine the weight dictionary corresponding to the updated model graph as the first format model weights.

[0074] Specifically, this operation is mainly used to modify the tensor dimensions of certain specific module weights, and it includes two operations: tensor location and tensor modification. Tensor location is basically the same as the related operations in weight modification, and will not be elaborated here. For tensor modification, this algorithm implements it through the following three steps: First, determine the new tensor for the organization module weights. For example, a new tensor of the target dimension size can be randomly initialized for a specific component. Second, assign values ​​to the new tensor to obtain the target weight tensor. Currently, the algorithm only considers implementing three operations: random initialization of weights, using the original weights at the same positions, and assigning all values ​​to a single value given by the user. Third, remove the organization module weights and point the target weight tensor to the organization module's weight tensor to obtain the target weight dictionary. Update the model graph corresponding to the target weight dictionary, and determine the weight dictionary corresponding to the updated model graph as the first format model weights.

[0075] The existing technology has the following drawbacks: (1) weak universality: On the one hand, the relevant methods are often bound to specific training / inference frameworks, and the weight self-processing function adapted to a certain framework is often difficult to use directly in other frameworks; on the other hand, the algorithm is often limited to specific model structures and specific weight processing operations. Processing the weights of newly designed model structures often requires researchers to develop more adaptation code, which hinders the progress of performance verification of new model structures; (2) high learning cost: Due to the existence of the above two defects, when developers encounter the need to switch training / inference frameworks or use new weight self-processing operations, they have to spend a lot of extra time and energy to familiarize themselves with and master the new weight processing methods.

[0076] Based on the above embodiments, Figure 2 This is a flowchart illustrating step 130 in the large language model weight transformation method provided by the present invention, as shown below. Figure 2 As shown, step 130 includes: Perform the following steps for all model components in the first format weight file: Step 131: Obtain the model components of the first format weight file and the first weight of the model components; based on the mapping relationship, determine the second weight in the second format weight file corresponding to the first weight; if the tensor dimensions of the first weight and the second weight are consistent, determine the second weight based on the first weight and store the second weight; if the tensor dimensions of the first weight and the second weight are inconsistent, return an exception message. Step 132: Export all the second weights as the second format model weights.

[0077] Specifically, for convenience, users need to use the Megatron-LM training framework to store a copy of the current model weights in zarr format (at this time, the weight values ​​are all invalid), and provide a mapping relationship between component names in HF and Zarr (i.e., the various model components corresponding to the two formats).

[0078] Specifically, the model components and the first weights of the model components are obtained from the first format weight file. Then, based on the mapping relationship, the second weights in the second format weight file corresponding to the first weights are determined. If the tensor dimensions of the first weights and the second weights are consistent, the second weights are determined based on the first weights and stored. If the tensor dimensions of the first weights and the second weights are inconsistent, an exception message is returned.

[0079] Finally, if all model components in the first format weight file have been traversed, all second weights are exported as second format model weights (i.e., overwriting the original invalid values).

[0080] The method provided in this invention can efficiently convert HF format weights to zarr format weights supported by the Megatron-LM framework, while ensuring that the processed weights can be used in most R&D scenarios. Unlike existing solutions that require the development of model implementation code supporting both HF and Megatron-LM frameworks for conversion, this solution only requires the user to provide "the latter + model composition conversion relationship" to complete the corresponding operation, greatly reducing the difficulty of use for users.

[0081] Based on any of the above embodiments Figure 3 This is a schematic diagram of the weight self-processing process based on the ideas of dimensional tensors and reconstructed graphs provided by the present invention, as shown below. Figure 3 As shown, firstly, HF format weights are loaded, and then mapped to a unified model structure as shown in Table 1 (which includes mapping relationship information) based on the model composition, generating a model graph and a weight dictionary. Next, appropriate weight self-processing operations (weight addition, weight deletion, and weight modification) are selected for weight self-processing (using auxiliary information, namely mapping relationship information), modifying tensors and maintaining the model graph structure. Finally, a weight rationality check is performed. If the check fails, an error message is returned; if the check passes, the weights are stored in HF format.

[0082] In summary, compared with existing solutions, the method of this invention has the following three technical advantages: (1) Good versatility and strong reusability. Unlike existing solutions that are limited to specific model structures and specific weight processing operations, the weight self-processing algorithm proposed in this invention realizes the inheritance of most commonly used weight processing operations by using tensor maintenance and graph reconstruction ideas. While ensuring low difficulty of use, it greatly improves the versatility of the solution and can be applied to a wider range of new model structure design R&D scenarios.

[0083] (2) Low cost of use. Existing algorithms are limited by specific frameworks, model structures, and processing operations, requiring developers to bear high learning costs and modify the code accordingly in specific application scenarios. In contrast, the algorithm proposed in this invention has high versatility, and developers often do not need to understand the details of the algorithm implementation to complete the relevant operations. Although additional auxiliary information is required, the overall cost of use is lower.

[0084] (3) Low operating cost. The weight conversion algorithm based on the Zarr format proposed in this invention, compared with existing conversion methods, can greatly improve the weight conversion efficiency by directly processing the weights to avoid the overhead of additional model structure loading, while ensuring that it can be applied to most training scenarios.

[0085] The following describes the weight conversion device for a large language model provided by the present invention. The weight conversion device for a large language model described below and the weight conversion method for a large language model described above can be referred to in correspondence.

[0086] Based on any of the above embodiments, the present invention provides a weight transformation device for a large language model. Figure 4 This is a schematic diagram of the weight transformation device for a large language model provided by the present invention, as shown below. Figure 4 The device includes: The determining unit 410 is used to determine the model graph and weight dictionary of the large language model. The weight dictionary is in the form of key-value pairs, where the key in the key-value pair is the name of the component module of the large language model, and the value in the key-value pair is the weight tensor of the large language model. The weight self-processing unit 420 is used to perform weight self-processing operations on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight conversion unit 430 is used to convert the weights of the first format model based on the mapping relationship between the first format weight file and the second format weight file to obtain the weights of the second format model.

[0087] The apparatus provided in this invention, on the one hand, based on the concept of tensor maintenance and graph reconstruction, highly integrates many commonly used custom weight processing operations, thereby broadening the application scope of weights in different scenarios and effectively solving key problems such as the overly singular application scenarios and difficulty in reuse of existing algorithms; on the other hand, based on the mapping relationship between the first format weight file and the second format weight file, the method performs weight conversion on the first format model weights to obtain the second format model weights, completing the weight conversion operation without additional code development, reducing the overhead of weight conversion for large language models, reducing the weight conversion time, and greatly reducing the difficulty of use for users.

[0088] Based on any of the above embodiments, the weight self-processing operation includes at least one of weight addition operation, weight deletion operation, and weight modification operation; The weight self-processing unit 420 specifically includes: The weight increment operation unit is used to perform the weight increment operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight increment operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight deletion operation unit is used to perform the weight deletion operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight deletion operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight modification operation unit is used to perform the weight modification operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight modification operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight.

[0089] Based on any of the above embodiments, the weight modification operation includes at least one of the following: weight modification operation, tensor dimension modification of the weights of the organization modules in the large language model, and hybrid expert modeling. The weight modification operation unit specifically includes: The weight modification unit is used to perform the weight modification operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight modification operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The tensor dimension modification unit is used to modify the tensor dimension of the organization module weights in the weight tensor, update the model graph corresponding to the weight dictionary after the tensor dimension modification of the organization module weights, and determine the weight dictionary corresponding to the updated model graph as the first format model weights. The hybrid expert modeling unit is used to perform hybrid expert modeling on the weight tensor, update the model graph corresponding to the weight dictionary after hybrid expert modeling, and determine the weight dictionary corresponding to the updated model graph as the first format model weight.

[0090] Based on any of the above embodiments, the hybrid expert modeling unit is specifically used for: The original model weights of the multilayer perceptron in the weight tensor are uniformly divided into a preset number of expert head sets, or the original model weights are copied to obtain the preset number of expert head sets. The weight dictionary after uniformly dividing or copying the original model weights is determined as the updated weight dictionary, and the multilayer perceptron in the model graph corresponding to the updated weight dictionary is replaced with a hybrid expert module. The weight dictionary corresponding to the updated model graph is determined as the expert weights. The routing header of the gated network in the hybrid expert model is generated by random initialization, and the weights of the first format model are determined based on the expert weights and the routing header.

[0091] Based on any of the above embodiments, the tensor dimension modification unit is specifically used for: A new tensor for determining the weights of the organizational modules is obtained by assigning values ​​to the new tensor to obtain the target weight tensor. Remove the weights of the organization modules and point the target weight tensor to the weight tensor of the organization modules to obtain the target weight dictionary. Update the model graph corresponding to the target weight dictionary and determine the weight dictionary corresponding to the updated model graph as the first format model weights.

[0092] Based on any of the above embodiments, the weight transformation unit 430 is specifically used for: Perform the following steps for all model components in the first format weight file: Obtain the model components of the first format weight file and the first weight of the model components; based on the mapping relationship, determine the second weight in the second format weight file corresponding to the first weight; if the tensor dimensions of the first weight and the second weight are consistent, determine the second weight based on the first weight and store the second weight; if the tensor dimensions of the first weight and the second weight are inconsistent, return an exception message. Export all the second weights as the second format model weights.

[0093] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a weight transformation method for a large language model. This method includes: determining a model graph and a weight dictionary for the large language model, wherein the weight dictionary is in key-value pair form, the key in each key-value pair is the name of a component module of the large language model, and the value in each key-value pair is a weight tensor of the large language model; performing a weight self-processing operation on the weight tensor, updating the model graph corresponding to the weight dictionary after the weight self-processing operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights; and performing weight transformation on the first format model weights based on the mapping relationship between the first format weight file and the second format weight file to obtain the second format model weights.

[0094] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the weight transformation method for a large language model provided by the above methods. The method includes: determining a model graph and a weight dictionary of a large language model, wherein the weight dictionary is in the form of key-value pairs, the key in the key-value pairs is the name of the constituent module of the large language model, and the value in the key-value pairs is the weight tensor of the large language model; performing a weight self-processing operation on the weight tensor, and updating the model graph corresponding to the weight dictionary after the weight self-processing operation, determining the weight dictionary corresponding to the updated model graph as the first format model weights; and performing weight transformation on the first format model weights based on the mapping relationship between the first format weight file and the second format weight file to obtain the second format model weights.

[0096] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a weight transformation method for a large language model provided by the methods described above. The method includes: determining a model graph and a weight dictionary of the large language model, wherein the weight dictionary is in the form of key-value pairs, the key in each key-value pair is the name of a component module of the large language model, and the value in each key-value pair is a weight tensor of the large language model; performing a weight self-processing operation on the weight tensor, and updating the model graph corresponding to the weight dictionary after the weight self-processing operation, determining the weight dictionary corresponding to the updated model graph as a first format model weight; and performing a weight transformation on the first format model weight based on the mapping relationship between the first format weight file and the second format weight file to obtain the second format model weight.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weight transformation method for a large language model, characterized in that, include: Determine the model graph and weight dictionary of the large language model. The weight dictionary is in the form of key-value pairs, where the key in the key-value pair is the name of the component module of the large language model, and the value in the key-value pair is the weight tensor of the large language model. Perform a weight self-processing operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. Based on the mapping relationship between the first format weight file and the second format weight file, the weights of the first format model are converted to obtain the weights of the second format model.

2. The weight transformation method for a large language model according to claim 1, characterized in that, The weight self-processing operation includes at least one of the weight addition operation, weight deletion operation, and weight modification operation; The step of performing a weight self-processing operation on the weight tensor and updating the model graph corresponding to the weight dictionary after the weight self-processing operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights, includes at least one of the following: Perform the weight increment operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight increment operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. Perform the weight deletion operation on the weight tensor, update the model graph corresponding to the weight dictionary after the weight deletion operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

3. The weight transformation method for a large language model according to claim 2, characterized in that, The weight modification operation includes at least one of the following: weight modification operation, tensor dimension modification of the weights of the organization modules in the large language model, and hybrid expert modeling. The step of performing the weight modification operation on the weight tensor, updating the model graph corresponding to the weight dictionary after the weight modification operation, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes at least one of the following: The weight tensor is subjected to the weight modification operation, and the model graph corresponding to the weight dictionary after the weight modification operation is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight. Modify the tensor dimension of the organization module weights in the weight tensor, update the model graph corresponding to the weight dictionary after the tensor dimension of the organization module weights is modified, and determine the weight dictionary corresponding to the updated model graph as the first format model weights. The weight tensor is subjected to the hybrid expert modeling, and the model graph corresponding to the weight dictionary after hybrid expert modeling is updated. The weight dictionary corresponding to the updated model graph is determined as the first format model weight.

4. The weight transformation method for a large language model according to claim 3, characterized in that, The step of performing hybrid expert modeling on the weight tensor, updating the model graph corresponding to the weight dictionary after hybrid expert modeling, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes: The original model weights of the multilayer perceptron in the weight tensor are uniformly divided into a preset number of expert head sets, or the original model weights are copied to obtain the preset number of expert head sets. The weight dictionary after uniformly dividing or copying the original model weights is determined as the updated weight dictionary, and the multilayer perceptron in the model graph corresponding to the updated weight dictionary is replaced with a hybrid expert module. The weight dictionary corresponding to the updated model graph is determined as the expert weights. The routing header of the gated network in the hybrid expert model is generated by random initialization, and the weights of the first format model are determined based on the expert weights and the routing header.

5. The weight transformation method for a large language model according to claim 3, characterized in that, The step of modifying the tensor dimension of the organization module weights in the weight tensor, updating the model graph corresponding to the weight dictionary after the tensor dimension modification of the organization module weights, and determining the weight dictionary corresponding to the updated model graph as the first format model weights includes: A new tensor for determining the weights of the organizational modules is obtained by assigning values ​​to the new tensor to obtain the target weight tensor. Remove the weights of the organization modules and point the target weight tensor to the weight tensor of the organization modules to obtain the target weight dictionary. Update the model graph corresponding to the target weight dictionary and determine the weight dictionary corresponding to the updated model graph as the first format model weights.

6. The weight transformation method for a large language model according to any one of claims 1 to 5, characterized in that, The step of performing weight transformation on the first format model weights based on the mapping relationship between the first format weight file and the second format weight file to obtain the second format model weights includes: Perform the following steps for all model components in the first format weight file: Obtain the model components of the first format weight file and the first weight of the model components; based on the mapping relationship, determine the second weight in the second format weight file corresponding to the first weight; if the tensor dimensions of the first weight and the second weight are consistent, determine the second weight based on the first weight and store the second weight; if the tensor dimensions of the first weight and the second weight are inconsistent, return an exception message. Export all the second weights as the second format model weights.

7. A weight transformation device for a large language model, characterized in that, include: A determining unit is used to determine the model graph and weight dictionary of a large language model. The weight dictionary is in the form of key-value pairs, where the key in each key-value pair is the name of the component module of the large language model, and the value in each key-value pair is the weight tensor of the large language model. The weight self-processing unit is used to perform weight self-processing operations on the weight tensor, update the model graph corresponding to the weight dictionary after the weight self-processing operation, and determine the weight dictionary corresponding to the updated model graph as the first format model weight. The weight conversion unit is used to convert the weights of the first format model based on the mapping relationship between the first format weight file and the second format weight file to obtain the weights of the second format model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the weight transformation method for the large language model as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the weight transformation method for a large language model as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the weight transformation method for a large language model as described in any one of claims 1 to 6.

Images