Model conversion method and device and storage medium
By using the model state dictionary and preset matching rules of the target model structure in the model transformation, the model storage format version dependence problem in the prior art is solved, and the adaptability and efficiency of the model transformation are achieved.
Patent Information
- Application Number
- CN202411918801.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-18
AI Technical Summary
The strong dependence of the specific version of the model storage format in the prior art leads to cumbersome model conversion process, affects flexibility and user experience, and reduces the efficiency of model migration and deployment.
The target model structure is determined by a model state dictionary based on the target model structure, segmenting the source model structure, generating model slicing results, and using preset matching rules to convert the slicing results into the target model structure, realizing the adaptive transformation of the model.
Improve the adaptability of model conversion, making the conversion process independent of specific versions, reduces the user's manual configuration needs, reduces the error rate, and improves the efficiency of model migration and deployment.
Smart Images
Figure CN119848118A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a model conversion method, device and storage medium. Background Art
[0002] Current deep learning models usually have different model storage formats, and different model storage formats are generally applied to different frameworks. Therefore, there is a need to convert the model to a specific format so that the model can run under different frameworks. However, there are obvious limitations in the existing technology for model format conversion, which is mainly manifested in the strong dependence on a specific version of the model storage format. After the version is switched, manual adjustments are required based on the switched version. These problems seriously affect the flexibility of model conversion and user experience, as well as the efficiency of model migration and deployment. Summary of the invention
[0003] In view of this, the present disclosure proposes a model conversion method, device and storage medium.
[0004] According to one aspect of the present disclosure, a model conversion method is provided. The method is used in a first processor, and the method includes:
[0005] Determining a target model structure based on a model state dictionary corresponding to the target model structure, wherein the model state dictionary stores parameters of the model in a dictionary form;
[0006] The source model structure is segmented based on the target model structure to obtain a model segmentation result;
[0007] Based on the target model structure and the model segmentation result, the model to be converted is converted from the source model structure to the target model structure to obtain a converted model, and the converted model is used to execute the text task on the second processor.
[0008] In a possible implementation, the target model structure includes a key and path information corresponding to the key, and determining the target model structure includes:
[0009] The model state dictionary under the target model structure is traversed to obtain the key corresponding to the parameter in the model state dictionary as the key, and the path traversed to the key in the model state dictionary is used as the path information corresponding to the key.
[0010] In a possible implementation, the source model structure is segmented based on the target model structure to obtain a model segmentation result, including:
[0011] The hierarchical structure in the source model structure is segmented based on a pipeline segmentation strategy, and a plurality of first model files are generated based on the segmented hierarchical structure, wherein the pipeline segmentation strategy is determined based on the target model structure;
[0012] A model segmentation result is obtained based on the first model file.
[0013] In a possible implementation, the plurality of first model files are sequentially distributed to different processors for pipeline parallel processing, and the pipeline segmentation strategy includes the scale of the pipeline parallelism.
[0014] In a possible implementation, the model segmentation result includes a plurality of second model files, and the model segmentation result is obtained based on the first model file, including:
[0015] The dimensions of the parameters in the first model file are segmented based on a tensor segmentation strategy, and a plurality of second model files are generated based on the segmented parameters. The tensor segmentation strategy is determined based on the target model structure.
[0016] In a possible implementation, based on the target model structure and the model segmentation result, the model to be converted is converted from the source model structure to the target model structure to obtain the converted model, including:
[0017] Assign the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules;
[0018] The converted model is obtained based on the assigned target model structure.
[0019] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0020] Determining whether the same path exists in the target model structure according to the path corresponding to the parameter in the second model file;
[0021] In response to the existence of the same path in the target model structure, the value of the corresponding parameter in the second model file is assigned to the key corresponding to the same path in the target model structure.
[0022] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0023] For a first module in the model to be converted, determining a path including a preset string in a target model structure, the first module including an embedding layer;
[0024] Assign the value of the parameter of the first module in the second model file to the key corresponding to the path including the first preset character string in the target model structure.
[0025] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0026] For a second module in the model to be converted, determining a path corresponding to the second module in the target model structure based on the number of layers of the model to be converted, the position of the second module in the model to be converted, and a preset field, wherein the second module includes a normalization layer;
[0027] Assign the value of the parameter of the second module in the second model file to the key corresponding to the second module in the target model structure.
[0028] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0029] Verify the data format of the last parameter that is not assigned a value in the second model file;
[0030] In response to the data format of the last unassigned parameter meeting the requirements for the third module in the target model structure, the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure, and the third module includes an output layer.
[0031] In a possible implementation, the model to be converted includes a model that complies with the Hugging Face framework format, and the converted model includes a model that complies with the Megatron framework format.
[0032] According to another aspect of the present disclosure, a model conversion device is provided. The device is used for a first processor, and the device includes:
[0033] A determination module, configured to determine a target model structure based on a model state dictionary corresponding to the target model structure, wherein the model state dictionary stores parameters of the model in a dictionary form;
[0034] A segmentation module is used to segment the source model structure based on the target model structure to obtain a model segmentation result;
[0035] The conversion module is used to convert the model to be converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain the converted model, and the converted model is used to execute the text task on the second processor.
[0036] In a possible implementation, the target model structure includes a key and path information corresponding to the key, and a determination module is used to:
[0037] The model state dictionary under the target model structure is traversed to obtain the key corresponding to the parameter in the model state dictionary as the key, and the path traversed to the key in the model state dictionary is used as the path information corresponding to the key.
[0038] In a possible implementation, the segmentation module is used to:
[0039] The hierarchical structure in the source model structure is segmented based on a pipeline segmentation strategy, and a plurality of first model files are generated based on the segmented hierarchical structure, wherein the pipeline segmentation strategy is determined based on the target model structure;
[0040] A model segmentation result is obtained based on the first model file.
[0041] In a possible implementation, the plurality of first model files are sequentially distributed to different processors for pipeline parallel processing, and the pipeline segmentation strategy includes the scale of the pipeline parallelism.
[0042] In a possible implementation, the model segmentation result includes a plurality of second model files, and the model segmentation result is obtained based on the first model file, including:
[0043] The dimensions of the parameters in the first model file are segmented based on a tensor segmentation strategy, and a plurality of second model files are generated based on the segmented parameters. The tensor segmentation strategy is determined based on the target model structure.
[0044] In a possible implementation, the conversion module is used to:
[0045] Assign the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules;
[0046] The converted model is obtained based on the assigned target model structure.
[0047] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0048] Determining whether the same path exists in the target model structure according to the path corresponding to the parameter in the second model file;
[0049] In response to the existence of the same path in the target model structure, the value of the corresponding parameter in the second model file is assigned to the key corresponding to the same path in the target model structure.
[0050] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0051] For a first module in the model to be converted, determining a path including a preset string in a target model structure, the first module including an embedding layer;
[0052] Assign the value of the parameter of the first module in the second model file to the key corresponding to the path including the first preset character string in the target model structure.
[0053] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0054] For a second module in the model to be converted, determining a path corresponding to the second module in the target model structure based on the number of layers of the model to be converted, the position of the second module in the model to be converted, and a preset field, wherein the second module includes a normalization layer;
[0055] Assign the value of the parameter of the second module in the second model file to the key corresponding to the second module in the target model structure.
[0056] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0057] Verify the data format of the last parameter that is not assigned a value in the second model file;
[0058] In response to the data format of the last unassigned parameter meeting the requirements for the third module in the target model structure, the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure, and the third module includes an output layer.
[0059] In a possible implementation, the model to be converted includes a model that complies with the Hugging Face framework format, and the converted model includes a model that complies with the Megatron framework format.
[0060] According to another aspect of the present disclosure, a model conversion device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0061] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.
[0062] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0063] According to the embodiment of the present application, by segmenting the source model structure based on the target model structure to obtain the model segmentation result, the model to be converted is converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain the converted model, which can improve the adaptability of the model conversion, so that the model conversion process does not depend on a specific version, and through the optimized model conversion process, no manual configuration by the user is required, the error rate is reduced, and the applicability and flexibility of the solution are improved. The method can be used for the first processor, and the converted model is obtained by the first processor, so that the converted model is used to execute text tasks on the second processor, which can improve the efficiency of model migration and deployment and save the hardware resources of the processor.
[0064] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0066] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application.
[0067] Figure 2 A flow chart of a model conversion method according to an embodiment of the present application is shown.
[0068] Figure 3 A flow chart of a model conversion method according to an embodiment of the present application is shown.
[0069] Figure 4 A structural diagram of a model conversion device according to an embodiment of the present application is shown.
[0070] Figure 5 It is a block diagram of a device 1900 for model conversion according to an exemplary embodiment. DETAILED DESCRIPTION
[0071] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0072] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0073] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0074] Current deep learning models usually have different model storage formats, and different model storage formats are generally applied to different frameworks. Therefore, there is a need to convert the model to a specific format so that the model can run under different frameworks. However, there are obvious limitations in the existing technology for model format conversion, which is mainly manifested in the strong dependence on a specific version of the model storage format. After the version is switched, manual adjustments are required based on the switched version. These problems seriously affect the flexibility of model conversion and user experience, as well as the efficiency of model migration and deployment.
[0075] In view of this, the present application provides a model conversion method, device and storage medium. The method of the embodiment of the present application determines the target model structure based on the model state dictionary corresponding to the target model structure, segments the source model structure based on the target model structure, obtains the model segmentation result, and converts the model to be converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain the converted model, which can improve the adaptability of the model conversion. For any version of the target model structure, the method provided by the embodiment of the present application can be used to complete the conversion, so that the process of model conversion does not depend on a specific version, and through the optimized model conversion process, the target model structure can be automatically determined, and the conversion is automatically realized without manual configuration by the user, which reduces the error rate and improves the applicability and flexibility of the solution. And the method can be used for the first processor, and the converted model is obtained by the first processor, so that the converted model is used to perform text tasks on the second processor, which can be applied to the scene where the first processor and the second processor run different model structures respectively, and the model migration needs to be performed between each other. The efficiency of model migration and deployment can be improved, and the hardware resources of the processor can be saved.
[0076] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application is shown. Figure 1 As shown, the model conversion system of the embodiment of the present application can be applied to the scenario of converting the storage format of the deep learning model, and the deep learning model can be used to perform text tasks. For example, if you download a model on the Hugging Face platform and want to use the Megatron framework to perform a fine-tuning task, you need to first convert the downloaded model into a model that conforms to the corresponding version under the Megatron framework, so that the converted model can be loaded into the Megatron framework to perform the fine-tuning task.
[0077] Among them, the deep learning model to be converted (i.e., the model to be converted in the figure) can be a model for running under the Hugging Face framework, which conforms to the format of the Hugging Face framework.
[0078] The model conversion system can obtain the model to be converted, and perform format conversion on the model to be converted based on the target model structure (such as the model structure under the Megatron framework) to obtain a converted model, and the converted model can be a model that conforms to the Megatron framework format.
[0079] The model conversion system can be applied to a first processor, and the first processor can send the converted model to a second processor, so that the converted model can be run on the second processor to perform a text task (such as text training or text reasoning task).
[0080] Since Megatron is a framework specifically designed for training large-scale deep learning models, it has excellent performance and efficiency. Converting a model that conforms to the Hugging Face framework format to a model that conforms to the Megatron framework format to perform training or reasoning tasks on the Megatron framework can bring better training and reasoning performance for large-scale deep learning models. Furthermore, in a scenario where the Hugging Face framework runs on the first processor and the Megatron framework runs on the second processor, by migrating the model, the resources of the first processor can also be saved and the operating efficiency of the second processor can be improved.
[0081] The first processor may be a processor for performing model format conversion, and the second processor may be a processor for executing text tasks using the converted model. The first processor and the second processor may be the same or different processors, for example, they may be a central processing unit (CPU), a graphics processing unit (GPU), etc. The first processor and the second processor may be processors in a terminal device or a server. The first processor and the second processor may be processors in the same terminal device or server, or may belong to different terminal devices or servers. The terminal device may be any one or more of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), and a vehicle-mounted device. The embodiments of the present disclosure do not impose any special restrictions on the specific type of the terminal device, and it may have a wired or wireless communication function.
[0082] The server can be located locally or in the cloud, and can be a physical device or a virtual device, such as a virtual machine, a container, etc., and has a wireless communication function, wherein the wireless communication function can be set in the chip (system) or other parts or components of the server. The wireless communication function can be realized, for example, through mobile communication technologies such as 2G / 3G / 4G / 5G, as well as Wi-Fi, Bluetooth, frequency modulation (FM), digital radio, satellite communication, etc. Communication can also be carried out through wired connection to achieve interaction with other devices.
[0083] Figure 2 FIG. 4 is a flow chart showing a model conversion method according to an embodiment of the present application. The method may be used in a first processor, such as Figure 2 As shown, the method includes:
[0084] Step S201 : determining a target model structure based on a model state dictionary corresponding to the target model structure.
[0085] The target model structure is the converted model structure, for example, the model storage structure under the corresponding version of the Megatron framework.
[0086] The target model structure can be represented by a model state dictionary, in which the parameters of the model can be stored in a dictionary form. For example, the model state dictionary can be a structure of a dictionary nested in a dictionary, and the parameters are stored in key-value pairs, where in the last layer of the dictionary, the "key" in the key-value pair can be a parameter identifier (such as a parameter name), and the "value" is the parameter (the parameter here is not assigned a value), and in each layer before the last layer, the "key" in the key-value pair can be a unique identifier, and the "value" indicates the next layer of the dictionary.
[0087] For example, the model includes x layers, each layer includes y modules, and each module includes z parameters. Then the first-layer dictionary includes x key-value pairs, each key is the unique identifier of a layer, and the value of the key indicates a second-layer dictionary. Each second-layer dictionary includes y key-value pairs, each key is the unique identifier of a module, and the value of the key indicates a third-layer dictionary. Each third-layer dictionary includes z key-value pairs, each key is the unique identifier of a parameter, and the value of the key indicates a parameter.
[0088] The parameters of the model can include hyperparameters (args parameters) and tensor parameters (tensor parameters). The args parameters are fixed parameters during model training or inference, and the tensor parameters are the model's weights, biases, and other parameters that can be learned during model training.
[0089] The target model structure may include keys and path information corresponding to the keys. In S201, it is possible to:
[0090] The model state dictionary under the target model structure is traversed to obtain the key corresponding to the parameter in the model state dictionary as the key, and the path traversed to the key in the model state dictionary is used as the path information corresponding to the key.
[0091] The parameters in the model state dictionary are the parameters of the above-mentioned model. For example, the model state dictionary can be recursively traversed, with the key of each parameter therein as the key, and the nested path of the dictionary of the key obtained by recursive traversal as the path information corresponding to the key. Taking the model state dictionary mentioned in the above text as an example, when a parameter name (key) is obtained by recursive traversal, the path information of the parameter name obtained recursively indicates which layer of the x layers, which module of the y modules, and which parameter of the z parameters the parameter belongs to, and the path information can be represented by the "key" in the model state dictionary passed during the traversal process. That is, the target model structure may include the "key" (such as the parameter name) of a parameter in the model state dictionary, and the path information of traversing the model state dictionary to obtain the "key".
[0092] The target model structure can be used as the basis for subsequent saving of the model under the Megatron framework.
[0093] The model structures under different versions of the Megatron framework can correspond to different model state dictionaries. Therefore, based on the above method, the model structures under different versions of the Megatron framework can be adaptively identified and extracted, so that the model conversion process does not need to rely on a specific version of the Megatron framework, and can also be compatible with multiple third-party versions based on the Megatron framework for secondary development, and has wide compatibility.
[0094] Step S202, segmenting the source model structure based on the target model structure to obtain a model segmentation result.
[0095] The source model structure is the model structure before conversion, for example, the model storage structure under the Hugging Face framework.
[0096] Taking the Hugging Face model structure as the source model structure and the Megatron model structure as the target model structure as an example, the reason why the Hugging Face model structure is different from the Megatron model structure is that the Megatron framework introduces the pipeline parallelism and tensor parallelism mechanisms. Therefore, the source model structure can be divided from these two levels to meet the model structure requirements under the Megatron framework.
[0097] The pipeline can be split at the parallel level first, so that each pipeline stage after the split can be executed in parallel on different processors to improve the computing efficiency. In S202, it is possible to:
[0098] The hierarchical structure in the source model structure is segmented based on the pipeline segmentation strategy, and a plurality of first model files are generated based on the segmented hierarchical structure;
[0099] A model segmentation result is obtained based on the first model file.
[0100] Among them, the pipeline segmentation strategy can be determined based on the target model structure, and the pipeline segmentation strategy includes the scale of pipeline parallelism. For example, the scale of pipeline parallelism (pipeline parallel size, indicating how many pipelines can be executed in parallel) can be determined based on the version of the Megatron framework corresponding to the target model structure. The larger the pipeline parallel scale, the more the number of first model files generated after the corresponding segmentation can be. Multiple first model files can be used to be sequentially distributed to different processors for pipeline parallel processing.
[0101] Taking the source model structure as a 32-layer transformer model as an example, if the pipeline parallel size is 4, the 32-layer transformer model can be divided into 4 pipeline stages, where each pipeline stage includes 8 layers of transformer (layers 0-7, 8-15, 16-23, and 23-31 respectively), and each pipeline stage can generate a first model file. Since the transformer model has an embedding layer before layer 0, the embedding layer can be divided into the first pipeline stage; and the output layer connected to the transformer model after layer 31 and the regularization of the output layer can be divided into the fourth pipeline stage.
[0102] Next, based on the pipeline parallel segmentation result, the tensor parameters can be further segmented at the tensor parallel level so that the parameters can be processed in parallel on different processors to further improve the computing efficiency. The above-mentioned generation of multiple first model files based on the segmented hierarchical structure may include:
[0103] The dimensions of the parameters in the first model file are segmented based on a tensor segmentation strategy, and a plurality of second model files are generated based on the segmented parameters.
[0104] The parameters in the first model file can be tensor parameters.
[0105] The tensor splitting strategy may be determined based on the target model structure. The tensor splitting strategy may include the size of tensor parallelism (tensor parallel size, indicating how many blocks the tensor is split into for parallel processing), for example, the tensor parallel size may be predetermined based on the target model structure.
[0106] For example, based on the version of the Megatron framework corresponding to the target model structure, the parameters of different modules in the first model file can be split in different dimensions according to a predetermined tensor parallel size. The larger the tensor parallel scale, the more second model files can be generated after the corresponding split. Multiple second model files can be used to be sequentially distributed to different processors for tensor parallel processing.
[0107] Step S203: based on the target model structure and the model segmentation result, the model to be converted is converted from the source model structure to the target model structure to obtain a converted model.
[0108] The model segmentation result may include multiple second model files.
[0109] The model to be converted may include a model conforming to the Hugging Face framework format, and the converted model may include a model conforming to the Megatron framework format. The converted model may be used to perform a text task, such as a text training task or a text reasoning task, on the second processor.
[0110] According to the embodiment of the present application, by segmenting the source model structure based on the target model structure to obtain the model segmentation result, the model to be converted is converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain the converted model, which can improve the adaptability of the model conversion, so that the model conversion process does not depend on a specific version, and through the optimized model conversion process, no manual configuration by the user is required, the error rate is reduced, and the applicability and flexibility of the solution are improved. The method can be used for the first processor, and the converted model is obtained by the first processor, so that the converted model is used to execute text tasks on the second processor, which can improve the efficiency of model migration and deployment and save the hardware resources of the processor.
[0111] In S203, regular expressions and exclusion methods can be used to determine the paths of various model components in the target model structure to dynamically match the various components of the model (i.e., model parameters), so that the converted model parameters are correct and unique in the target model structure, and the universality and flexibility of the model conversion process are improved.
[0112] Figure 3 FIG. 1 is a flow chart showing a model conversion method according to an embodiment of the present application. Figure 3 As shown, step S203 may include:
[0113] Step S301, assigning the values of the parameters in the model segmentation result to the corresponding keys in the target model structure based on the preset matching rules.
[0114] The preset matching rules may include preset regular expressions and exclusion rules. Since the paths and parameter values corresponding to each module in the second model file usually appear only once in the target model structure, the corresponding path can be found in the target model structure based on the preset matching rules, and the parameter value is assigned to the key under the corresponding path in the target model.
[0115] In S301, you can:
[0116] Determine whether the same path exists in the target model structure according to the path corresponding to the parameter in the second model file; and in response to the existence of the same path in the target model structure, assign the value of the corresponding parameter in the second model file to the key corresponding to the same path in the target model structure.
[0117] As described above, the target model structure obtained in step S201 includes keys and path information corresponding to the keys. If the path corresponding to a parameter in the second model file and a path information in the target model structure are the same path, the value of the parameter is assigned to the key corresponding to the path information, so that the parameter corresponding to the "key" (for example, parameter name) in the target model structure is assigned.
[0118] Here, a regular expression may be used to match the path corresponding to the parameter in the second model file in the target model structure to determine whether the same path exists.
[0119] For modules whose model names usually appear only once in the conventional Megatron framework, a preset string can be used to match them in the target model structure. In S301, it is also possible to:
[0120] For the first module in the model to be converted, a path including a preset string is determined in the target model structure; and the value of the parameter of the first module in the second model file is assigned to the key corresponding to the path including the first preset string in the target model structure.
[0121] The first module may include an embedding layer. For example, the embedding layer in the transformer model usually only appears once in a conventional model structure, so the corresponding path in the target model structure can be matched based on the string corresponding to the module name, such as "embedding", using a regular expression containing the string. The matched path can locate a module corresponding to the first module in the target model structure, and one or more parameter values of the first module can be assigned to the key corresponding to the corresponding parameter in the corresponding module in the target model structure based on the parameter name.
[0122] For modules whose names may be partially modified in different versions of the Megatron framework, you can use preset fields to match them in the target model structure. In S301, you can also:
[0123] For the second module in the model to be converted, determine the path corresponding to the second module in the target model structure based on the number of layers of the model to be converted, the position of the second module in the model to be converted, and the preset field; assign the value of the parameter of the second module in the second model file to the key corresponding to the second module in the target model structure.
[0124] Among them, the second module may include a normalization layer. For example, for the normalization layer (layernorm) after the multilayer perceptron (multilayerperceptron, MLP), it may be modified to other names containing the "norm" field for storage in some versions of the Megatron framework. Therefore, matching can be performed based on the number of layers of the model to be converted, the position of the normalization layer in the model to be converted, and the "norm" field, rather than being limited to matching using the string "layernorm". For example, the model to be converted has x layers, and the normalization layer (second module) is in the nth layer. The path (there may be multiple) that matches the nth layer in the x layers can be determined in the target model structure, and the path containing the "norm" field can be matched therefrom, that is, a module corresponding to the second module is found in the target model structure, and one or more parameter values of the second module can be assigned to the key corresponding to the corresponding parameter in the corresponding module in the target model structure based on the parameter name.
[0125] For module parameters that may have completely different names in different versions of the Megatron framework, you can also match based on the exclusion method. In S301, you can also:
[0126] The data format of the last unassigned parameter in the second model file is verified; in response to the data format of the last unassigned parameter meeting the requirements for the third module in the target model structure, the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure.
[0127] Among them, the third module may include an output layer. The data format of the parameter includes the dimension of the parameter, and the data format meets the requirements that the size of the dimension of the parameter meets the preset output dimension size of the output layer. For the corresponding keys in the output layer, the naming methods in different versions of the Megatron framework may be completely different, such as "output_layer", "lm_head", etc., so the assigned keys can be popped to remove the assigned keys, and the last unassigned key can be used as a possible outputlayer. The data format corresponding to the last parameter can be checked. If the data format meets the requirements of the output layer for the output dimension, the dictionary is recursively entered to determine the last unassigned key, and the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure (that is, the last unassigned key).
[0128] Through the above-mentioned process of automated model structure segmentation and parameter assignment and preservation, human intervention is reduced, the efficiency and accuracy of model conversion are improved, and at the same time, the source model structure and parameter settings can be more accurately maintained, further improving the performance of the converted model.
[0129] Step S302, obtaining a converted model based on the assigned target model structure.
[0130] The assigned key value can be saved to obtain the assigned target model structure, and the converted model is the model under the target model structure. The converted model can be sent to the second processor to execute the text task.
[0131] Figure 4 FIG. 1 shows a structural diagram of a model conversion device according to an embodiment of the present application. The device is used in a first processor, such as Figure 4 As shown, the device comprises:
[0132] A determination module 401 is used to determine the target model structure based on a model state dictionary corresponding to the target model structure, wherein the model state dictionary stores the parameters of the model in a dictionary form;
[0133] A segmentation module 402 is used to segment the source model structure based on the target model structure to obtain a model segmentation result;
[0134] The conversion module 403 is used to convert the model to be converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain a converted model, and the converted model is used to execute the text task on the second processor.
[0135] In a possible implementation, the target model structure includes keys and path information corresponding to the keys, and the determination module 401 is used to:
[0136] The model state dictionary under the target model structure is traversed to obtain the key corresponding to the parameter in the model state dictionary as the key, and the path traversed to the key in the model state dictionary is used as the path information corresponding to the key.
[0137] In a possible implementation, the segmentation module 402 is configured to:
[0138] The hierarchical structure in the source model structure is segmented based on a pipeline segmentation strategy, and a plurality of first model files are generated based on the segmented hierarchical structure, wherein the pipeline segmentation strategy is determined based on the target model structure;
[0139] A model segmentation result is obtained based on the first model file.
[0140] In a possible implementation, the plurality of first model files are sequentially distributed to different processors for pipeline parallel processing, and the pipeline segmentation strategy includes the scale of the pipeline parallelism.
[0141] In a possible implementation, the model segmentation result includes a plurality of second model files, and the model segmentation result is obtained based on the first model file, including:
[0142] The dimensions of the parameters in the first model file are segmented based on a tensor segmentation strategy, and a plurality of second model files are generated based on the segmented parameters. The tensor segmentation strategy is determined based on the target model structure.
[0143] In a possible implementation, the conversion module 403 is configured to:
[0144] Assign the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules;
[0145] The converted model is obtained based on the assigned target model structure.
[0146] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0147] Determining whether the same path exists in the target model structure according to the path corresponding to the parameter in the second model file;
[0148] In response to the existence of the same path in the target model structure, the value of the corresponding parameter in the second model file is assigned to the key corresponding to the same path in the target model structure.
[0149] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0150] For a first module in the model to be converted, determining a path including a preset string in a target model structure, the first module including an embedding layer;
[0151] Assign the value of the parameter of the first module in the second model file to the key corresponding to the path including the first preset character string in the target model structure.
[0152] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0153] For a second module in the model to be converted, determining a path corresponding to the second module in the target model structure based on the number of layers of the model to be converted, the position of the second module in the model to be converted, and a preset field, wherein the second module includes a normalization layer;
[0154] Assign the value of the parameter of the second module in the second model file to the key corresponding to the second module in the target model structure.
[0155] In a possible implementation, assigning the value of the parameter in the model segmentation result to the corresponding key in the target model structure based on a preset matching rule includes:
[0156] Verify the data format of the last parameter that is not assigned a value in the second model file;
[0157] In response to the data format of the last unassigned parameter meeting the requirements for the third module in the target model structure, the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure, and the third module includes an output layer.
[0158] In a possible implementation, the model to be converted includes a model that complies with the Hugging Face framework format, and the converted model includes a model that complies with the Megatron framework format.
[0159] According to the embodiment of the present application, by segmenting the source model structure based on the target model structure to obtain the model segmentation result, the model to be converted is converted from the source model structure to the target model structure based on the target model structure and the model segmentation result to obtain the converted model, which can improve the adaptability of the model conversion, so that the model conversion process does not depend on a specific version, and through the optimized model conversion process, no manual configuration by the user is required, the error rate is reduced, and the applicability and flexibility of the solution are improved. The method can be used for the first processor, and the converted model is obtained by the first processor, so that the converted model is used to execute text tasks on the second processor, which can improve the efficiency of model migration and deployment and save the hardware resources of the processor.
[0160] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0161] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.
[0162] The embodiment of the present disclosure also proposes a model conversion device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0163] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0164] Figure 51 is a block diagram of a device 1900 for model conversion according to an exemplary embodiment. For example, the device 1900 can be provided as a server or a terminal device. Figure 5 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0165] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.
[0166] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.
[0167] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0168] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0169] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0170] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0171] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0172] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0173] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0174] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0175] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A model conversion method, characterized in that: The method is used for a first processor, and the method comprises: Determining a target model structure based on a model state dictionary corresponding to the target model structure, wherein the model state dictionary stores parameters of the model in a dictionary form; Segmenting the source model structure based on the target model structure to obtain a model segmentation result; Based on the target model structure and the model segmentation result, the model to be converted is converted from the source model structure to the target model structure to obtain a converted model, and the converted model is used to execute the text task on the second processor.
2. The method according to claim 1, characterized in that The target model structure includes keys and path information corresponding to the keys, and determining the target model structure includes: The model state dictionary under the target model structure is traversed to obtain the key corresponding to the parameter in the model state dictionary as the key, and the path traversed to the key in the model state dictionary is used as the path information corresponding to the key.
3. The method according to claim 1, characterized in that: The segmenting of the source model structure based on the target model structure to obtain a model segmentation result includes: The hierarchical structure in the source model structure is segmented based on a pipeline segmentation strategy, and a plurality of first model files are generated based on the segmented hierarchical structure, wherein the pipeline segmentation strategy is determined based on the target model structure; A model segmentation result is obtained based on the first model file.
4. The method according to claim 3, characterized in that: The multiple first model files are used to be distributed to different processors in sequence for pipeline parallel processing, and the pipeline segmentation strategy includes the scale of pipeline parallelism.
5. The method according to claim 3, characterized in that: The model segmentation result includes a plurality of second model files, and obtaining the model segmentation result based on the first model file includes: The dimensions of the parameters in the first model file are segmented based on a tensor segmentation strategy, and a plurality of second model files are generated based on the segmented parameters, wherein the tensor segmentation strategy is determined based on the target model structure.
6. The method according to claim 1, characterized in that The step of converting the model to be converted from a source model structure to a target model structure based on the target model structure and the model segmentation result to obtain a converted model includes: Assign the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules; The converted model is obtained based on the assigned target model structure.
7. The method according to claim 6, characterized in that The assigning the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules includes: Determining whether the same path exists in the target model structure according to the path corresponding to the parameter in the second model file; In response to the existence of the same path in the target model structure, the value of the corresponding parameter in the second model file is assigned to the key corresponding to the same path in the target model structure.
8. The method according to claim 6, characterized in that The assigning the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules includes: For a first module in the model to be converted, determining a path including a preset string in the target model structure, the first module including an embedding layer; Assign the value of the parameter of the first module in the second model file to the key corresponding to the path including the first preset character string in the target model structure.
9. The method according to claim 6, characterized in that The assigning the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules includes: For a second module in the model to be converted, determining a path corresponding to the second module in the target model structure based on the number of layers of the model to be converted, the position of the second module in the model to be converted, and a preset field, wherein the second module includes a normalization layer; Assign the value of the parameter of the second module in the second model file to the key corresponding to the second module in the target model structure.
10. The method according to claim 6, characterized in that The assigning the values of the parameters in the model segmentation results to the corresponding keys in the target model structure based on the preset matching rules includes: Verify the data format of the last parameter that is not assigned a value in the second model file; In response to the data format of the last unassigned parameter meeting the requirements for the third module in the target model structure, the value of the last unassigned parameter is assigned to the key corresponding to the third module in the target model structure, and the third module includes an output layer.
11. The method according to claim 1, characterized in that The model to be converted includes a model that conforms to the Hugging Face framework format, and the converted model includes a model that conforms to the Megatron framework format.
12. A model conversion device, characterized in that: The device is used for a first processor, and the device includes: A determination module, configured to determine a target model structure based on a model state dictionary corresponding to the target model structure, wherein the model state dictionary stores parameters of the model in a dictionary form; A segmentation module, used to segment the source model structure based on the target model structure to obtain a model segmentation result; A conversion module is used to convert the model to be converted from a source model structure to a target model structure based on the target model structure and the model segmentation result to obtain a converted model, and the converted model is used to execute a text task on a second processor.
13. A model conversion device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 11 when executing the instructions stored in the memory.
14. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.