Adaptive low-precision model training method and device, and electronic equipment

Abstract

The present disclosure provides a self-adaptive low-precision model training method, which relates to the technical fields of large models, deep learning, intelligent cloud, model training and the like. The self-adaptive low-precision model training method comprises: generating a training image of a target model according to model configuration information, parallel configuration information and hardware environment information of the target model; determining a to-be-tested module according to the training image, performing benchmark testing based on a candidate low-precision path and a candidate benchmark precision path to obtain a strategy file; in an initialization stage of the target model, obtaining a strategy query result according to module information of the target module; in response to the strategy query result meeting a preset requirement, determining the candidate low-precision path as a target precision path, otherwise, determining the candidate benchmark precision path as the target precision path; and performing training of the target model according to the target precision path of the target module. The present disclosure achieves the purpose of adaptively binding different modules with corresponding precision paths and completing target model training.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of artificial intelligence technology such as large models, deep learning, intelligent cloud, and model training. It provides an adaptive low-precision model training method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Existing technologies typically employ a full low-precision strategy or require manual differentiation of low-precision training modules when training models with low precision. This results in poor flexibility and the inability to generate benefits from low-precision training.

[0003] Therefore, how to improve the flexibility of the model during low-precision training and achieve adaptive low-precision training has become an urgent technical problem to be solved. Summary of the Invention

[0004] According to a first aspect of this disclosure, an adaptive low-precision model training method is provided, comprising: generating a training profile of the target model based on model configuration information, parallel configuration information, and hardware environment information of the target model; determining a module to be tested based on the training profile, and performing benchmark testing on the module to be tested based on candidate low-precision paths and candidate benchmark precision paths, and obtaining a strategy file based on the benchmark test report; during the initialization phase of the target model, querying the strategy file based on the module information of the target module to obtain a strategy query result; in response to the strategy query result satisfying a preset requirement, determining the candidate low-precision path as the target precision path of the target module, otherwise determining the candidate benchmark precision path as the target precision path of the target module; and performing training of the target model based on the target precision path of the target module.

[0005] According to a second aspect of this disclosure, an adaptive low-precision model training apparatus is provided, comprising: a generation unit, configured to generate a training profile of the target model based on model configuration information, parallel configuration information, and hardware environment information of the target model; a benchmark testing unit, configured to determine a module to be tested based on the training profile, and perform benchmark testing on the module to be tested based on candidate low-precision paths and candidate benchmark precision paths, and obtain a strategy file based on the benchmark testing report; a query unit, configured to query the strategy file based on module information of the target module during the initialization phase of the target model, and obtain a strategy query result; a binding unit, configured to determine the candidate low-precision path as the target precision path of the target module in response to the strategy query result meeting a preset requirement, otherwise determine the candidate benchmark precision path as the target precision path of the target module; and a training unit, configured to perform training of the target model based on the target precision path of the target module.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0009] According to a sixth aspect of this disclosure, an autonomous vehicle is provided, including the electronic devices described above.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0016] Figure 5 This is a block diagram of an electronic device used to implement the adaptive low-precision model training method of the embodiments of this disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0018] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown in the figure, an adaptive low-precision model training method in this embodiment specifically includes the following steps:

[0019] S101. Generate a training profile of the target model based on the model configuration information, parallel configuration information, and hardware environment information of the target model.

[0020] S102. Determine the module to be tested based on the training profile, and perform benchmark testing on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path, and obtain the strategy file based on the benchmark test report.

[0021] S103. During the initialization phase of the target model, the strategy file is queried according to the module information of the target module to obtain the strategy query result.

[0022] S104. In response to the strategy query result meeting the preset requirements, the candidate low-precision path is determined as the target precision path of the target module; otherwise, the candidate baseline precision path is determined as the target precision path of the target module.

[0023] S105. Execute training of the target model according to the target accuracy path of the target module.

[0024] The adaptive low-precision model training method in this embodiment generates strategy files and binds target precision paths (especially low-precision paths) at the module level of the target model. This achieves the goal of adaptively binding different modules to corresponding precision paths and completing the training of the target model. It avoids the problem of directly using low-precision paths to cover all modules in the model without bringing any benefits. This improves the flexibility of model training. Furthermore, by completing the binding between precision paths and modules during the initialization phase of the target model, there is no need for repeated trial and error during the training process, which improves the stability of model training.

[0025] In this embodiment, when executing S101, a training profile of the corresponding target model can be generated based on the model structure, model components, distributed parallel parameters (tensor parallelism TP, expert parallelism / EP / expert tensor parallelism ETP), model sequence length, batch size, hardware type and attributes, low-precision recipe, etc., extracted from model configuration information, parallel configuration information, and hardware environment information.

[0026] The training profile obtained by executing S101 in this embodiment is used to determine the scope of modules for benchmark testing and precision path binding modules.

[0027] In this embodiment, if the target model is a VLM (Vision Language Model), the model components corresponding to the target model may include a language backbone component, an image encoding component, a modal projection component, and an expert computing component, and different components may include one or more modules.

[0028] In this embodiment, after executing S101 to obtain the training profile of the corresponding target model, S102 is executed to determine the module to be tested based on the training profile, and benchmark testing is performed on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path, and a strategy file is obtained based on the benchmark test report.

[0029] In this embodiment, when executing S102 to determine the module to be tested based on the training profile, the following implementation method can be adopted: in response to the target model not being loaded, a parallel layer is constructed based on the training profile; the module selected from the constructed parallel layer is determined as the module to be tested.

[0030] In other words, this embodiment determines the module to be tested by dynamically creating parallel layers using training profiles when the target model is not loaded, thus avoiding the problem of not being able to obtain the strategy file due to the target model not being loaded.

[0031] In addition, when executing S102 in this embodiment, the target model can be determined from the loaded models based on the training profile, that is, the model corresponding to the training profile in the loaded models is taken as the target model, and then the module selected from the target model is determined as the module to be tested.

[0032] In this embodiment, the candidate low-precision paths are FP8, FP4, INT8, etc., and the candidate reference precision paths are BF16, FP16, FP32, etc.; preferably, in this embodiment, the candidate low-precision path is FP8 and the candidate reference precision path is BF16.

[0033] Specifically, in this embodiment, when performing S102 benchmark testing of the module under test based on candidate low-precision paths and candidate benchmark precision paths, the implementation method can be as follows: A first test environment is set up based on fixed parameters and candidate low-precision paths; a second test environment is set up based on fixed parameters and candidate benchmark precision paths. In this embodiment, the fixed parameters can be parallel configuration parameters, input / output shapes, etc., obtained based on the training profile and the module under test. Benchmark testing is performed on the module under test in both the first and second test environments to obtain a first test result for the corresponding candidate low-precision path and a second test result for the corresponding candidate benchmark precision path. In this embodiment, the test result can be the module's running time. The measured speedup is obtained based on the first and second test results. A benchmark test report for the corresponding module under test is obtained based on the measured speedup and the module information of the module under test. The measured speedup is used to quantify the actual acceleration effect of the candidate low-precision path relative to the candidate benchmark path.

[0034] In this embodiment, the benchmark report may include fields such as module type (module_kind), TE communication or user buffer name (ub_name), parallel configuration parameters (TP or ETP), number of general matrix multiplications in the expert model (num_gemms), measured speedup (measured_speedup), and minimum token number threshold (min_tokens).

[0035] In this embodiment, when executing S102 to obtain the strategy file based on the benchmark test report, the following may also be included: obtaining multiple benchmark test reports obtained under different parallel configuration parameters, such as different reports generated under configurations such as TP=1, TP=2, EP=1; merging the multiple benchmark test reports obtained, and obtaining the strategy file based on the merging result.

[0036] Since different parallel configuration parameters will affect the benchmark test results of the module under test, this embodiment will perform multiple benchmark tests on the module under test according to different parallel configuration parameters when there are multiple parallel configuration parameters. In this case, multiple benchmark test reports will be merged and a strategy file will be obtained based on the merged results.

[0037] In this embodiment, when executing S102 to obtain the strategy file based on the benchmark test report, the following may also be included: in response to the target model being a visual language model, determining the model component to which the module to be tested belongs; aggregating the benchmark test reports of the modules to be tested belonging to the same model component, and obtaining the strategy file corresponding to different model components based on the aggregation result.

[0038] In other words, since the computational forms and numerical risks of different components in the visual language model are different, directly applying the strategy of the corresponding language backbone component (i.e., LLM) to the image coding component, modal projection component, or expert computation component may lead to negative optimization. Therefore, this embodiment obtains different strategy files based on the component type, thereby achieving the purpose of component-level strategy isolation.

[0039] In this embodiment, when executing S102 to obtain the strategy file based on the benchmark test report, the following may also be included: in response to the measured speedup ratio in the benchmark test report being greater than the preset speedup ratio threshold (speedup_threshold), the strategy file is obtained based on the benchmark test report. That is, in this embodiment, only the benchmark test report in the strategy file that meets certain requirements for the measured speedup ratio is saved, thereby improving the effectiveness of the obtained strategy file.

[0040] In this embodiment, after executing S102 to obtain the strategy file, S103 is executed during the initialization phase of the target model. Based on the module information of the target module, a query is performed in the strategy file to obtain the strategy query result.

[0041] In this embodiment, when executing S103, the module information of the target module obtained may include module type, module name, component type, parallel configuration parameters, number of general matrix multiplications, number of currently valid tokens, input / output shape, etc.

[0042] In this embodiment, when executing S103 to query the strategy file based on the module information of the target module and obtain the strategy query result, the following implementation method can be adopted: determine the target query rule based on the module type of the target module; extract the target module information corresponding to the determined target query rule from the module information; and query the strategy file based on the extracted target module information to obtain the strategy query result.

[0043] For example, if the target module is a Dense layer module, the determined target query rule can be to query based on the module type, the parallel configuration parameter TP, and the module name; if the target module is a hybrid expert model layer module, the determined target query rule can be to query based on the module type, the parallel configuration parameter ETP, and the number of general matrix multiplications.

[0044] In other words, this embodiment selects relevant information from the module information of the target module according to the module type of the target module to realize the strategy query, which can improve the query accuracy and query speed.

[0045] In this embodiment, when performing S103 to query the strategy file based on the module information of the target module and obtain the strategy query result, the following may also be included: determining the model component to which the target module belongs based on the module type of the target module; determining the target strategy file of the corresponding model component from the strategy file; and querying the determined target strategy file based on the module information of the target module to obtain the strategy query result.

[0046] In other words, in this embodiment, when the target module is determined to be a module in a certain model component of the visual language model, the target strategy file corresponding to the model component is first determined from the strategy file, and then a query is performed in the determined target strategy file according to the module information to obtain the strategy query result, so as to achieve the purpose of completing the strategy query under the component-level strategy isolation.

[0047] It is understood that when this embodiment performs S103 to query the determined target strategy file based on the module information, it can combine the above-mentioned method of extracting target module information based on the determined target query rules.

[0048] In this embodiment, after obtaining the strategy query result by executing S103, S104 is executed in response to the strategy query result meeting the preset requirements. If the result meets the preset requirements, the candidate low-precision path is determined as the target precision path of the target module; otherwise, the candidate baseline precision path is determined as the target precision path of the target module.

[0049] In this embodiment, when executing S104, if it is determined that the policy query result is found in the policy file, the preset requirements can be met.

[0050] In this embodiment, when executing S104, the following method can also be adopted: in response to the fact that the number of valid tokens of the target module is greater than the minimum token number threshold in the strategy query result and the measured speedup ratio in the strategy query result is greater than the preset speedup ratio threshold, the strategy query result is determined to meet the preset requirements.

[0051] In this embodiment, the number of valid tokens for the target module can be estimated based on parameters such as sequence length and batch size in the training profile, or the number of valid tokens can be obtained based on the actual input of the target module.

[0052] In this embodiment, after executing S104 to determine that the strategy query result meets the preset requirements, the candidate low-precision path is used as the target precision path, thereby binding it to the target module. Binding the candidate low-precision path to the target module means that during the training process of the target model, the calculation of the target module will use the low-precision path (e.g., FP8) for forward and backward propagation.

[0053] In this embodiment, if the strategy query result does not meet the preset requirements when executing S104, the accuracy path is backed up, and the candidate benchmark accuracy path is used as the target accuracy path, thereby binding it to the target module. Binding the candidate benchmark accuracy path to the target module means that during the training process of the target model, the calculation of the target module will use the benchmark accuracy path (e.g., BF16) for forward and backward propagation.

[0054] In this embodiment, after determining the target accuracy path in S104, S105 is executed to train the target model according to the target accuracy path of the target module.

[0055] In other words, after determining the target precision path corresponding to the target module in the target model, this embodiment can complete the training based on the target precision path of each target module during the actual training process, thus achieving the purpose of low-precision adaptation at the module level during model training.

[0056] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure. Figure 2 The diagram shows the structure of the adaptive low-precision model training system of this embodiment: The model training system of this embodiment includes a benchmark testing module, a policy generation module, a precision binding module, a training execution module, and a policy verification module.

[0057] The system comprises the following modules: a benchmarking module for benchmarking the module under test using candidate low-precision paths and candidate benchmark precision paths, generating a benchmarking report; a policy generation module for generating a policy file based on the benchmarking report generated by the offline benchmarking module, such as merging benchmarking reports and isolating policies at the component level; a precision binding module for binding the target module and its corresponding target precision path during model initialization; a training execution module for training the model based on the target precision path bound to the target module; and a policy verification module for recording metrics such as throughput, loss, and NaN / inf of the target module during training, and updating the policy file based on the recorded metrics.

[0058] For example, if it is determined from the recorded metrics that a certain module has not gained any benefit during training, this embodiment can update the policy file by re-benchmarking the target model or by increasing the speedup threshold in the policy file.

[0059] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure. Figure 3The flowchart of this embodiment during adaptive low-precision model training is shown: S301, Obtain a training profile based on the target model, the obtained training profile corresponding to the training environment information of the target model; S302, Determine the module to be tested based on the training profile, and perform benchmark testing on the module to be tested under the candidate low-precision path and the candidate baseline precision path; S303, Generate a strategy file based on the benchmark test report obtained from the benchmark test, the strategy file includes multiple strategies, each strategy corresponding to different modules to be tested, minimum token number threshold, parallel configuration parameters, measured speedup ratio, etc.; S304, Load the strategy file during the training initialization phase; S305, Query the strategy corresponding to the target module in the strategy file, and determine whether the strategy is hit and whether the hit strategy has a benefit; S306, In response to a strategy not being hit or the hit strategy having no benefit, the candidate baseline precision path is used as the target precision path of the target module; S307, In response to a strategy being hit and the hit strategy having a benefit, the candidate low-precision path is used as the target precision path of the target module; S308, Execute the training of the target model and perform a review.

[0060] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, an adaptive low-precision model training device 400 according to this embodiment includes:

[0061] The generation unit 401 is used to generate a training profile of the target model based on the model configuration information, parallel configuration information and hardware environment information of the target model.

[0062] The benchmark testing unit 402 is used to determine the module to be tested based on the training profile, and to perform benchmark testing on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path, and to obtain a strategy file based on the benchmark testing report.

[0063] The query unit 403 is used to query the strategy file according to the module information of the target module during the initialization phase of the target model to obtain the strategy query result;

[0064] Binding unit 404 is used to determine the candidate low-precision path as the target precision path of the target module in response to the strategy query result meeting the preset requirements; otherwise, it determines the candidate baseline precision path as the target precision path of the target module.

[0065] Training unit 405 is used to train the target model according to the target accuracy path of the target module.

[0066] The generation unit 401 can generate a training profile of the corresponding target model based on the model structure, model components, distributed parallel parameters (tensor parallelism TP, expert parallelism / EP / expert tensor parallelism ETP), model sequence length, batch size, hardware type and attributes, low-precision recipe, etc. extracted from model configuration information, parallel configuration information and hardware environment information.

[0067] The training profile obtained by the generation unit 401 is used to determine the scope of modules for benchmark testing and precision path binding.

[0068] In this embodiment, if the target model is a VLM (Vision Language Model), the model components corresponding to the target model may include a language backbone component, an image encoding component, a modal projection component, and an expert computing component, and different components may include one or more modules.

[0069] In this embodiment, after the generation unit 401 obtains the training profile of the corresponding target model, the benchmark testing unit 402 determines the module to be tested based on the training profile, and performs benchmark testing on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path, and obtains the strategy file based on the benchmark test report.

[0070] When the benchmark unit 402 determines the module to be tested based on the training profile, it can adopt the following implementation method: in response to the target model not being loaded, construct a parallel layer based on the training profile; and determine the module to be tested as the module selected from the constructed parallel layer.

[0071] In other words, when the target model is not loaded, the benchmark unit 402 determines the module to be tested by dynamically creating parallel layers using training profiles, thus avoiding the problem of not being able to obtain the policy file due to the target model not being loaded.

[0072] In addition, the benchmark unit 402 can also determine the target model from the loaded models based on the training profile, that is, the model corresponding to the training profile in the loaded models is taken as the target model, and then the module selected from the target model is determined as the module to be tested.

[0073] In this embodiment, the candidate low-precision paths are FP8, FP4, INT8, etc., and the candidate reference precision paths are BF16, FP16, FP32, etc.; preferably, in this embodiment, the candidate low-precision path is FP8 and the candidate reference precision path is BF16.

[0074] Specifically, when benchmarking the module under test based on the candidate low-precision path and the candidate benchmark precision path, the benchmark testing unit 402 can adopt the following implementation method: A first test environment is set up according to fixed parameters and the candidate low-precision path; a second test environment is set up according to fixed parameters and the candidate benchmark precision path. In this embodiment, the fixed parameters can be parallel configuration parameters, input / output shapes, etc., obtained based on the training profile and the module under test. Benchmarking is performed on the module under test in the first and second test environments respectively, obtaining a first test result for the corresponding candidate low-precision path and a second test result for the corresponding candidate benchmark precision path. In this embodiment, the test result can be the running time of the module. The measured speedup ratio is obtained based on the obtained first and second test results. Based on the obtained measured speedup ratio and the module information of the module under test, a benchmark test report for the corresponding module under test is obtained. The measured speedup ratio is used as a quantitative indicator to reflect the actual acceleration effect of the candidate low-precision path relative to the candidate benchmark path.

[0075] In this embodiment, the benchmark report may include fields such as module type, TE communication or user buffer name, parallel configuration parameters, number of general matrix multiplications in the expert model, measured speedup ratio, and minimum token number threshold.

[0076] When obtaining the strategy file based on the benchmark test report, the benchmark test unit 402 may also include the following: obtaining multiple benchmark test reports obtained under different parallel configuration parameters; merging the multiple benchmark test reports obtained, and obtaining the strategy file based on the merging result.

[0077] Since different parallel configuration parameters will affect the benchmark test results of the module under test, when there are multiple parallel configuration parameters, the benchmark test unit 402 will perform multiple benchmark tests on the module under test according to the different parallel configuration parameters, thereby merging the reports when multiple benchmark test reports are obtained, and obtaining the strategy file based on the merged results.

[0078] When obtaining the strategy file based on the benchmark test report, the benchmark test unit 402 may also include the following: in response to the target model being a visual language model, determining the model component to which the module under test belongs; aggregating the benchmark test reports of modules under test belonging to the same model component, and obtaining the strategy file corresponding to different model components based on the aggregation result.

[0079] In other words, since the computational forms and numerical risks of different components in the visual language model are different, directly applying the strategy of the corresponding language backbone component (i.e., LLM) to the image coding component, modal projection component, or expert computation component may lead to negative optimization. Therefore, the benchmark unit 402 obtains different strategy files based on the component type, thereby achieving the purpose of component-level strategy isolation.

[0080] When obtaining the strategy file based on the benchmark test report, the benchmark test unit 402 may also include the following: in response to the measured speedup ratio in the benchmark test report being greater than a preset speedup ratio threshold, the strategy file is obtained based on the benchmark test report. That is, the benchmark test unit 402 only saves the benchmark test report in the strategy file when the measured speedup ratio meets certain requirements, thereby improving the effectiveness of the obtained strategy file.

[0081] In this embodiment, after the benchmark unit 402 obtains the strategy file, the query unit 403 queries the strategy file based on the module information of the target module during the initialization phase of the target model to obtain the strategy query result.

[0082] The module information of the target module obtained by query unit 403 may include module type, module name, component type, parallel configuration parameters, number of general matrix multiplications, number of currently valid tokens, input / output shape, etc.

[0083] When query unit 403 queries the strategy file based on the module information of the target module to obtain the strategy query result, the following implementation methods can be adopted: determine the target query rule based on the module type of the target module; extract the target module information corresponding to the determined target query rule from the module information; and query the strategy file based on the extracted target module information to obtain the strategy query result.

[0084] In other words, the query unit 403 will select the corresponding information from the module information of the target module according to the module type of the target module to realize the strategy query, which can improve the query accuracy and query speed of the strategy.

[0085] When querying the strategy file based on the module information of the target module and obtaining the strategy query result, the query unit 403 may also include the following: determining the model component to which the target module belongs based on the module type of the target module; determining the target strategy file of the corresponding model component from the strategy file; and querying the determined target strategy file based on the module information of the target module to obtain the strategy query result.

[0086] In other words, when the query unit 403 determines that the target module is a module in a certain model component of the visual language model, it first determines the target strategy file corresponding to the model component from the strategy file, and then performs a query in the determined target strategy file according to the module information to obtain the strategy query result, so as to achieve the purpose of completing the strategy query under the component-level strategy isolation.

[0087] It is understandable that when querying the target strategy file based on the module information, query unit 403 can combine the above-mentioned method of extracting target module information based on the determined target query rules.

[0088] In this embodiment, after the query unit 403 obtains the strategy query result, the binding unit 404 determines the candidate low-precision path as the target precision path of the target module in response to the strategy query result meeting the preset requirements; otherwise, it determines the candidate baseline precision path as the target precision path of the target module.

[0089] Binding unit 404 can determine that the preset requirements are met if the policy query result is found in the policy file.

[0090] Binding unit 404 can also determine whether the strategy query result meets the preset requirements in the following way: in response to the number of valid tokens of the target module being greater than the minimum token number threshold in the strategy query result, and the measured speedup ratio in the strategy query result being greater than the preset speedup ratio threshold, the strategy query result is determined to meet the preset requirements.

[0091] In this embodiment, the binding unit 404 can estimate the number of valid tokens for the target module based on parameters such as sequence length and batch size in the training profile, or it can obtain the number of valid tokens based on the actual input of the target module.

[0092] After determining that the strategy query result meets the preset requirements, the binding unit 404 uses the candidate low-precision path as the target precision path and binds it to the target module. Binding the candidate low-precision path to the target module means that during the training process of the target model, the calculation of the target module will use the low-precision path (e.g., FP8) for forward and backward propagation.

[0093] If the strategy query result does not meet the preset requirements, the accuracy path is rolled back. The binding unit 404 uses the candidate benchmark accuracy path as the target accuracy path and binds it to the target module. Binding the candidate benchmark accuracy path to the target module means that during the training process of the target model, the calculation of the target module will use the benchmark accuracy path (e.g., BF16) for forward and backward propagation.

[0094] In this embodiment, after the binding unit 404 determines the target accuracy path, the training unit 405 performs training of the target model according to the target accuracy path of the target module.

[0095] In other words, after determining the target precision path corresponding to the target module in the target model, this embodiment can complete the training based on the target precision path of each target module during the actual training process, thus achieving the purpose of low-precision adaptation at the module level during model training.

[0096] The acquisition, storage, and application of various types of user personal information involved in the technical solutions disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0097] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0098] like Figure 5 The diagram shown is a block diagram of an electronic device for an adaptive low-precision model training method according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0100] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of displays, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] Computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 501 performs the various methods and processes described above, such as adaptive low-precision model training methods. For example, in some embodiments, the adaptive low-precision model training method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508.

[0102] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the adaptive low-precision model training method described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform the adaptive low-precision model training method by any other suitable means (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable adaptive low-precision model training device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0108] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An adaptive low-precision model training method, comprising: Based on the model configuration information, parallel configuration information, and hardware environment information of the target model, a training profile of the target model is generated. The module to be tested is determined based on the training profile, and benchmark testing is performed on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path. A strategy file is obtained based on the benchmark test report. During the initialization phase of the target model, the strategy file is queried based on the module information of the target module to obtain the strategy query results; If the strategy query result meets the preset requirements, the candidate low-precision path is determined as the target precision path of the target module; otherwise, the candidate baseline precision path is determined as the target precision path of the target module. Training of the target model is performed according to the target accuracy path of the target module.

2. The method according to claim 1, wherein, The step of determining the module to be tested based on the training profile includes: In response to the target model not being loaded, a parallel layer is constructed based on the training profile; The module selected from the parallel layer is determined as the module to be tested.

3. The method according to claim 1, wherein, The benchmark test of the module under test based on candidate low-precision paths and candidate benchmark precision paths includes: A first test environment is set up based on fixed parameters and the candidate low-precision path, and a second test environment is set up based on fixed parameters and the candidate reference precision path. Benchmark tests are performed on the module under test in the first test environment and the second test environment respectively to obtain a first test result corresponding to the candidate low-precision path and a second test result corresponding to the candidate benchmark precision path. The measured speedup ratio is obtained based on the first test result and the second test result. Based on the measured speedup ratio and the module information of the module under test, a benchmark test report for the corresponding module under test is obtained.

4. The method according to claim 1, wherein, The strategy document obtained from the benchmark report includes: Obtain multiple benchmark reports under different parallel configuration parameters; The multiple benchmark reports are merged, and the strategy file is obtained based on the merged result.

5. The method according to claim 1, wherein, The strategy document obtained from the benchmark report includes: In response to the target model being a visual language model, the model component to which the module to be tested belongs is determined; The benchmark test reports of modules under test belonging to the same model component are aggregated, and the strategy files corresponding to different model components are obtained based on the aggregation results.

6. The method according to claim 1, wherein, The strategy document obtained from the benchmark report includes: In response to the measured speedup ratio in the benchmark report being greater than a preset speedup ratio threshold, the strategy file is obtained based on the benchmark report.

7. The method according to claim 1, wherein, The step of querying the policy file based on the module information of the target module to obtain the policy query results includes: Determine the target query rules based on the module type of the target module; Extract the target module information corresponding to the target query rule from the module information; The policy file is queried based on the target module information to obtain the policy query results.

8. The method according to claim 1, wherein, The step of querying the policy file based on the module information of the target module to obtain the policy query results includes: Based on the module type of the target module, determine the model component to which the target module belongs; Determine the target strategy file corresponding to the model component from the strategy file; The target policy file is queried based on the module information of the target module to obtain the policy query result.

9. The method according to claim 1, wherein, Determining that the query results of the strategy meet the preset requirements includes: If the number of valid tokens in the target module is greater than the minimum token number threshold in the strategy query result, and the measured speedup ratio in the strategy query result is greater than the preset speedup ratio threshold, then the strategy query result is determined to meet the preset requirements.

10. An adaptive low-precision model training device, comprising: The generation unit is used to generate a training profile of the target model based on the model configuration information, parallel configuration information and hardware environment information of the target model. The benchmark testing unit is used to determine the module to be tested based on the training profile, and to perform benchmark testing on the module to be tested based on the candidate low-precision path and the candidate benchmark precision path, and to obtain a strategy file based on the benchmark testing report. The query unit is used to query the strategy file based on the module information of the target module during the initialization phase of the target model to obtain the strategy query result; A binding unit is used to determine the candidate low-precision path as the target precision path of the target module in response to the strategy query result meeting the preset requirements; otherwise, the candidate baseline precision path is determined as the target precision path of the target module. The training unit is used to train the target model according to the target accuracy path of the target module.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images