Network performance evaluation method and device
By evaluating the iteration time of test networks with different number of computing nodes in language model training, calculating and verifying the set specification processing, computing and communication time, and calculating the iteration time of the target network, it solves how to evaluate the improvement of language model training efficiency of increasing computing resources. The problem provides reference for resources, efficiency and cost, helping users make informed decisions.
Patent Information
- Application Number
- CN202410695046.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-31
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-05-31
AI Technical Summary
How to evaluate the improvement of language model training efficiency by increasing computing resource investment and balancing cost and efficiency, the existing technology is difficult to effectively solve this problem.
By determining the iteration time of test networks with different number of computing nodes in the process of training language model, estimating the set specification processing, calculation and point-to-point communication duration, and performing deviation verification, the single iteration time estimate of the target network is calculated to evaluate network performance .
It provides a reference basis for evaluating the resources, efficiency and cost of language model training networks, helping users make informed investment decisions and achieve a balance between cost and efficiency.
Smart Images

Figure CN118282894B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a method and device for evaluating network performance. Background Art
[0002] In recent years, the research on language models has made great progress in both academia and industry. LLM refers to a large language model, which represents an advanced technology in natural language processing (NLP) in the field of artificial intelligence. LLM refers to a deep learning model trained on a large corpus, which can generate high-quality natural language text.
[0003] The LLM model can adopt the Transformer structure and can process large-scale text data. Compared with the traditional NLP model, the LLM model has stronger generalization ability and higher generation quality. It can be used for tasks such as text classification, sentiment analysis, summary generation, machine translation, etc., with stronger adaptability and higher performance.
[0004] Language models such as LLM (Large Language Model, LLM) need to be trained with a large corpus, which requires a lot of time and computing resources. Since the computing resource cost of LLM is very high, users need to evaluate the performance improvement of model training brought about by increasing resource investment in the network, so as to balance the cost and efficiency. Based on this, how to evaluate the performance improvement brought about by increasing computing resource investment has become an urgent problem to be solved. Summary of the invention
[0005] To overcome the problems existing in the related art, this specification provides a network performance evaluation method and device.
[0006] According to a first aspect of an embodiment of the present specification, a network performance evaluation method is provided, the method comprising: determining a first single iteration duration of a first test network in a language model training process, a second single iteration duration of a second test network in a language model training process, and a third single iteration duration of a third test network in a language model training process, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; calculating an ensemble contract processing duration, a calculation duration, and a point-to-point communication duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration; performing deviation verification on the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration; when the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification, calculating a single iteration duration estimation value of a target network according to the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration; and determining an evaluation result of the target network according to the single iteration duration estimation value.
[0007] According to a second aspect of an embodiment of this specification, a network performance evaluation device is provided, including: a duration determination module, used to determine a first single iteration duration of a first test network in a language model training process, a second single iteration duration of a second test network in a language model training process, and a third single iteration duration of a third test network in a language model training process, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; and a calculation module, used to calculate the first single iteration duration, the second single iteration duration, and the third single iteration duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration. duration, calculating the collective contract processing duration, calculation duration and point-to-point communication duration; a verification module, used to perform deviation verification on the collective contract processing duration, the calculation duration and the point-to-point communication duration; an estimation module, used to calculate the single iteration duration estimate of the target network according to the collective contract processing duration, the calculation duration and the point-to-point communication duration when the collective contract processing duration, the calculation duration and the point-to-point communication duration pass the deviation verification; a result determination module, used to determine the evaluation result of the target network according to the single iteration duration estimate.
[0008] According to a third aspect of an embodiment of this specification, a network performance evaluation device is provided, including:
[0009] processor;
[0010] a memory for storing processor-executable instructions;
[0011] The processor is configured to: determine a first single iteration duration of a first test network in a language model training process, a second single iteration duration of a second test network in a language model training process, and a third single iteration duration of a third test network in a language model training process, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; calculate the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration; perform deviation verification on the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration; when the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification, calculate a single iteration duration estimation value of the target network according to the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration; and determine an evaluation result of the target network according to the single iteration duration estimation value.
[0012] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:
[0013] In the embodiments of this specification, by using the test data of the test network to evaluate the training performance of the target network, a reference basis for resources, efficiency and cost can be provided for the network training the language model. The evaluation results can provide a valuable reference for the user's investment in the model field and help the user make wise decisions.
[0014] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.
[0016] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.
[0017] Figure 2 It is a flowchart of a network performance evaluation method shown in this specification according to an exemplary embodiment.
[0018] Figure 3 The figure is a schematic diagram of a network performance evaluation device according to an exemplary embodiment.
[0019] Figure 4 It is a hardware structure diagram of the computer device where the file processing device in the embodiment of this specification is located.
[0020] Figure 5It is a block diagram of a network performance evaluation device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION
[0021] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.
[0022] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0023] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0024] Next, the embodiments of this specification are described in detail.
[0025] The following combination Figure 1 The system architecture to which the network performance evaluation method and device in the embodiments of this specification can be applied is described. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of this specification can be applied, to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments or scenarios.
[0026] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.
[0027] like Figure 1 As shown, the system architecture may include, for example, an evaluation device and a plurality of computing nodes. The plurality of computing nodes may form one or more networks.
[0028] The computing node may include, for example, a server. The server may include a GPU (Graphics Computing Unit) and a network card, etc. The GPU in the server may be used to perform the computing tasks required to train a large model. The network card in the server may be used to communicate with other servers.
[0029] The evaluation device can be used to test and evaluate the performance of the network when training the model. In this embodiment, three smaller-scale networks can be configured using computing nodes, and the number of computing nodes in each network is different. The evaluation device can control the three networks to perform model training and obtain test data of the three networks (for example, including the duration of a single iteration). The test data of the three networks can be used to estimate the performance of a larger-scale network training model, that is, it can be used to estimate the performance improvement brought about by increasing the computing resource investment in the network. By configuring the network scale by estimating the performance, a balance between cost and efficiency can be achieved.
[0030] The network performance evaluation method provided in the embodiments of this specification is described in detail below. Figure 2 As shown, Figure 2 This is a flowchart of a network performance evaluation method shown in this specification according to an exemplary embodiment. The method can be applied to the evaluation device shown above, for example. The network performance evaluation method provided in the embodiment of this specification may include the following steps.
[0031] In step 210, a first single iteration duration of the first test network in the process of training the language model, a second single iteration duration of the second test network in the process of training the language model, and a third single iteration duration of the third test network in the process of training the language model are determined.
[0032] According to an embodiment of the present specification, the first test network, the second test network, and the third test network can be used to perform model training tasks. The first test network, the second test network, and the third test network can each include multiple computing nodes. The number of computing nodes in the first test network, the second test network, and the third test network is different.
[0033] According to the embodiments of this specification, the language model can be used to complete natural language processing tasks, such as machine translation, automatic summarization, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, Chinese OCR, etc. The language model can include a large language model (LLM), for example.
[0034] According to an embodiment of the present specification, the process of training a language model using a network may include one or more iterative calculation processes. In the case where the process of training a language model includes one iterative calculation process, the duration used for the iterative calculation is used as the duration of a single iteration. In the case where the process of training a language model includes multiple iterative calculation processes, the average duration of the multiple iterative calculations may be calculated as the duration of a single iteration.
[0035] In step 220, based on the first single iteration duration, the second single iteration duration and the third single iteration duration, the aggregate contract processing duration, the calculation duration and the point-to-point communication duration are calculated.
[0036] According to the embodiments of this specification, the ensemble contract processing time can be the time taken to collect the calculation results of each GPU for gradient update, parameter update and then send down the calculation. The calculation time can be the time taken for the GPU to calculate the batch size. The peer communication time can be the time taken for the computing nodes to communicate through the network card.
[0037] In step 230, deviation verification is performed on the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration.
[0038] According to an embodiment of the present specification, for example, the degree of deviation of the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration from the theoretical value can be determined. If the degree of deviation is within a predetermined range, it means that the data can be used for subsequent evaluation operations. If the degree of deviation exceeds the predetermined range, the data cannot be used for evaluation operations. The predetermined range can be set according to actual needs.
[0039] In step 240, when the collective contract processing time, the calculation time and the point-to-point communication time pass the deviation verification, a single iteration time estimate of the target network is calculated based on the collective contract processing time, the calculation time and the point-to-point communication time.
[0040] According to an embodiment of the present specification, for example, a single iteration duration estimation value of the target network may be calculated according to the following formula:
[0041] .
[0042] Wherein, Time(train) can be the estimated value of a single iteration duration of the target network, Time(computation) can be the computation duration, Time(collective contract processing) can be the collective contract processing duration, and Time(communication) can be the point-to-point communication duration. N is the number of computing nodes in the target network. O(N) can be the network complexity of the target network. Network complexity can include, for example , lnN, N squared, N square root, etc. Exemplarily, corresponding network networking models can be set in advance for different network types, and the network networking model includes the network complexity corresponding to the network type. Based on this, the corresponding network networking model can be determined according to the network type of the target network, and the network complexity of the target network can be obtained from the network networking model.
[0043] In step 250, the evaluation result of the target network is determined according to the estimated value of the single iteration duration.
[0044] According to an embodiment of the present specification, for example, a linear ratio can be calculated based on an estimated value of a single iteration duration as an evaluation result. The linear ratio is the ratio of the performance improvement when the scale of the network is linearly expanded. For example, the theoretical value of a single iteration duration can be divided by the estimated value of a single iteration duration to obtain a linear ratio. For example, when the network contains 1 computing node, the single iteration duration obtained by performing a model training test can be used as the theoretical value of a single iteration duration. When the network is expanded to m computing nodes (m is a positive integer), the theoretical value of a single iteration duration can be divided by the estimated value of a single iteration when the network is expanded to m computing nodes to obtain a linear ratio when the network is expanded to m computing nodes. Optionally, in addition to the linear ratio, the evaluation result may also include model data, degree of deviation, single iteration duration, etc.
[0045] According to the embodiments of this specification, by using the test data of the test network to evaluate the training performance of the target network, a reference basis for resources, efficiency and cost can be provided for the network training the language model. The evaluation results can provide a valuable reference for the user's investment in the model field and help the user make wise decisions.
[0046] Optionally, the evaluation device can evaluate networks of different sizes, and then the evaluation device (or other devices communicating with the evaluation device) can automatically configure the constituent computing nodes of the network based on the evaluation results and the cost data of the computing nodes, so as to train the language model more efficiently.
[0047] Optionally, the network can generate a log when training the language model, which records the number of iterations and the total training time. The evaluation device can read the number of iterations and the total training time in the log and calculate the duration of a single iteration according to the following formula: Time (train) = total training time / number of iterations.
[0048] Optionally, the target network may include a target number of computing nodes. Based on this, a first number, a second number, and a third number that are less than the target number may be determined. The first number of computing nodes is configured as a first test network. The second number of computing nodes is configured as a second test network, the second number being less than the target number. The third number of computing nodes is configured as a third test network.
[0049] Therefore, the performance of large-scale networks can be evaluated through the test results of small-scale tests, reducing the workload of evaluation.
[0050] Optionally, the process of determining the first single iteration duration of the first test network in the process of training the language model, the second single iteration duration of the second test network in the process of training the language model, and the third single iteration duration of the third test network in the process of training the language model may include: using the first test network to train the language model to obtain multiple first iteration durations. Calculate the average of multiple first iteration durations to obtain the first single iteration duration. Use the second test network to train the language model to obtain multiple second iteration durations. Calculate the average of multiple second iteration durations to obtain the second single iteration duration. Use the third test network to train the language model to obtain multiple third iteration durations. Calculate the average of multiple third iteration durations to obtain the third single iteration duration. It is understandable that the steps of determining the first single iteration duration, the second single iteration duration, and the third single iteration duration can be performed in any order. By calculating the average of multiple iteration durations as the single iteration duration, the accuracy of inferring the performance of the target network can be improved.
[0051] Optionally, the process of performing deviation verification on the ensemble contract processing time, calculation time, and point-to-point communication time may include: calculating the estimated value of the single iteration time of the fourth test network according to the ensemble contract processing time, calculation time, and point-to-point communication time. Performing model training using the fourth test network to obtain the actual value of the single iteration time. Then, the difference between the estimated value of the single iteration time and the actual value of the single iteration time is calculated. If the difference is within a predetermined range, it is determined that the ensemble contract processing time, calculation time, and point-to-point communication time pass the deviation verification. Among them, the actual value of the single iteration time can be obtained by performing language model training on the fourth test network.
[0052] Optionally, the process of calculating the estimated value of a single iteration duration of the target network based on the ensemble contraction processing time, the calculation time and the point-to-point communication time may include: calculating the quotient between the calculation time and the number of computing nodes in the target network to obtain a first parameter; calculating the product between the point-to-point communication time and the complexity of the target network to obtain a second parameter; and adding the first parameter, the second parameter and the ensemble contraction processing time to obtain an estimated value of a single iteration duration.
[0053] The network performance evaluation method disclosed in this specification is described below in conjunction with specific embodiments. It is understandable that the following is only an example and does not mean that the network performance evaluation method disclosed in this specification cannot be applied to other application scenarios.
[0054] For example, in this embodiment, the first test network N1 includes 1 server, the second test network N2 includes 2 servers, the third test network N3 includes 4 servers, the fourth test network N4 includes 8 computing nodes, and the target test network may include 128 computing nodes. The first test network, the second test network, the third test network, the fourth test network and the target network may be based on the spine-leaf architecture. According to the network networking model of the spine-leaf architecture, the network complexity can be expressed as To express.
[0055] Based on this, in this embodiment, the single iteration time can be expressed as:
[0056] .
[0057] The first test network, the second test network, the third test network and the fourth test network are respectively subjected to LLM training with a parameter amount of 7B to obtain test results.
[0058] For the first test network, the single iteration time (Time = 1) is 1.9091s. For the second test network, the single iteration time (Time = 2) is 1.9976s. For the third test network, the single iteration time (Time = 4) is 2.1172s. Based on this, the following equations can be obtained:
[0059] ;
[0060] ;
[0061] .
[0062] Through the above equations, we can calculate that Time (aggregate contract processing) is 1.7847s, Time (calculation) is 0.1244s, and Time (communication) is 0.1507s.
[0063] Then, based on the calculated Time (collection contract processing), Time (computation), and Time (communication), the single iteration duration of the fourth test network can be estimated to be Time (N=8) = 1.7847 + 0.1244 / 8 + 0.1507*3 = 2.25.
[0064] Since the actual single iteration time of the fourth test network is 2.2014s, it can be calculated that the difference between the estimated value and the actual test value is 0.0486s. This difference is in the predetermined range [-0.1, 0.1], indicating that the error is small, that is, it can be confirmed that the formula can be used to estimate the performance of networks of other sizes when training the LLM model.
[0065] In addition, the single iteration duration of the first test network can be used as the theoretical value of the single iteration duration. Based on this, the linear ratio of the second test network can be calculated as r2=1.9976 / 1.9091=0.955696836, the third test network as r3=2.1172 / 1.9091=0.901709805, and the fourth test network as r4=2.2014 / 1.9091=0.867220859.
[0066] The test results are shown in Table 1.
[0067]
[0068] Based on this, it can be inferred that the single iteration time for LLM model training on a network containing 128 computing nodes is The linear ratio is 1.9091 / 2.84=67%.
[0069] Figure 3 The figure is a schematic diagram of a network performance evaluation device according to an exemplary embodiment.
[0070] like Figure 3 As shown, the network performance evaluation device may include an LLM test data collector, an iteration duration calculator, and a result display tool.
[0071] When the network performs LLM model training, a log is generated, which records the number of iterations and the total training time. The LLM test data collector can read the number of iterations and the total training time in the log, and calculate the single iteration time according to the formula: Time (train) = total training time / number of iterations. The LLM test data collector can collect the logs of three networks respectively, and obtain three single iteration times Time (N1), Time (N2) and Time (N3) under different network scales, and output them to the scale iteration time calculator.
[0072] The iteration duration calculator can receive Time (N1), Time (N2) and Time (N3), combined with the size N to be calculated, the parameter network model (built-in common complexity models such as , , square, square root, etc.), calculate Time (N) and output it to the result display tool.
[0073] The LLM test data collector reads the data Time (N4) of the N4 node as the data for verifying the deviation and outputs the scale iteration time calculator.
[0074] The LLM test data collector reads the data Time (N=1) of a single node when N=1 is used as the linear ratio basic data and outputs the result display tool.
[0075] The iteration time calculator can use N4 and the parameter network model to calculate Time (N4 theoretical value) and calculate the deviation L = Time (N4) - Time (N4 theoretical value).
[0076] The result display tool can calculate the linear ratio = Time (N=1) / Time (N), and display the parameter network model, verification data deviation L, single node iteration time Time (N=1), target network single iteration time Time (N), linear ratio, etc.
[0077] Corresponding to the embodiments of the aforementioned methods, this specification also provides embodiments of a network performance evaluation device and a terminal used therein.
[0078] The embodiments of the network performance evaluation device of this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 4 The figure is a hardware structure diagram of the computer device where the network performance evaluation device of the embodiment of this specification is located, except Figure 4 In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the server or electronic device where the device 431 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.
[0079] like Figure 5 As shown, Figure 5 This is a block diagram of a network performance evaluation device according to an exemplary embodiment of the present specification, the device comprising:
[0080] The duration determination module 510 is used to determine the duration of a first single iteration of the first test network in the process of training the language model, the duration of a second single iteration of the second test network in the process of training the language model, and the duration of a third single iteration of the third test network in the process of training the language model, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different;
[0081] A calculation module 520, configured to calculate the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration;
[0082] Verification module 530, used to verify the deviation of the aggregate contract processing time, calculation time and point-to-point communication time;
[0083] An estimation module 540, configured to calculate an estimated value of a single iteration duration of a target network according to the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration, when the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification;
[0084] The result determination module 550 is used to determine the evaluation result of the target network according to the estimated value of the single iteration duration.
[0085] Optionally, the target network may include a target number of computing nodes, and the above apparatus may further include:
[0086] A first configuration module, configured to configure a first number of computing nodes into a first test network, the first number being less than a target number;
[0087] A second configuration module, configured to configure a second number of computing nodes into a second test network, the second number being smaller than the target number;
[0088] The third configuration module is used to configure a third number of computing nodes into a third test network, where the third number is smaller than the target number.
[0089] Optionally, the duration determination module may include:
[0090] A first training submodule is used to train a language model using a first test network to obtain a plurality of first iteration durations;
[0091] A first calculation submodule, used to calculate an average of multiple first iteration durations to obtain a first single iteration duration;
[0092] A second training submodule is used to train the language model using the second test network to obtain multiple second iteration durations;
[0093] A second calculation submodule is used to calculate an average of multiple second iteration durations to obtain a second single iteration duration;
[0094] A third training submodule is used to train the language model using a third test network to obtain a plurality of third iteration durations;
[0095] The third calculation submodule is used to calculate the average value of multiple third iteration durations to obtain a third single iteration duration.
[0096] Optionally, the verification module may include:
[0097] A fourth calculation submodule, used to calculate an estimated value of a single iteration duration of a fourth test network according to the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration;
[0098] A fourth training submodule is used to perform model training using a fourth test network to obtain an actual value of a single iteration duration;
[0099] The difference calculation submodule is used to calculate the difference between the estimated value of the single iteration duration and the actual value of the single iteration duration;
[0100] The determination submodule is used to determine whether the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification if the difference is within a predetermined range.
[0101] Optionally, the estimation module may include:
[0102] A first parameter calculation submodule, used to calculate the quotient between the calculation time and the number of computing nodes in the target network to obtain a first parameter;
[0103] A second parameter calculation submodule is used to calculate the product between the point-to-point communication duration and the target network complexity to obtain a second parameter;
[0104] The estimation submodule is used to add the first parameter, the second parameter and the collection standard processing time to obtain an estimated value of the single iteration time.
[0105] According to the embodiments of this specification, by using the test data of the test network to evaluate the training performance of the target network, a reference basis for resources, efficiency and cost can be provided for the network training the language model. The evaluation results can provide a valuable reference for the user's investment in the model field and help the user make wise decisions.
[0106] Correspondingly, the present specification also provides a network performance evaluation device, which includes a processor; a memory for storing processor executable instructions; wherein the processor is configured to: determine a first single iteration duration of a first test network in a language model training process, a second single iteration duration of a second test network in a language model training process, and a third single iteration duration of a third test network in a language model training process, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; calculate the collective contract processing duration, the calculation duration, and the point-to-point communication duration based on the first single iteration duration, the second single iteration duration, and the third single iteration duration; perform deviation verification on the collective contract processing duration, the calculation duration, and the point-to-point communication duration; when the collective contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification, calculate the single iteration duration estimate of the target network based on the collective contract processing duration, the calculation duration, and the point-to-point communication duration; determine the evaluation result of the target network based on the single iteration duration estimate.
[0107] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.
[0108] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.
[0109] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0110] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.
[0111] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.
[0112] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.
Claims
1. A network performance evaluation method, characterized in that: The method comprises: Determining a first single iteration duration of a first test network in a process of training a language model, a second single iteration duration of a second test network in a process of training a language model, and a third single iteration duration of a third test network in a process of training a language model, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; Calculate the aggregate contract processing duration, calculation duration, and point-to-point communication duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration; Performing deviation verification on the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration; When the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification, calculating an estimated value of a single iteration duration of the target network according to the ensemble contract processing duration, the calculation duration, the point-to-point communication duration, a target number of computing nodes in the target network, and a target network complexity of the target network; An evaluation result of the target network is determined according to the single iteration duration estimation value.
2. The method according to claim 1, characterized in that The method further comprises: configuring a first number of computing nodes as a first test network, the first number being less than the target number; configuring a second number of computing nodes as a second test network, the second number being less than the target number; A third number of computing nodes is configured as a third test network, the third number being smaller than the target number.
3. The method according to claim 1, characterized in that The determining of a first single iteration duration of the first test network in the process of training the language model, a second single iteration duration of the second test network in the process of training the language model, and a third single iteration duration of the third test network in the process of training the language model includes: Using the first test network to train the language model, obtaining multiple first iteration durations; Calculate the average of the multiple first iteration durations to obtain the first single iteration duration; Using the second test network to train the language model, obtaining multiple second iteration durations; Calculate the average of the multiple second iteration durations to obtain the second single iteration duration; Using the third test network to train the language model, obtaining multiple third iteration durations; An average value of the plurality of third iteration durations is calculated to obtain the third single iteration duration.
4. The method according to claim 1, characterized in that: The performing deviation verification on the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration includes: Calculate a single iteration duration estimate of a fourth test network according to the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration; The fourth test network is used to train the model and obtain the actual value of the single iteration duration; Calculating the difference between the estimated value of the single iteration duration and the actual value of the single iteration duration; If the difference is within a predetermined range, it is determined that the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification.
5. The method according to claim 1, characterized in that The calculating, according to the aggregate contract processing duration, the calculation duration and the point-to-point communication duration, an estimated value of a single iteration duration of the target network comprises: Calculating a quotient between the computing time and the number of computing nodes in the target network to obtain a first parameter; Calculate the product of the point-to-point communication duration and the target network complexity to obtain a second parameter; The first parameter, the second parameter and the ensemble reduction processing duration are added together to obtain the single iteration duration estimation value.
6. A network performance evaluation device, characterized in that: The device comprises: A duration determination module, used to determine a first single iteration duration of a first test network in a language model training process, a second single iteration duration of a second test network in a language model training process, and a third single iteration duration of a third test network in a language model training process, wherein the number of computing nodes in the first test network, the second test network, and the third test network are all different; a calculation module, configured to calculate the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration according to the first single iteration duration, the second single iteration duration, and the third single iteration duration; A verification module, used to perform deviation verification on the aggregate contract processing duration, the calculation duration and the point-to-point communication duration; an estimation module, configured to calculate an estimated value of a single iteration duration of a target network according to the ensemble contract processing duration, the calculation duration, the point-to-point communication duration, a target number of computing nodes in a target network, and a target network complexity of the target network, when the ensemble contract processing duration, the calculation duration, and the point-to-point communication duration pass deviation verification; The result determination module is used to determine the evaluation result of the target network according to the single iteration duration estimation value.
7. The device according to claim 6, characterized in that The device also includes: A first configuration module, configured to configure a first number of computing nodes as a first test network, wherein the first number is less than the target number; A second configuration module, configured to configure a second number of computing nodes as a second test network, wherein the second number is smaller than the target number; A third configuration module is used to configure a third number of computing nodes into a third test network, where the third number is smaller than the target number.
8. The device according to claim 6, characterized in that The duration determination module comprises: A first training submodule is used to train a language model using a first test network to obtain a plurality of first iteration durations; A first calculation submodule, configured to calculate an average of the plurality of first iteration durations to obtain the first single iteration duration; A second training submodule is used to train the language model using the second test network to obtain multiple second iteration durations; A second calculation submodule, used to calculate an average value of the plurality of second iteration durations to obtain the second single iteration duration; A third training submodule is used to train the language model using a third test network to obtain a plurality of third iteration durations; The third calculation submodule is used to calculate the average value of the multiple third iteration durations to obtain the third single iteration duration.
9. The device according to claim 6, characterized in that The verification module comprises: a fourth calculation submodule, configured to calculate an estimated value of a single iteration duration of a fourth test network according to the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration; A fourth training submodule is used to perform model training using a fourth test network to obtain an actual value of a single iteration duration; A difference calculation submodule, used to calculate the difference between the estimated value of the single iteration duration and the actual value of the single iteration duration; The determination submodule is used to determine whether the aggregate contract processing duration, the calculation duration, and the point-to-point communication duration pass the deviation verification if the difference is within a predetermined range.
10. The device according to claim 7, characterized in that The estimation module comprises: A first parameter calculation submodule, used to calculate the quotient between the calculation duration and the number of computing nodes in the target network to obtain a first parameter; A second parameter calculation submodule, used to calculate the product between the point-to-point communication duration and the target network complexity to obtain a second parameter; The estimation submodule is used to add the first parameter, the second parameter and the ensemble reduction processing duration to obtain the single iteration duration estimation value.
