Resource allocation method and system based on tensor parallelism

Abstract

The invention relates to the field of large language models, in particular to a resource allocation method and system based on tensor parallelism, and provides a resource allocation method in order to solve the problems of how to meet calculation requirements under the condition of insufficient calculation resources and how to prolong the service life of equipment. Comprising the steps that calculation groups are divided, and the optimal calculation load and the maximum calculation load of the calculation groups are calculated; dividing the sub-modules into tensor modules or assembly line modules, and dividing priorities; calculating load capacity; calculating communication efficiency; calculating the load margin; allocating the sub-modules to each computing group to obtain a computing resource allocation plan; if the current load is greater than or equal to the maximum calculation load, preferentially calculating a part of sub-modules, and marking the sub-modules which are not preferentially calculated as to-be-distributed modules; and distributing the to-be-distributed module to the calculation group for assisted calculation.

Description

Technical Field

[0001] The technology of the present invention relates to the field of large language models, and specifically, to a resource allocation method and system based on tensor parallelism. Background Art

[0002] With the rapid development of artificial intelligence technology, the power grid field has also ushered in unprecedented changes. As an important part of artificial intelligence technology, large language models are gradually penetrating into all aspects of the power system, providing new solutions for the intelligence and efficiency of the power grid, helping power companies to predict future power demand and make plans in a smarter way, optimize resource allocation, and improve the overall service quality. However, in actual practice, due to the need to train the model and obtain massive power grid operation data such as measurement data and meteorological data for a long time, there is a long-term demand for computing resources. The existing computing resource allocation method usually gives priority to computing efficiency, which puts the computing equipment in a high-load and high-performance state, resulting in accelerated degradation of computing equipment performance, which does not meet the needs of long-term use in the power grid environment, greatly increasing the cost of using large language models. Furthermore, due to the huge amount of system data and complex data types in the power grid operation, it is easy to cause the equipment load of the computing equipment of each module to be unbalanced, resulting in inefficient thrust processing of the model and a large amount of computing resources wasted. Summary of the invention

[0003] The problem solved by the present invention is how to meet computing needs when computing resources are insufficient and extend the service life of equipment.

[0004] To solve the above problems, an embodiment of the present invention provides a resource allocation method based on tensor parallelism, the resource allocation method comprising: dividing the computing device into multiple computing groups according to the remaining working life of the computing device, obtaining the hardware information of the computing device, and calculating the optimal computing load and the maximum computing load of the computing group according to the hardware information and the computing group; splitting the data to be calculated into multiple sub-modules according to the data type and hierarchical structure information, determining the computing method applicable to the sub-module, dividing the sub-module into tensor modules or pipeline modules, and dividing the priorities; testing the sub-modules in turn to obtain test data, and calculating the computing load of each computing group corresponding to each sub-module according to the test data; and calculating the data transmission between each computing group according to the test data. communication efficiency when the optimal computing load and the current load are used; the current load of each computing group is obtained, and the load margin is calculated according to the optimal computing load and the current load; the tensor module and the pipeline module are allocated to each computing group according to the load margin, the computing load and the communication efficiency, and the computing resource allocation plan is obtained; if the current load is less than the maximum computing load, the computing resources are continuously allocated according to the computing resource allocation plan; if the current load is greater than or equal to the maximum computing load, the computing resources are reallocated according to the priority of the sub-module, and some sub-modules are given priority calculation, and the sub-modules that are not given priority calculation after the reallocation are marked as modules to be allocated; if the load margin of the computing group is greater than or equal to the computing load of the module to be allocated, the module to be allocated is allocated to the computing group for assisting calculation.

[0005] Compared with the prior art, the technical effects achieved by adopting this technical solution are as follows: by dividing computing devices into multiple computing groups according to their working life, the management strategy and computing resource allocation of computing devices can be optimized, the reliability and stability of the equipment can be improved, and the reduction in computing efficiency due to the large performance gap between computing devices can be avoided, which further facilitates the subsequent maintenance and updating of computing devices. By dividing tensor modules and pipeline modules, the processing efficiency of a large amount of data to be calculated is improved, and the overload of computing equipment due to excessive data volume is avoided. Different data can choose the appropriate computing method according to their own characteristics, which reduces the demand for computing resources. The calculation of computing load can obtain the performance of each sub-module in The specific load conditions during calculation in each computing group provide data support for the subsequent sub-module allocation. By calculating the communication efficiency, the specific conditions of data transmission between each computing device can be obtained, avoiding the subsequent reduction in computing efficiency due to low communication efficiency. By calculating the load margin, the load conditions of each computing group during calculation of the sub-module can be obtained, thereby providing data support for the subsequent optimization task allocation. By obtaining the computing resource allocation plan, the consumption of computing resources can be reduced as much as possible while meeting the computing efficiency requirements for the data to be calculated. The module to be allocated is allocated to the computing group for assisting calculation, which can optimize the utilization of computing resources and improve the overall computing efficiency.

[0006] In one embodiment of the present invention, computer devices with similar remaining working lives are divided into several computing groups; based on hardware information, the load size that the computing group can withstand when running at the expected power is calculated to obtain the optimal computing load, and the load size that the computing group can withstand when running at full power is calculated to obtain the maximum computing load.

[0007] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: by calculating the optimal computing load, the load capacity of each computing group under ideal conditions can be clarified, providing data support for subsequent allocation sub-modules, making the load distribution of computing equipment more reasonable, and extending the working life of computing equipment; by calculating the maximum computing load, the maximum load that each computing group can withstand can be known, ensuring that when working at high load, the computing group will not cause computing equipment damage and data loss due to load overload.

[0008] In one embodiment of the present invention, data of the same type and layer are merged according to the data type and hierarchical structure information, and divided into several sub-modules according to the performance of the computing device; the computing method applicable to the sub-module is determined according to the data type, and if the sub-module is more suitable for tensor parallelism, it is marked as a tensor module; if the sub-module is more practical for pipeline parallelism, it is marked as a pipeline module; based on the hierarchical structure information, the computing subordinate structure of each sub-module in the computing process is determined to obtain the priority.

[0009] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: by obtaining the priority, the task scheduling and resource allocation of the computing device in the computing process can be made more reasonable, ensuring that key data can be processed with priority. Furthermore, it provides data support for subsequent calculations according to the parallel strategy of the sub-modules when computing resources are tight, thereby optimizing the overall data calculation logic.

[0010] In one embodiment of the present invention, all sub-modules are sent to each computing group, and the sub-modules are tested in each computing group in turn to obtain the load changes and calculation results of the computing group to obtain test data; based on the test data, the maximum load value of the computing group during the test time is obtained to obtain the calculation load.

[0011] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: by obtaining test data, the actual processing status of each computing group on the sub-module can be accurately obtained, making the subsequent computing resource allocation planning more reasonable; by obtaining the computing load, it can help allocate each sub-module and further improve the utilization rate of computing resources.

[0012] In one embodiment of the present invention, test data of each submodule is obtained, and each calculation result is tested for transmission between all calculation groups. The change in transmission speed of the calculation results of each submodule between the calculation groups is obtained to obtain communication data; based on the communication data, the average value of the transmission speed during the test time is calculated to obtain the communication efficiency.

[0013] Compared with the prior art, the technical effect achieved by adopting this technical solution is as follows: by obtaining communication data, it is possible to clearly understand the transmission status of the calculation results of different sub-modules between different calculation groups, provide data support for subsequent communication efficiency calculation, and make the calculation of communication efficiency more accurate. By calculating the average value of the transmission speed, the actual transmission rate value can be obtained more accurately, thereby improving the accuracy of communication efficiency.

[0014] In one embodiment of the present invention, tensor modules and pipeline modules are allocated to various computing groups according to the load margin and the computing load, and the scheme with the smallest total computing load is calculated, which is recorded as the first allocation scheme; the computing efficiency of the first allocation scheme is calculated according to the first allocation scheme and the communication efficiency to obtain the first computing efficiency; if the first computing efficiency is greater than or equal to the expected efficiency, planning is performed according to the first allocation scheme to obtain a computing resource allocation plan; if the first computing efficiency is less than the expected efficiency, the first allocation scheme is adjusted according to the communication efficiency to obtain a second allocation scheme, and the computing efficiency of the second allocation scheme is calculated to obtain a second computing efficiency; when the second computing efficiency is greater than or equal to the expected efficiency, planning is performed according to the second allocation scheme to obtain a computing resource allocation plan.

[0015] Compared with the prior art, the technical effect achieved by adopting this technical solution is as follows: by obtaining the first allocation scheme, the allocation logic of the sub-module can be simplified, the allocation scheme with the smallest total load can be quickly calculated, and the allocation efficiency of the sub-module can be improved. By calculating the first calculation efficiency and the second calculation efficiency, it can be intuitively obtained whether the allocation scheme meets the minimum calculation efficiency requirement, ensuring that the processed data can be calculated in time, improving the rationality of the allocation of computing resources, and improving the utilization rate of computing resources.

[0016] In one embodiment of the present invention, each submodule is allocated and calculated according to the computing resource allocation plan; if the current load is less than the maximum computing load, each submodule continues to be allocated and calculated according to the computing resource classification plan; if the current load is greater than or equal to the computing load, the priority of all submodules in the computing group is obtained, and the submodules are reallocated according to the priority and the maximum computing load, and some submodules are preferentially calculated; all submodules that have not been preferentially calculated are obtained to obtain the modules to be allocated.

[0017] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: by comparing the current load and the maximum computing load, it can ensure that the computing group will not be overloaded, causing equipment damage or data loss, thereby improving the security of the computing equipment. By giving priority to the calculation of some sub-modules, it ensures that normal calculation can be maintained in the case of insufficient computing resources, further ensuring the computing efficiency and equipment safety. By marking the modules to be allocated, the sub-modules that have stopped calculating can be reallocated and calculated in time, thereby improving the overall computing efficiency.

[0018] In one embodiment of the present invention, the load margin of each computing group is compared with the computing load of each module to be allocated; when the load margin is greater than or equal to the computing load, the module to be allocated is allocated to the computing group for assisted calculation; if there is no load margin less than the computing load, no assisted calculation is performed.

[0019] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: by comparing the load margin and the computing load of the module to be allocated, it is determined whether to assist the calculation of the module to be allocated, thereby optimizing the allocation logic of the sub-modules, balancing the load size between each computing group, and improving the computing resource utilization and overall computing efficiency.

[0020] In one embodiment of the present invention, a resource allocation system based on tensor parallelism is also provided. The resource allocation method based on tensor parallelism recorded in the above embodiment is applied to the resource allocation system. The resource allocation system includes: a storage module, the storage module is used to store hardware information and sub-modules; a processing module, the processing module is used to split and acquire sub-modules and divide them into tensor modules and pipeline modules; a computing module, the computing module is used to calculate computing load, communication efficiency and load margin; an allocation module, the allocation module is used to execute computing resource allocation planning. The resource allocation system has all the technical features of the above-mentioned resource allocation method, which will not be repeated here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 One of the flow charts of the resource allocation method based on tensor parallelism of the present invention; Figure 2 This is the second flowchart of the resource allocation method based on tensor parallelism of the present invention; Figure 3 This is the third flowchart of the resource allocation method based on tensor parallelism of the present invention; Figure 4 This is the fourth flow chart of the resource allocation method based on tensor parallelism of the present invention; Figure 5 It is a system schematic diagram of a resource allocation system based on tensor parallelism of the present invention; Description of reference numerals: 100 - resource allocation system; 110 - storage module; 120 - processing module; 130 - computing module; 140 - allocation module. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] [First embodiment] See also Figure 1 In a specific embodiment, the present invention provides a resource allocation method based on tensor parallelism, the resource allocation method comprising: S100, dividing the computing devices into a plurality of computing groups according to the remaining working life of the computing devices, obtaining hardware information of the computing devices, and calculating the optimal computing load and the maximum computing load of the computing groups according to the hardware information and the computing groups; S200, splitting the data to be calculated into multiple sub-modules according to the data type and hierarchical structure information, determining the calculation method applicable to the sub-module, dividing the sub-module into a tensor module or a pipeline module, and dividing the priority; S300, testing the submodules in sequence to obtain test data, and calculating the computing load of each submodule corresponding to each computing group according to the test data; S400, calculating the communication efficiency of data transmission between various computing groups according to the test data; S500, obtaining the current load of each calculation group, and calculating the load margin according to the optimal calculation load and the current load; S600, allocating tensor modules and pipeline modules to each computing group according to load margin, computing load and communication efficiency, and obtaining a computing resource allocation plan; S700, if the current load is less than the maximum computing load, continue to allocate computing resources according to the computing resource allocation plan; if the current load is greater than or equal to the maximum computing load, reallocate computing resources according to the priority of the submodules, give priority to computing some submodules, and mark the submodules that are not given priority after reallocation as modules to be allocated; S800: If the load margin of the computing group is greater than or equal to the computing load of the module to be allocated, the module to be allocated is allocated to the computing group to assist in computing.

[0024] In step S100, normally, all computing devices should be of the same model. If there are computing devices of different models, the computing power needs to be converted according to the hardware of the computing devices. The remaining working life refers to the remaining time that the computing device can work normally. Due to the aging, replacement and expansion of computing devices, there are differences in the remaining working life of each computing device. There are also differences in the computing performance of computing devices with different remaining working lives. Furthermore, there are also differences in the optimal computing load and maximum computing load of each computing device.

[0025] It should be noted that the working life of computing equipment depends on the equipment model and actual usage.

[0026] In step S200, the data to be calculated refers to large language model training data, measurement data, meteorological data, power distribution data and other data related to the operation of the power grid. Usually, the data types of these data are complex and the data volume is extremely large. If these data are calculated directly, it will generate too much computing load on the computing device. Therefore, the same type or related data in the data to be calculated are integrated and processed and split into multiple sub-modules.

[0027] It is worth noting that the data volume of the sub-module is set according to the performance of the computing device. If the performance of the computing device is relatively good, the data volume of the sub-module can be set relatively large. If the performance of the computing device is relatively poor, the data volume of the sub-module can be set relatively small. The specific standard depends on the actual situation, and the data volume of each sub-module may vary greatly.

[0028] The tensor module refers to a data set that uses tensor parallel calculations, which is usually suitable for high-dimensional data such as images, voices, and texts in power grid operation data. For example, data such as cable laying images and power grid meeting voice records. The pipeline module refers to a data set that uses pipeline parallel calculations, which is usually suitable for data for training large prediction models, for example, training smart power supply system models.

[0029] In step S300, since the performance of computing devices in different computing groups varies, and the data types and data volumes of different sub-modules also differ, the computing loads of different computing groups when performing calculations on each sub-module also differ. Therefore, part of the data in each sub-module is selected as test data and sent to each computing group for testing, so as to avoid subsequently allocating the sub-module to a computing group with an excessively large computing load, which would result in excessive computing resources being occupied.

[0030] In step S400, the communication efficiency refers to the ratio of the actual amount of data transmitted per unit time between the calculation groups to the expected amount of data transmitted, and the calculation formula is: H c =C e / C a ×100%,H c is the communication efficiency, C e is the actual transmission rate, C a is the expected transmission rate.

[0031] For example, if the expected transmission rate is 20 gbps and the actual transmission rate is 16.5 gbps, the communication efficiency H c =C e / C a ×100%=(16.5gbps / 20gbps)×100%. =82.5%.

[0032] Because some sub-modules may need to collaborate in calculation when calculating tensor modules and pipeline modules, the communication efficiency between various computing groups may affect the overall computing efficiency. For example, if computing group A is responsible for calculating tensor module A, and computing group B is responsible for calculating tensor module B, the calculation of tensor module B requires the calculation result of tensor module A as support. If the communication efficiency between computing group A and computing group B for the calculation result is not 100%, it means that the computing efficiency will be affected due to the inability to communicate in real time.

[0033] It should be noted that, under normal circumstances, the communication efficiency during data transmission within a computing group defaults to 100%.

[0034] In step S500, since there is a long-term demand for the use of computing equipment in the power grid environment, it is necessary to ensure that all computing equipment is working under the optimal load as much as possible while ensuring that the computing needs are met, so as to extend the working life of the computing equipment. Therefore, the load margin refers to the difference between the current load and the optimal computing load. It is worth noting that in the actual calculation process, the value of the current load usually fluctuates. Therefore, the average load value of the calculation group within a certain time period is obtained as the current load, and the length of the time period depends on specific needs.

[0035] In step S600, since the load margins of various computing groups are different, the computing loads of various sub-modules in various computing groups are different, and the communication efficiency between various computing groups is also different, in order to avoid excessive load on computing equipment due to excessive power grid operation data, which affects the overall system operation, when classifying the sub-modules, full consideration should be given to factors such as load margin, computing load and communication efficiency. Under the premise of ensuring computing efficiency, the total computing load should be reduced to a minimum as much as possible to reduce the pressure of insufficient computing resources and make the calculation of the data to be calculated more efficient.

[0036] In step S700, since load fluctuations may occur during the calculation process of the sub-module during the actual operation of the calculation group, the current load cannot be maintained at the optimal calculation load or even exceeds the maximum calculation load. When there are too many data to be calculated, the calculation efficiency cannot be guaranteed under the optimal calculation load. Therefore, in order to be able to process the data to be calculated in time, the sub-module will still be allocated to the calculation group whose current load is greater than the optimal calculation load and less than the maximum calculation load. However, when the current load is greater than or equal to the maximum calculation load, it means that the calculation resources of the calculation group can no longer support the calculation of the current data volume, that is, the calculation group cannot bear the current load. In order to avoid equipment damage and data loss due to overload, the calculation of one or more sub-modules needs to be suspended, and the sub-modules with high priority are calculated first to ensure that the current load is always less than the maximum calculation load to protect equipment safety and data security.

[0037] In step S800, since different sub-modules have different data types and data volumes, the time required for calculation also varies. Some calculation groups may complete the calculation of sub-modules faster than other calculation groups, and the current load will be reduced accordingly, and the load margin will increase. In order to reduce the burden on other calculation groups and avoid wasting computing resources, if the load margin is sufficient to support the calculation of the module to be assigned, that is, when the load margin is greater than or equal to the computing load of the module to be assigned, the module to be assigned will be assigned to the calculation group to assist in the calculation.

[0038] By dividing computing devices into multiple computing groups according to their working life, the management strategy and computing resource allocation of computing devices can be optimized, the reliability and stability of the equipment can be improved, and the reduction of computing efficiency due to the large performance gap between computing devices can be avoided, which further facilitates the subsequent maintenance and update of computing devices. By dividing tensor modules and pipeline modules, the processing efficiency of a large amount of data to be calculated is improved, and the overload of computing equipment due to excessive data volume is avoided. Different data can choose the appropriate calculation method according to its own characteristics, reducing the demand for computing resources. The calculation of the computing load can obtain the time when each sub-module is calculated in each computing group. The specific load conditions provide data support for the subsequent sub-module allocation. By calculating the communication efficiency, the specific conditions of data transmission between various computing devices can be obtained, avoiding the subsequent reduction in computing efficiency due to low communication efficiency. By calculating the load margin, the load conditions of each computing group when calculating the sub-module can be obtained, thereby providing data support for the subsequent optimization task allocation. By obtaining the computing resource allocation plan, the computing resource consumption can be reduced as much as possible while meeting the computing efficiency requirements. The module to be allocated is allocated to the computing group for assisting calculation, which can optimize the utilization of computing resources and improve the overall computing efficiency.

[0039] [Second embodiment] In a specific embodiment, according to the remaining working life of the computing device, the computing device is divided into multiple computing groups, the hardware information of the computing device is obtained, and the optimal computing load and the maximum computing load of the computing group are calculated according to the hardware information and the computing group, which specifically includes: S110, dividing computing devices with similar remaining working lives into a plurality of computing groups; S120. According to the hardware information, the load size that the computing group can withstand when running at the expected power is calculated to obtain the optimal computing load, and the load size that the computing group can withstand when running at full power is calculated to obtain the maximum computing load.

[0040] In step S110, the computing devices are divided according to their remaining working life. The specific division standard depends on the actual situation. For example, if the working life of the computing devices is 10 years, the computing devices can be divided into five computing groups with remaining working lives of (0, 2] years, (2, 4] years, (4, 6] years, (6, 8] years and (8, 10] years respectively.

[0041] It is worth noting that the number of computing groups may be multiple or one. If the remaining working life of all computing devices is close, they only need to be divided into one computing group.

[0042] In step S120, the setting of the expected power should ensure that the device can operate in the best condition, while taking into account the energy efficiency and life of the computing device. It can be 280W or 300W, and can be adjusted according to specific needs. The optimal computing load refers to the sum of the loads that all computing devices in the computing group can withstand when running at the expected power.

[0043] Full power refers to the maximum rated power of the computing device, and the maximum computing load refers to the sum of the loads that all computing devices in the computing group can withstand when running at full power. It is worth noting that since the load may fluctuate during the data calculation process, in order to avoid the load exceeding the maximum limit, a certain margin should be reserved when setting the maximum computing load. The margin can be 10% or 15%, with the principle of ensuring that the maximum value of the fluctuation range does not exceed the maximum computing load.

[0044] By calculating the optimal computing load, we can clarify the load capacity of each computing group under ideal conditions, provide data support for subsequent allocation sub-modules, make the load distribution of computing equipment more reasonable, and extend the working life of computing equipment. By calculating the maximum computing load, we can know the maximum load that each computing group can withstand, ensuring that when working at high load, the computing group will not cause computing equipment damage and data loss due to load overload.

[0045] [Third embodiment] See also Figure 2 In a specific embodiment, according to the data type and hierarchical structure information, the data to be calculated is divided into multiple sub-modules, the calculation method applicable to the sub-module is determined, the sub-module is divided into a tensor module or a pipeline module, and the priority is divided, which specifically includes: S210, merging data of the same type and the same layer according to the data type and the layered structure information, and dividing the data into several sub-modules according to the performance of the computing device; S220, judging the computing method applicable to the submodule according to the data type, if the submodule is more applicable to tensor parallelism, it is marked as a tensor module, if the submodule is more applicable to pipeline parallelism, it is marked as a pipeline module; S230: According to the hierarchical structure information, determine the calculation subordination structure of each submodule in the calculation process to obtain the priority.

[0046] In step S210, since the data type of the power grid data is rich and the data structure is complex, and it contains a large number of complex and multi-dimensional tasks, these tasks usually require more computing resources and a larger computing load. A single computing device cannot complete the calculation independently. Therefore, data of the same type and layer are merged, and the complex tasks are further decomposed into multiple sub-modules. Each sub-module solves a specific problem. It is worth noting that the size of the sub-module should take into account the load capacity of a single computing device, so as to ensure that each computing device can independently calculate at least one sub-module.

[0047] In steps S220 to S230, since tensor parallelism is achieved by dividing data into multiple tensor modules and distributing them to different computing devices for parallel computing, pipeline parallelism is achieved by distributing different pipeline modules of data to different computing devices to form a pipeline for parallel computing, and tensor parallelism and pipeline parallelism are usually used in combination in a complex data processing task to achieve finer-grained parallel computing, in actual calculations, there may be a subordinate relationship between tensor modules and pipeline modules. For example, pipeline module B needs to be calculated based on the calculation results of pipeline module A, and tensor module A needs to be calculated based on the calculation results of pipeline modules B and pipeline modules C. Therefore, priority should be set for each sub-module based on the subordinate relationship between the sub-modules to ensure that when there are too many sub-modules and the computing device cannot calculate all sub-modules at the same time, the more basic sub-modules can be calculated first to ensure that the data can be processed normally.

[0048] By obtaining the priority, the task scheduling and resource allocation of the computing device during the calculation process can be made more reasonable, ensuring that key data can be processed first. Furthermore, it provides data support for subsequent calculations according to the parallel strategy of the sub-module when computing resources are tight, thereby optimizing the overall data calculation logic.

[0049] [Fourth embodiment] In a specific embodiment, the submodules are tested in sequence to obtain test data, and the computing load of each submodule corresponding to each computing group is calculated based on the test data, specifically including: S310, sending all submodules to each computing group, testing the submodules in each computing group in turn, obtaining load changes and calculation results of the computing group, and obtaining test data; S320. According to the test data, obtain the maximum load value of the computing group within the test time to obtain the computing load amount.

[0050] In step S310, when conducting the test, it should be ensured that each computing group is running at the corresponding expected power, the test time for each test is the same, and the test data includes the load changes and calculation results of the computing group, where the calculation results are used for subsequent testing of the communication efficiency between each computing group.

[0051] In step S320, the test time can be one minute or ten minutes, depending on the specific computing requirements and equipment performance. The longer the test time, the more accurate the test data obtained. Since the load fluctuates in the actual computing process, in order to avoid the load of the computing group exceeding the limit that the equipment can withstand due to load fluctuations, the maximum value in the load change curve during the test time is used as the calculation load. The calculation load refers to the load generated when the sub-module is processed in the computing group. It is worth noting that since different computing groups have different remaining working lives, their performance is different, and the calculation load when processing the same sub-module is also different.

[0052] By obtaining test data, we can accurately obtain the actual processing status of each computing group on the sub-module, making the subsequent computing resource allocation planning more reasonable. By obtaining the computing load, we can help allocate each sub-module and further improve the utilization of computing resources.

[0053] [Fifth embodiment] In a specific embodiment, according to the test data, the communication efficiency of each computing group when performing data transmission between each other is calculated, which specifically includes: S410, obtaining test data of each submodule, performing transmission test on each calculation result between all calculation groups, obtaining the transmission speed change of the calculation result of each submodule between each calculation group, and obtaining communication data; S420. Calculate the average value of the transmission speed during the test time according to the communication data to obtain the communication efficiency.

[0054] In step S410, the communication data includes changes in all transmission speeds during the test time. Since the data types and data complexities of different sub-modules are different, the transmission speeds of the calculation results of different sub-modules are also different. For example, the transmission speed of text data is much faster than the transmission speed of image data. Therefore, each calculation result is tested for transmission between all calculation groups to obtain the changes in the transmission speed of the calculation results of each sub-module between each calculation group. For example, for calculation result A, the transmission rate between calculation group A and calculation group B is 20 gbps, and the transmission rate between calculation group B and calculation group C is 16.5 gbps. For calculation result B, the transmission rate between calculation group A and calculation group B is 25 gbps, and the transmission rate between calculation group B and calculation group C is 19.3 gbps.

[0055] In step S420, since for a certain calculation result, the transmission speed fluctuates during the transmission process, in order to make the calculation of communication efficiency more accurate, the average value of the transmission speed during the calculation test time is calculated as the actual transmission rate.

[0056] By obtaining the communication data, we can clearly understand the transmission of the calculation results of different sub-modules between different calculation groups, provide data support for the subsequent communication efficiency calculation, make the calculation of communication efficiency more accurate, and by calculating the average value of the transmission speed, we can more accurately obtain the actual transmission rate value, thereby improving the accuracy of communication efficiency.

[0057] [Sixth embodiment] See also Figure 3 In a specific embodiment, according to the load margin, computing load and communication efficiency, the tensor modules and pipeline modules are allocated to each computing group to obtain a computing resource allocation plan, which specifically includes: S610, allocating tensor modules and pipeline modules to various computing groups according to the load margin and the computing load, and calculating a scheme with the smallest total load, which is recorded as a first allocation scheme; S620, calculating the computing efficiency of the first allocation scheme according to the first allocation scheme and the communication efficiency to obtain a first computing efficiency; S630: If the first computing efficiency is greater than or equal to the expected efficiency, perform planning according to the first allocation scheme to obtain a computing resource allocation plan; S640: If the first computing efficiency is less than the expected efficiency, the first allocation scheme is adjusted according to the communication efficiency to obtain a second allocation scheme, and the computing efficiency of the second allocation scheme is calculated to obtain a second computing efficiency; S650: When the second computing efficiency is greater than or equal to the expected efficiency, planning is performed according to the second allocation scheme to obtain a computing resource allocation plan.

[0058] In step S610, in order to reduce the consumption of computing resources as much as possible, when allocating sub-modules, the solution with the smallest total load should be given priority, and the total load refers to the sum of the computing loads of all tensor modules and pipeline modules.

[0059] It should be noted that during the allocation process, in order to protect the life of the computing equipment, it should be ensured as much as possible that the computing load of the sub-modules allocated to each computing group does not exceed the load margin of the computing group. If the load margin is not sufficient to meet the computing load of the sub-module, it should be ensured that the load of each computing group does not exceed the maximum computing load. If necessary, only some sub-modules are calculated to reduce the computing load pressure.

[0060] In step S620, since in the first allocation scheme, sub-modules with subordinate relationships may be assigned to different computing groups, the communication efficiency between computing groups will affect the overall computing efficiency and may cause the computing efficiency to fail to meet expectations. The first computing efficiency refers to the computing efficiency when calculating according to the first allocation scheme. For example, for a certain power grid data processing task, tensor module A is assigned to computing group A, tensor module B is assigned to computing group B, and pipeline module A is assigned to computing group B. The calculation of tensor module B is based on the calculation data of tensor module A, and the calculation of pipeline module A is based on the calculation data of tensor module B. Then, the communication efficiency between tensor module A and tensor module B is 85%, and the communication efficiency between tensor module B and pipeline module A is 100%. According to the subordinate relationship, the first computing efficiency = 85% × 100% = 85%.

[0061] In step S630 to step S650, the expected efficiency refers to the minimum computing efficiency requirement set to ensure the normal operation of the power grid. For different data to be processed, the expected efficiency is different, which may be 50% or 90%. If the first computing efficiency is greater than or equal to the expected efficiency, it means that the computing efficiency of the first allocation scheme is sufficient to meet the needs of the power grid, and it can be directly used as a computing resource allocation plan to calculate all sub-modules. If the first computing efficiency is less than the expected efficiency, it means that the computing efficiency is too low to meet the needs of the power grid. It is necessary to adjust the allocation of sub-modules while ensuring that the current load of each computing group does not exceed the maximum computing load. For example, based on According to the first allocation plan, tensor module A and pipeline module A are allocated to computing group A, tensor module B and pipeline module B are allocated to computing group B, the calculation of tensor module B is based on the calculation data of tensor module A, and the calculation of pipeline module B is based on the calculation data of pipeline module B. The first computing efficiency is calculated to be 73%, and the expected efficiency is 95%. The first allocation plan is adjusted, and tensor module B is allocated to computing group A, and pipeline module B is allocated to computing group B. The second allocation plan is obtained. The calculation method of the first computing efficiency is similar, and the second computing efficiency is calculated to be 100%, which is greater than the expected efficiency. The second allocation plan is used as the computing resource allocation plan.

[0062] It is worth noting that in the process of adjusting the first allocation plan to obtain the second allocation plan, the total load should also be minimized as much as possible.

[0063] By obtaining the first allocation plan, the allocation logic of the sub-module can be simplified, the allocation plan with the smallest total load can be quickly calculated, and the allocation efficiency of the sub-module can be improved. By calculating the first computing efficiency and the second computing efficiency, it can be intuitively determined whether the allocation plan meets the minimum computing efficiency requirement, ensuring that the processed data can be calculated in a timely manner, improving the rationality of computing resource allocation, and improving the utilization rate of computing resources.

[0064] [Seventh embodiment] See also Figure 4 In a specific embodiment, if the current load is less than the maximum computing load, the computing resources are continuously allocated according to the computing resource allocation plan; if the current load is greater than or equal to the maximum computing load, the computing resources are reallocated according to the priority of the submodules, and some submodules are preferentially calculated, and the submodules that are not preferentially calculated after the reallocation are marked as modules to be allocated, specifically including: S710, allocating and calculating each submodule according to the computing resource allocation plan; S720: If the current load is less than the maximum computing load, continue to allocate and calculate each submodule according to the computing resource classification plan; S730, if the current load is greater than or equal to the computing load, obtain the priorities of all submodules in the computing group, reallocate the submodules according to the priorities and the maximum computing load, and perform priority computing on some submodules; S740: Obtain all submodules that have not been calculated in priority, and obtain modules to be allocated.

[0065] In step S710 to step S730, each sub-module is allocated according to the computing resource allocation plan, and the current load of each computing group is monitored in real time. When multiple sub-modules are calculated in a computing group, the computing load of each sub-module may fluctuate. For some computing groups whose current load is close to the maximum computing load, there is a situation where the current load is greater than the maximum computing load due to load fluctuations, which may damage the computing equipment or cause data loss. Therefore, the current load and the maximum computing load are compared. If the current load is less than the maximum computing load, no adjustment is required. If the current load is greater than or equal to the computing load, the priority of all sub-modules in the computing group should be obtained, and the sub-modules with high priority should be calculated first, and the sub-modules with low priority should stop calculating, so as to ensure that the current load of the computing group is always less than the maximum computing load.

[0066] In step S740, the module to be allocated refers to a submodule that has not been allocated to a computing group for calculation, and is used for subsequent reallocation. It should be noted that the module to be allocated is still calculated in parallel with the originally divided tensors or in parallel with pipelines.

[0067] By comparing the current load and the maximum computing load, we can ensure that the computing group will not be overloaded, causing equipment damage or data loss, thereby improving the security of the computing equipment. By giving priority to the calculation of some sub-modules, we can ensure that normal calculations can be maintained when computing resources are insufficient, further ensuring computing efficiency and equipment safety. By marking the modules to be allocated, the sub-modules that have stopped calculating can be reallocated and calculated in time, thereby improving the overall computing efficiency.

[0068] [Eighth embodiment] In a specific embodiment, if the load margin of the computing group is greater than or equal to the computing load of the module to be allocated, the module to be allocated is allocated to the computing group for assisting computing, specifically including: S810, comparing the load margin of each computing group with the computing load of each module to be allocated; S820, when the load margin is greater than or equal to the calculated load, the module to be allocated is allocated to the calculation group for assisting calculation; S830: If there is no load margin less than the calculated load, no auxiliary calculation is performed.

[0069] In step S810 to step S830, in the actual calculation process, some calculation groups may have more load margins left due to various reasons such as low difficulty in calculating sub-modules, which is sufficient to calculate some modules to be assigned. Therefore, when the load margin is greater than or equal to the calculation load, the module to be assigned is assigned to the calculation group for assisting calculation. For example, if module A to be assigned originally belongs to calculation group A, module A to be assigned is now assigned to calculation group B for calculation. After completing the calculation of module A to be assigned, the calculation result is still sent to calculation group A for subsequent processing. If the load margin is not sufficient to support the calculation of the module to be assigned, no change is made and the module to be assigned is not calculated temporarily.

[0070] By comparing the load margin and the computing load of the module to be assigned, it is determined whether to assist the calculation for the module to be assigned, which optimizes the allocation logic of the sub-modules, balances the load size between each computing group, and improves the computing resource utilization and overall computing efficiency.

[0071] [Ninth embodiment] In a specific embodiment, the present invention also provides a resource allocation system 100 based on tensor parallelism. The resource allocation method based on tensor parallelism recorded in the above embodiment is applied to the resource allocation system 100. The resource allocation system 100 includes: a storage module 110, the storage module 110 is used to store hardware information and sub-modules; a processing module 120, the processing module 120 is used to split and acquire sub-modules and divide them into tensor modules and pipeline modules; a computing module 130, the computing module 130 is used to calculate the computing load, communication efficiency and load margin; an allocation module 140, the allocation module 140 is used to execute computing resource allocation planning. The resource allocation system 100 has all the technical features of the above-mentioned resource allocation method, which will not be repeated here one by one.

[0072] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A resource allocation method based on tensor parallelism, characterized in that: The resource allocation method comprises: According to the remaining working life of the computing device, the computing device is divided into a plurality of computing groups, hardware information of the computing device is obtained, and the optimal computing load and the maximum computing load of the computing group are calculated according to the hardware information and the computing group; According to the data type and hierarchical structure information, the data to be calculated is divided into multiple sub-modules, the calculation method applicable to the sub-module is determined, the sub-module is divided into a tensor module or a pipeline module, and the priority is divided; Testing the submodules in sequence to obtain test data, and calculating the computing load of each submodule corresponding to each computing group according to the test data; Calculating the communication efficiency of data transmission between the computing groups according to the test data; Acquire the current load of each of the calculation groups, and calculate the load margin according to the optimal calculation load and the current load; Allocating the tensor module and the pipeline module to each of the computing groups according to the load margin, the computing load and the communication efficiency to obtain a computing resource allocation plan; Adjust the computing resource allocation plan according to the current load to obtain a module to be allocated; If the load margin of the computing group is greater than or equal to the computing load of the module to be allocated, the module to be allocated is allocated to the computing group for assisting computing.

2. The resource allocation method based on tensor parallelism according to claim 1, characterized in that: The step of dividing the computing devices into a plurality of computing groups according to the remaining working life of the computing devices, acquiring hardware information of the computing devices, and calculating the optimal computing load and the maximum computing load of the computing groups according to the hardware information and the computing groups specifically includes: Dividing the computing devices with similar remaining working lives into a plurality of computing groups; According to the hardware information, the load size that the computing group can withstand when running at the expected power is calculated to obtain the optimal computing load, and the load size that the computing group can withstand when running at full power is calculated to obtain the maximum computing load.

3. The resource allocation method based on tensor parallelism according to claim 2, characterized in that: The method of dividing the data to be calculated into multiple sub-modules according to the data type and the hierarchical structure information, determining the calculation method applicable to the sub-modules, dividing the sub-modules into tensor modules or pipeline modules, and dividing the priorities specifically includes: According to the data type and the hierarchical structure information, data of the same type and the same layer are merged, and divided into a plurality of the sub-modules according to the performance of the computing device; According to the data type, determine the computing method applicable to the submodule. If the submodule is more applicable to the tensor parallel computing method, mark it as a tensor module; if the submodule is more applicable to the pipeline parallel computing method, mark it as a pipeline module; According to the hierarchical structure information, the calculation subordinate structure of each of the submodules in the calculation process is determined to obtain the priority.

4. The resource allocation method based on tensor parallelism according to claim 3, characterized in that: The testing of the submodules in sequence to obtain test data, and calculating the computing load of each submodule corresponding to each computing group according to the test data, specifically includes: Sending all the submodules to each of the computing groups, testing the submodules in each of the computing groups in turn, acquiring the load changes and calculation results of the computing groups, and obtaining the test data; According to the test data, the maximum load value of the computing group within the test time is obtained to obtain the computing load amount.

5. The resource allocation method based on tensor parallelism according to claim 4, characterized in that: The calculating, according to the test data, the communication efficiency when each of the computing groups performs data transmission between each other specifically includes: Acquire the test data of each of the submodules, perform transmission test on each of the calculation results between all the calculation groups, acquire the change of transmission speed of the calculation results of each of the submodules between the calculation groups, and obtain communication data; The communication efficiency is obtained by calculating an average value of the transmission speed within the test time according to the communication data.

6. The resource allocation method based on tensor parallelism according to claim 5, characterized in that: The allocating the tensor module and the pipeline module to each of the computing groups according to the load margin, the computing load and the communication efficiency to obtain a computing resource allocation plan specifically includes: According to the load margin and the computational load, the tensor modules and the pipeline modules are allocated to the computational groups, and a scheme with the smallest total computational load is recorded as a first allocation scheme; Calculating the computing efficiency of the first allocation scheme according to the first allocation scheme and the communication efficiency to obtain a first computing efficiency; If the first computing efficiency is greater than or equal to the expected efficiency, planning is performed according to the first allocation scheme to obtain the computing resource allocation plan; If the first computing efficiency is less than the expected efficiency, adjusting the first allocation scheme according to the communication efficiency to obtain a second allocation scheme, and calculating the computing efficiency of the second allocation scheme to obtain a second computing efficiency; When the second computing efficiency is greater than or equal to the expected efficiency, planning is performed according to the second allocation scheme to obtain the computing resource allocation plan.

7. The resource allocation method based on tensor parallelism according to claim 6, characterized in that: The adjusting the computing resource allocation plan according to the current load to obtain the module to be allocated specifically includes: Allocate and calculate each of the submodules according to the computing resource allocation plan; If the current load is less than the maximum computing load, continue to allocate and calculate each of the submodules according to the computing resource classification plan; If the current load is greater than or equal to the computing load, the priorities of all the submodules in the computing group are obtained, and the submodules are reallocated according to the priorities and the maximum computing load, and some of the submodules are preferentially computed; All the submodules that have not been calculated preferentially are acquired to obtain the modules to be allocated.

8. The resource allocation method based on tensor parallelism according to claim 7, characterized in that: If the load margin of the computing group is greater than or equal to the computing load of the module to be allocated, the module to be allocated is allocated to the computing group for assisting computing, specifically including: Comparing the load margin of each of the computing groups with the computing load of each of the modules to be allocated; When the load margin is greater than or equal to the computing load, allocating the module to be allocated to the computing group for assisting computing; If the load margin that is less than the calculated load does not exist, no auxiliary calculation is performed.

9. A resource allocation system based on tensor parallelism, characterized in that: The resource allocation method according to any one of claims 1 to 8 is applied to the resource allocation system, and the resource allocation system comprises: A storage module, the storage module is used to store the hardware information and the sub-modules; A processing module, the processing module is used to split and obtain the sub-modules and divide them into the tensor module and the pipeline module; A calculation module, the calculation module is used to calculate the calculation load, the communication efficiency and the load margin; An allocation module, wherein the allocation module is used to execute the computing resource allocation plan.

Images