Computing resource allocation method and system based on data dimension conversion

Abstract

The present invention relates to the field of computer technology, and in particular to a method and system for allocating computing resources based on data dimension conversion. The present invention solves the problem of how to reasonably allocate computing tasks to different hardware devices through data dimension conversion, thereby improving overall computing performance and efficiency. To solve the above problem, the present invention provides a computing resource allocation method based on data dimension conversion, comprising: converting current computing data of different dimensions into one-dimensional tensors according to a minimum alignment granularity; dynamically dividing the memory space in the computing device according to memory usage and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor; recording the computing device that calculates fixed computing data as a locked device, and calculating the available memory capacity of the locked device; and determining pre-allocation information of newly added computing data according to the computing status of the fixed computing data and the available memory capacity.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for allocating computing resources based on data dimension conversion. Background Art

[0002] With the rapid development of artificial intelligence (AI), the parameter size of deep learning models is growing, ranging from millions to trillions. Single-device training is no longer sufficient, making distributed training an inevitable choice. Tensor parallelism and pipeline parallelism are core strategies for distributed training. Tensor parallelism involves parallelizing computation within a single operation, splitting and distributing large tensors across multiple computing units to accelerate computations, such as matrix-matrix multiplication. Therefore, from another perspective, tensor parallelism can be viewed as intra-layer parallelism.

[0003] Significant progress has been made in distributed training using tensor parallelism, but several challenges remain. First, when partitioning large amounts of data into tensors, the fixed alignment granularity can easily lead to mismatches between tensor data blocks and the memory pool, causing memory fragmentation and resource waste. Second, task allocation strategies overly rely on the static logical structure of tensor data while ignoring hardware load and cache efficiency, potentially leading to uneven loads on execution units. Furthermore, fixed data layouts may not dynamically adapt to changes in hardware state, leading to access conflicts and bandwidth waste.

[0004] Therefore, in distributed training, how to reasonably allocate computing tasks to different hardware devices through tensor parallelism and improve the utilization of hardware devices has become a key issue that needs to be solved urgently. Summary of the Invention

[0005] The problem solved by the present invention is how to reasonably allocate computing tasks to different hardware devices through data dimension conversion, thereby improving overall computing performance and efficiency.

[0006] To solve the above problems, an embodiment of the present invention provides a computing resource allocation method based on data dimension conversion, which includes: obtaining the minimum alignment granularity of all computing devices in the working library, and converting current computing data of different dimensions into one-dimensional tensors according to the minimum alignment granularity; dynamically dividing the memory space in the computing device according to the memory occupancy of each tensor data block in the one-dimensional tensor and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor; planning fixed computing data according to the predetermined storage layout, recording the computing device that calculates the fixed computing data as a locked device, and calculating the available memory capacity of the locked device; when the working library receives new computing data, determining the pre-allocation information of the new computing data according to the computing situation and available memory capacity of the fixed computing data; determining the theoretical load of each computing device according to the pre-allocation information, and adjusting the pre-allocation information according to the theoretical load and the average access hit rate of the tensor data block.

[0007] Compared with the existing technology, the technical effects achieved by adopting this technical solution are as follows: the work library can effectively organize and manage various computing resources, making resource scheduling and allocation more efficient and intelligent. By dividing the current computing data with the minimum alignment granularity, it can ensure that the tensor data blocks are correctly aligned with the memory, thereby reducing access latency. The memory usage reflects the actual space occupied by each tensor data block in the memory, which helps to accurately allocate memory resources, optimize device usage, and avoid resource waste. Through dynamic memory partitioning, it can more flexibly respond to different computing needs. The advance determination of fixed data and locked devices can effectively avoid performance jitter caused by frequent data migration and ensure the stability and continuity of computing. The determination of available memory capacity helps to reasonably allocate computing resources and prevent memory overload. The reasonable allocation of new computing data can avoid memory resource waste and ensure the efficient execution of computing tasks. Understanding the computing situation helps to timely adjust memory resource allocation, avoid local bottlenecks, and achieve a smooth transition in dynamically changing computing needs. The calculation of theoretical load helps to calculate the maximum carrying capacity of the computing device, so that tasks can be reasonably allocated to avoid overloading of individual devices. The average access hit rate reflects the latency of memory access and helps to adjust pre-allocation information.

[0008] In one embodiment of the present invention, the minimum alignment granularity of all computing devices in the working library is obtained, and the current computing data of different dimensions are converted into one-dimensional tensors according to the minimum alignment granularity, specifically including: obtaining the memory alignment granularity of each computing device, and screening to obtain the minimum alignment granularity corresponding to the working library; dividing the current computing data on different dimensions into tensors in parallel according to the minimum alignment granularity to obtain multiple tensor data blocks; splicing the multiple tensor data blocks to convert the current computing data into a one-dimensional tensor.

[0009] Compared with the existing technology, the technical effect achieved by adopting this technical solution is: by dividing data of different dimensions into tensor parallel at the minimum alignment granularity, it can ensure that the computing device can process multiple tensor data blocks in parallel, thereby reducing the computing time; by converting multi-dimensional data into one-dimensional tensors, the storage layout of the current computing data can be simplified, thereby reducing the complexity of memory management and improving the computing efficiency of tensor parallelism.

[0010] In one embodiment of the present invention, the memory space in the computing device is dynamically divided according to the memory occupancy and the minimum alignment granularity of each tensor data block in the one-dimensional tensor to obtain a predetermined storage layout of the one-dimensional tensor, specifically including: screening the tensor data blocks according to the memory occupancy to obtain fully loaded data blocks and missing data blocks; allocating the fully loaded data blocks to each computing device according to the memory occupancy; calculating the missing memory of each missing data block, merging the missing data blocks according to the missing memory to obtain merged data blocks; allocating the merged data blocks to each computing device according to the memory occupancy, and obtaining a predetermined storage layout according to the distribution of the merged data blocks and the fully loaded data blocks.

[0011] Compared with the existing technology, the technical effect achieved by adopting this technical solution is: by identifying fully loaded data blocks and missing data blocks, the tensor data blocks that need to be merged are clearly identified, which helps to optimize the memory allocation strategy and improve the memory utilization efficiency of the computing device. Missing memory reflects the degree of memory waste and can effectively guide the merging strategy. By merging missing data blocks into merged data blocks, memory fragmentation can be effectively reduced, excessive memory waste can be avoided, and the overall utilization of memory resources can be improved.

[0012] In one embodiment of the present invention, fixed computing data is planned according to a predetermined storage layout, a computing device that calculates the fixed computing data is recorded as a locking device, and the available memory capacity of the locking device is calculated, specifically including: recording the memory occupancy of the fixed computing data as the target occupancy, and calculating the memory margin of each computing device according to the predetermined storage layout; obtaining a processing plan for the fixed computing data, and when the fixed computing data is planned to be processed by a single locking device, calculating the available memory capacity according to the target occupancy and the memory margin; when the fixed computing data is planned to be processed by multiple locking devices, calculating the average occupancy according to the target occupancy and the target number of locking devices; calculating the theoretical occupancy corresponding to the reduction of the locking devices according to the target number and the target occupancy; and calculating the available memory capacity of the locking device according to the average occupancy, the theoretical occupancy and the memory margin.

[0013] Compared with the existing technology, the technical effects achieved by adopting this technical solution are: the target occupancy reflects the memory requirement of fixed computing data, providing a benchmark for the subsequent resource allocation of memory resources; the calculation of the memory margin reflects the available memory resources of the computing device, which helps to dynamically allocate memory resources and ensure that the computing device will not fail or perform poorly due to insufficient memory; the clarity of the processing plan helps to ensure the rationality of memory resource allocation, avoid or reduce unnecessary computing device occupancy, thereby reducing the number of times the calculation results of tensor data blocks are merged and improving computing efficiency.

[0014] In one embodiment of the present invention, the available memory capacity of the locking device is calculated based on the average occupancy, theoretical occupancy and memory margin, specifically including: when the memory margin of each locking device is greater than or equal to the average occupancy and less than each theoretical occupancy, the available memory capacity is determined based on the average occupancy and the memory margin; when there is a memory margin greater than or equal to the theoretical occupancy, whether the theoretical occupancy is reasonable is judged based on the theoretical occupancy and the number of locking devices corresponding to the theoretical occupancy; if so, the available memory capacity is calculated based on the theoretical occupancy and the memory margin; if not, a reasonable occupancy is determined based on the theoretical occupancy and the memory margin, and the available memory capacity is calculated based on the reasonable occupancy and the memory margin.

[0015] Compared with the existing technology, the technical effect achieved by adopting this technical solution is: when multiple devices collaborate to process tasks, the average occupancy is used as a unified allocation benchmark to ensure that each device bears the same load, avoiding resource waste or overload due to differences in device performance. Through the matching analysis of theoretical occupancy and memory margin, the utilization rate of memory resources can be optimized, thereby dynamically adjusting the number of devices participating in the computing task. Through scenario-based decision-making based on reasonable theoretical occupancy and unreasonable theoretical occupancy, accurate allocation of memory resources is achieved, avoiding resource waste or overload caused by a "one-size-fits-all" strategy.

[0016] In one embodiment of the present invention, when the work library receives newly added calculation data, the pre-allocation information of the newly added calculation data is determined based on the calculation status of the fixed calculation data and the available memory capacity, specifically including: recording the calculation devices other than the locked device as regular devices, and when the fixed calculation data is not calculated, allocating the newly added calculation data according to the available memory capacity corresponding to the regular device to obtain pre-allocation information; when the fixed calculation data starts to be calculated, determining the allocable margin of the locked device according to the completion degree of the fixed calculation data; allocating the newly added calculation data according to the available memory capacity and allocable margin corresponding to the regular device to obtain pre-allocation information.

[0017] Compared with the existing technology, the technical effects achieved by adopting this technical solution are: the classification of locking devices and conventional devices can effectively avoid interference with key computing tasks, reduce memory fragmentation caused by task competition, and thus improve the utilization rate of memory resources. The execution progress of fixed tasks can be tracked in real time through completion, providing a quantitative basis for the allocation strategy of new computing data, realizing time-sharing reuse of memory resources, and dynamically allocating new computing data according to the allocable margin, which can effectively avoid idle resources of locking devices under low load and improve the utilization efficiency of locking devices.

[0018] In one embodiment of the present invention, when fixed calculation data is not calculated, the newly added calculation data is allocated according to the available memory capacity corresponding to the conventional device to obtain pre-allocation information, specifically including: obtaining the newly added memory capacity corresponding to the newly added calculation data, and when the newly added memory capacity is less than or equal to the available memory capacity, allocating the newly added calculation data to the conventional device; when the newly added memory capacity is greater than the available memory capacity, filtering the newly added calculation data according to the available memory capacity to obtain excess calculation data; and dividing the excess calculation data to the corresponding conventional device according to the data processing progress of the conventional device.

[0019] Compared with the existing technology, the technical effects achieved by adopting this technical solution are: the newly added memory capacity quantifies the memory occupation of the newly added computing data, which is the basic basis for the allocation of available memory resources. The comparison between the newly added memory capacity and the available memory capacity of conventional devices is the key condition for triggering the excess data screening mechanism. Tracking the data processing progress of conventional devices can effectively avoid overload or idleness of conventional devices. Load balancing is achieved by dynamically adjusting the division of newly added computing data, thereby maximizing the utilization of the memory resources of conventional devices and improving computing efficiency.

[0020] In one embodiment of the present invention, the theoretical load of each computing device is determined based on pre-allocation information, and the pre-allocation information is adjusted based on the theoretical load and the average access hit rate of the tensor data block, specifically including: correcting the theoretical load based on the average access hit rate and communication efficiency to obtain the actual load of the memory of each computing device; obtaining the normal operating load of the computing device, and judging whether the actual load is less than or equal to the normal operating load; if so, allocating new computing data based on the pre-allocation information; if not, recording the computing device with an actual load greater than the normal operating load as an overloaded device, and recording the computing device with an actual load less than the normal operating load as an idle device; obtaining the overloaded memory and overloaded data block corresponding to the newly added computing data in the overloaded device; and migrating the overloaded data block to the idle device based on the device distance and overloaded memory between the overloaded device and the idle device.

[0021] Compared with the existing technology, the technical effect achieved by adopting this technical solution is as follows: the theoretical load is corrected by averaging the access hit rate and the communication efficiency, so as to obtain an actual load that is closer to the actual operating state of the computing device. By comparing the actual load with the normal operating load, it is determined whether the actual load is within the normal operating load of the computing device, thereby effectively avoiding the decline in computing efficiency or equipment failure caused by improper allocation of data resources. The identification of overloaded devices and the determination of overloaded data blocks provide clear goals and directions for the adjustment and migration of data resources, thereby effectively reducing the probability of failure of computing devices and improving the stability and reliability of computing devices. The selection of idle devices according to the distance between devices can reduce data transmission costs and improve communication efficiency.

[0022] In one embodiment of the present invention, in one embodiment of the present invention, a computing resource allocation system based on data dimension conversion is also provided, and the computing resource allocation method recorded in the above embodiment is applied to the allocation system, and the allocation system includes: a storage module, the storage module is used to store the minimum alignment granularity and current computing data of all computing devices; a processing module, the processing module is used to convert the current computing data of different dimensions into a one-dimensional tensor; a computing module, the computing module is used to calculate the available memory capacity of the locked device; an allocation module, the allocation module is used to process the pre-allocation information of the newly added computing data. The allocation system has all the technical features of the above-mentioned computing resource allocation method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 One of the flow charts of the method for allocating computing resources;

[0024] Figure 2 This is the second flowchart of the computing resource allocation method;

[0025] Figure 3 This is the third flowchart of the computing resource allocation method;

[0026] Figure 4 This is the fourth flowchart of the computing resource allocation method;

[0027] Figure 5 This is the fifth flowchart of the computing resource allocation method;

[0028] Figure 6 A system diagram for a computing resource allocation system;

[0029] Description of reference numerals:

[0030] 100 - distribution system; 110 - storage module; 120 - processing module; 130 - computing module; 140 - distribution module. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0032] [First embodiment]

[0033] See also Figure 1 In a specific embodiment, the present invention provides a computing resource allocation method based on data dimension conversion, the computing resource allocation method comprising:

[0034] S100, obtaining the minimum alignment granularity of all computing devices in the working library, and converting current computing data of different dimensions into one-dimensional tensors according to the minimum alignment granularity;

[0035] S200, dynamically partitioning the memory space in the computing device according to the memory usage of each tensor data block in the one-dimensional tensor and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor;

[0036] S300, planning fixed computing data according to a predetermined storage layout, recording a computing device that computes the fixed computing data as a locked device, and calculating the available memory capacity of the locked device;

[0037] S400: When the work library receives newly added calculation data, it determines pre-allocation information of the newly added calculation data according to the calculation status of the fixed calculation data and the available memory capacity;

[0038] S500 , determining the theoretical load of each computing device according to the pre-allocation information, and adjusting the pre-allocation information according to the theoretical load and the average access hit rate of the tensor data block.

[0039] In step S100 and step S200, the work library refers to a storage and scheduling system that centrally manages computing tasks, data resources and device information. The work library usually contains information such as computing devices, computing data and their corresponding resource configurations required for computing tasks. Computing devices refer to hardware devices or hardware units used to perform computing tasks, including CPU, GPU and TPU, etc. Current computing data refers to the core data set being used or processed when performing computing tasks, such as model parameters or activation values ​​in training. A one-dimensional tensor is a continuous one-dimensional data structure obtained by expanding multidimensional data according to the minimum alignment granularity. Memory usage refers to the actual space occupied by a tensor data block in memory. The predetermined storage layout refers to the specified memory layout method for a tensor data block based on memory usage and minimum alignment granularity before the computing task starts.

[0040] It should be noted that when a one-dimensional tensor is divided according to the minimum alignment granularity, since the actual memory usage of the tensor data block may be a non-integer multiple of the minimum alignment granularity, the memory space occupied by the divided tensor data block may not be completely consistent with the minimum alignment granularity. Therefore, the predetermined storage layout needs to be determined based on the memory usage of the tensor data block and the minimum alignment granularity.

[0041] For example, when the minimum alignment granularity is 64 bytes, the actual size of the tensor data block is 50 bytes. After 64-byte alignment, it occupies 64 bytes, wasting 14 bytes. When the actual size of the tensor data block is 70 bytes, it needs to be aligned to 128 bytes (64×2), wasting 58 bytes.

[0042] In step S300, fixed computing data refers to a steady-state data set that needs to occupy the computing device for a long time during the calculation process, including data such as the weights, training sets and verification sets of the deep learning model. The locked device refers to a dedicated computing device designated to process the fixed computing data. The available memory capacity is the remaining memory space that can be allocated and used in the normal operating memory of the locked device, excluding the memory occupied by the fixed computing data.

[0043] In step S400 and step S500, newly added computing data refers to the data that is newly added in the computing task and needs to participate in the calculation, usually including new data samples or information that enter after the computing task starts, and new intermediate data generated during the computing process. The computing situation refers to the working status of the device, computing load, and task progress during the computing process. The pre-allocation information refers to the memory allocation plan planned in advance for the newly added computing data based on computing requirements and available memory capacity. The theoretical load refers to the load capacity of the computing device estimated based on factors such as the complexity of the computing task and the performance of the computing device. The average access hit rate refers to the proportion of required tensor data blocks that are successfully accessed within a reasonable time.

[0044] It should be noted that the average access hit rate is usually closely related to the access frequency and access distance. The closer the physical or logical distance between the computing unit and the tensor data block to be accessed in the storage hierarchy, the shorter the access time and the higher the access hit rate. When the tensor data block to be accessed is frequently accessed by multiple computing units, the access waiting time of some computing units will be extended, resulting in a lower access hit rate.

[0045] The work library effectively organizes and manages various computing resources, making resource scheduling and allocation more efficient and intelligent. By partitioning current computational data at the minimum alignment granularity, tensor data blocks are correctly aligned with memory, reducing access latency. Memory usage reflects the actual memory space occupied by each tensor data block, helping to accurately allocate memory resources, optimize device usage, and avoid resource waste. Dynamic memory partitioning enables more flexible response to diverse computing needs. Preemptive determination of fixed data and locked devices effectively avoids performance fluctuations caused by frequent data migration and ensures computational stability and continuity. Determining available memory capacity helps rationally allocate computing resources and prevent memory overload. Proper allocation of newly added computational data avoids memory waste and ensures efficient computational tasks. Understanding the computational status facilitates timely adjustment of memory resource allocation, avoiding local bottlenecks and enabling a smooth transition amidst dynamically changing computational demands. Calculating theoretical load helps calculate the maximum carrying capacity of the compute device, enabling rational task allocation and avoiding overloading of individual devices. The average access hit rate reflects the latency of memory access and helps adjust pre-allocation information.

[0046] [Second embodiment]

[0047] See also Figure 1 In a specific embodiment, obtaining the minimum alignment granularity of all computing devices in the working library and converting current computing data of different dimensions into one-dimensional tensors according to the minimum alignment granularity specifically includes:

[0048] S110, obtaining the memory alignment granularity of each computing device, and filtering to obtain the minimum alignment granularity corresponding to the working library;

[0049] S120, dividing the current calculation data on different dimensions in tensor parallel according to the minimum alignment granularity to obtain multiple tensor data blocks;

[0050] S130 , concatenate multiple tensor data blocks to convert current calculation data into a one-dimensional tensor.

[0051] In steps S110 to S130, the memory alignment granularity refers to the smallest unit of data alignment in the memory, and the minimum alignment granularity refers to the smallest unit of memory allocation required by the hardware device when storing tensor data blocks, that is, the minimum value of the memory alignment granularity.

[0052] By dividing data of different dimensions into tensor parallels at the minimum alignment granularity, it is possible to ensure that the computing device can process multiple tensor data blocks in parallel, thereby reducing computing time. By converting multi-dimensional data into one-dimensional tensors, the storage layout of the current computing data can be simplified, thereby reducing the complexity of memory management and improving the computing efficiency of tensor parallelism.

[0053] [Third embodiment]

[0054] See also Figure 2 In a specific embodiment, the memory space in the computing device is dynamically divided according to the memory usage of each tensor data block in the one-dimensional tensor and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor, specifically including:

[0055] S210, filtering the tensor data blocks according to the memory usage to obtain fully loaded data blocks and missing data blocks;

[0056] S220, allocating the fully loaded data blocks to each computing device according to the memory usage;

[0057] S230, calculating the missing memory of each missing data block, and merging the missing data blocks according to the missing memory to obtain a merged data block;

[0058] S240: Distribute the merged data blocks to each computing device according to the memory usage, and obtain a predetermined storage layout according to the distribution of the merged data blocks and the fully loaded data blocks.

[0059] In step S210 , a fully loaded data block refers to a tensor data block whose memory occupancy is a multiple of the minimum alignment granularity, and a missing data block refers to a tensor data block whose memory occupancy is not a multiple of the minimum alignment granularity.

[0060] For example, when the minimum alignment granularity is 64 bytes, if the memory occupancy of a tensor data block is 64 bytes or 128 bytes, then this tensor data block is a full data block; if the memory occupancy of a tensor data block is 54 bytes or 74 bytes, then this tensor data block is a missing data block.

[0061] In steps S220 to S240, missing memory refers to the memory vacancy caused by missing data blocks due to alignment requirements, that is, the amount of memory in the computing device that is not allocated to any data block. The merged data block is a new data block formed by splicing multiple missing data blocks, and the total memory occupancy of the merged data block is close to or equal to an integer multiple of the alignment granularity. The distribution situation refers to the allocation and layout of fully loaded data blocks and merged data blocks in each computing device.

[0062] For example, when the minimum alignment granularity is 64 bytes, if there are four missing data blocks, and the memory occupancy of the four missing data blocks is 28 bytes, 54 bytes, 74 bytes and 100 bytes respectively, then when memory allocation is performed according to the minimum alignment granularity, the missing memory corresponding to the three missing data blocks is 36 bytes, 10 bytes, 54 bytes and 28 bytes. Therefore, according to the missing memory, the missing data blocks of 28 bytes and 100 bytes are merged, and the missing data blocks of 54 bytes and 74 bytes are merged at the same time, resulting in two merged memory blocks with a total memory occupancy of 128 bytes. At this time, the missing memory is 0 bytes.

[0063] It should be noted that although the space saved each time the missing data blocks are merged is at the byte level, when the amount of data is large and there are many computing devices, the memory space saved by merging the missing data blocks can reach gigabytes or even higher levels. The saved memory space is used to allocate tensor data, thereby effectively reducing the number of computing devices. In addition, when allocating tensor data blocks to each computing device, the total memory occupied by the tensor data blocks in each computing device needs to be less than or equal to the normal operating memory of each computing device, so as to ensure the normal calculation of the computing device.

[0064] By identifying fully loaded data blocks and missing data blocks, the tensor data blocks that need to be merged are clearly identified, which helps to optimize the memory allocation strategy and improve the memory usage efficiency of the computing device. Missing memory reflects the degree of memory waste and can effectively guide the merging strategy. By merging missing data blocks into merged data blocks, memory fragmentation can be effectively reduced, excessive memory waste can be avoided, and the overall utilization of memory resources can be improved.

[0065] [Fourth embodiment]

[0066] See also Figure 3 In a specific embodiment, the fixed computing data is planned according to a predetermined storage layout, a computing device that computes the fixed computing data is recorded as a locked device, and the available memory capacity of the locked device is calculated, specifically including:

[0067] S310, recording the memory usage of the fixed computing data as the target usage, and calculating the memory margin of each computing device according to the predetermined storage layout;

[0068] S320: Obtain a processing plan for fixed computing data. When the fixed computing data is planned to be processed by a single locked device, calculate the available memory capacity based on the target occupancy and the memory margin.

[0069] S330, when the fixed calculation data is planned to be processed by multiple locking devices, calculating the average occupancy based on the target occupancy and the target number of locking devices;

[0070] S340, calculating the theoretical occupancy corresponding to the reduction of locked devices based on the target number and the target occupancy;

[0071] S350: Calculate the available memory capacity of the locked device according to the average occupancy, the theoretical occupancy, and the memory margin.

[0072] In step S310 and step S320, the target occupancy refers to the size of the space required for the fixed computing data in the memory, the memory margin is the currently remaining available memory space in the computing device calculated based on the predetermined storage layout, and the processing plan refers to the specific execution plan formulated for the fixed computing data. The processing plan usually includes information such as the computing device used, processing steps and time nodes.

[0073] In steps S330 to S350, the target number refers to the number of locked devices used to process fixed computing data in the processing plan, the average occupancy refers to the average value of the target occupancy that each locked device needs to bear in the scenario of collaborative computing of multiple locked devices, and the theoretical occupancy is the memory occupancy of fixed computing data in each locked device after the number of locked devices is reduced.

[0074] It should be noted that when the fixed computing data plan is processed by a single locking device, the available memory capacity is the difference between the sum of the memory margins of each computing device and the target occupancy. When the fixed computing data plan is processed by multiple locking devices, the average occupancy is the ratio of the target occupancy to the target number. The number of locked devices to be reduced is determined based on the target occupancy of the fixed computing data and the memory margin of each computing device.

[0075] For example, when the number of computing devices is 5, the normal operating memory is 128GB, of which there is 1 locked device, and the memory margin of the computing devices is 25GB, 18GB, 24GB, and 31GB, the memory margin of the locked device is 108GB. If the target occupancy of the fixed computing data is 90GB, then the fixed computing data will be allocated to the memory of the locked device. At this time, the available memory capacity of the computing device is 116GB. When the number of computing devices is 9, of which there are 3 locked devices, if the sum of the memory margins of the computing devices other than the locked device is 125GB, the memory margins of the locked devices are all 80GB. When the target occupancy of the fixed computing data is 90GB, one locked device can be reduced, and the number of locked devices becomes 2, and the theoretical occupancy is 40GB.

[0076] The target occupancy reflects the memory requirement of fixed computing data and provides a benchmark for subsequent memory resource allocation. The calculation of memory margin reflects the available memory resources of the computing device, which helps to dynamically allocate memory resources and ensure that the computing device will not fail or perform poorly due to insufficient memory. A clear processing plan helps to ensure the rationality of memory resource allocation, avoid or reduce unnecessary computing device occupancy, thereby reducing the number of times the tensor data block calculation results are merged and improving computing efficiency.

[0077] [Fifth embodiment]

[0078] See also Figure 3 In a specific embodiment, calculating the available memory capacity of the locked device based on the average occupancy, the theoretical occupancy, and the memory margin specifically includes:

[0079] S351. When the memory remaining of each locked device is greater than or equal to the average occupancy and less than each theoretical occupancy, determine the available memory capacity according to the average occupancy and the memory remaining;

[0080] S352: When the remaining memory capacity is greater than or equal to the theoretical capacity, determine whether the theoretical capacity is reasonable based on the theoretical capacity and the number of locked devices corresponding to the theoretical capacity.

[0081] S353: If yes, calculate the available memory capacity based on the theoretical memory usage and the memory margin;

[0082] S354. If not, determine a reasonable occupancy based on the theoretical occupancy and the memory margin, and calculate the available memory capacity based on the reasonable occupancy and the memory margin.

[0083] In steps S351 to S354, the reasonable occupancy is the amount of memory evenly allocated to each locked device for fixed calculation data after the reasonable number of locked devices is calculated based on the theoretical occupancy and the memory margin.

[0084] It should be noted that the reduced locking devices are recorded as adjustment devices. When the memory margin of each locking device is greater than or equal to the average occupancy and less than each theoretical occupancy, it means that the theoretical occupancy of the fixed calculation data in the adjustment device has exceeded the memory margin of the locking device, that is, the fixed calculation data cannot be fully allocated to the adjustment device. At this time, reducing the locking device will lead to a further increase in the load of a single device, so the original number of locking devices needs to be maintained; when there is a memory margin greater than or equal to the theoretical occupancy, it means that the theoretical occupancy of the fixed calculation data in the adjustment device does not exceed the memory margin of the locking device, that is, the fixed calculation data can be fully allocated to the adjustment device, but the number of adjustment devices is different, and the corresponding theoretical occupancy is also different. Therefore, it is necessary to judge whether the theoretical occupancy is reasonable based on the theoretical occupancy and the number of adjustment devices, and select the corresponding calculation method of the available memory capacity.

[0085] For example, when the target number of locked devices is 5 and the average occupancy is 10GB, the number of adjusted devices can be 4, 3, 2 and 1, and the corresponding theoretical occupancy is 12.5GB, 16.7GB, 25GB and 50GB respectively. If the theoretical occupancy is 12.5GB, there are 4 adjusted devices, and the memory margin of the adjusted devices is 18GB. At this time, the theoretical occupancy is less than the memory margin. The 4 locked devices carrying the 50GB task will cause resource redundancy. Therefore, the theoretical occupancy and the number of adjusted devices corresponding to the theoretical occupancy are judged to be unreasonable. At this time, the reasonable occupancy is determined based on the theoretical occupancy and the memory margin. When the memory margin is 18GB, the closest theoretical occupancy is 16.7GB. Therefore, the reasonable occupancy is 16.7GB. The minimum number of adjusted devices is 3, that is, the locked devices can be reduced by at most 2. When the theoretical occupancy is unreasonable, the root available memory capacity is the difference between the total memory margin and the total reasonable occupancy, that is, 4GB.

[0086] When multiple devices collaborate to process tasks, the average occupancy is used as a unified allocation benchmark to ensure that each device bears the same load and avoid resource waste or overload due to differences in device performance. By matching the theoretical occupancy with the memory margin, the utilization of memory resources can be optimized, thereby dynamically adjusting the number of devices participating in the computing task. By making scenario-based decisions based on reasonable theoretical occupancy and unreasonable theoretical occupancy, accurate allocation of memory resources can be achieved, avoiding resource waste or overload caused by a "one-size-fits-all" strategy.

[0087] [Sixth embodiment]

[0088] See also Figure 4 In a specific embodiment, when the work library receives newly added computing data, the pre-allocation information of the newly added computing data is determined based on the computing status of the fixed computing data and the available memory capacity, specifically including:

[0089] S410: Record computing devices other than the locked device as regular devices. When fixed computing data is not being computed, allocate new computing data according to the available memory capacity corresponding to the regular devices to obtain pre-allocation information.

[0090] S420: After the fixed calculation data starts to be calculated, the allocable margin of the locked device is determined according to the completion degree of the fixed calculation data;

[0091] S430 , allocating the newly added computing data according to the available memory capacity and allocable margin corresponding to the conventional device to obtain pre-allocation information.

[0092] In steps S410 to S430, conventional devices refer to computing devices other than locked devices, which are used to process newly added computing data or non-fixed tasks. Completion refers to the computing progress of fixed computing data on the locked device, which is usually expressed as a percentage. Allocatable margin refers to the available memory capacity released by the locked device during the processing of fixed computing data, and the allocatable margin is determined by the completion of the fixed computing data.

[0093] It should be noted that the memory of the locked device always gives priority to meeting the needs of fixed calculation data. When the fixed calculation data has not started calculation, the locked device is in a pre-occupied state. If the new calculation data occupies the locked device before the fixed calculation data starts calculation, it may cause the fixed calculation task to be delayed or fail due to insufficient memory resources, affecting the calculation efficiency. Therefore, the new calculation data can only be allocated to regular devices; when the calculation of the fixed calculation data is started, as the calculation task progresses, part of the memory is gradually released, but the calculation unit of the locked device may still be completely occupied by the fixed task. At this time, even if the memory is idle, there is no remaining computing power to process the new task. Therefore, when allocating new calculation data to the locked device, it is necessary to consider the theoretical calculation process of the fixed calculation data.

[0094] For example, according to the theoretical calculation process of fixed computing data, when the completion degree of the fixed computing data is determined to be 60%~80%, the computing unit of the locked device will be fully occupied by the fixed task again. Therefore, the safety threshold of the allocatable margin in the locked device is 60%~80% of the memory occupancy of the fixed computing data. Therefore, when the allocatable margin is 50%, the new computing data can be allocated to the locked device, and when the allocatable margin is 70%, the new computing data cannot be allocated to the locked device.

[0095] The classification of locked devices and regular devices can effectively avoid interference with key computing tasks, reduce memory fragmentation caused by task competition, and thus improve the utilization of memory resources. It can track the execution progress of fixed tasks in real time through completion, provide a quantitative basis for the allocation strategy of new computing data, realize time-sharing reuse of memory resources, and dynamically allocate new computing data according to the allocatable margin, which can effectively avoid idle resources of locked devices under low load and improve the utilization efficiency of locked devices.

[0096] [Seventh embodiment]

[0097] See also Figure 4 In a specific embodiment, when fixed calculation data is not calculated, newly added calculation data is allocated according to the available memory capacity corresponding to the conventional device to obtain pre-allocation information, which specifically includes:

[0098] S411. Acquire the newly added memory capacity corresponding to the newly added computing data. When the newly added memory capacity is less than or equal to the available memory capacity, allocate the newly added computing data to the conventional device.

[0099] S412: When the newly added memory capacity is greater than the available memory capacity, the newly added calculation data is filtered according to the available memory capacity to obtain excess calculation data;

[0100] S413. Divide the excess calculation data into corresponding conventional devices according to the data processing progress of the conventional devices.

[0101] In steps S411 to S413, the newly added memory capacity refers to the amount of memory required for the newly added computing data, the excess computing data refers to the portion of data whose memory requirements exceed the available memory capacity of the conventional device, and the data processing progress refers to the computing progress of the current computing data on the conventional device, usually expressed as a percentage.

[0102] It should be noted that when the memory demand of the newly added computing data does not exceed the available memory capacity of the conventional device, the allocation shall be based on the principle of "minimizing the number of devices", that is, the newly added computing data shall be allocated to a single or a few conventional devices as much as possible to avoid resource fragmentation caused by dispersing it to multiple conventional devices. When the newly added memory capacity exceeds the available memory capacity of the conventional device, the computing tasks in the newly added computing data shall be prioritized and the tasks shall be retained in order from high to low priority until the total occupied memory is close to the available memory of the conventional device. The remaining unallocated computing tasks shall enter the excess computing data queue. The method for allocating excess computing data according to the data processing progress of the conventional device shall refer to step S420.

[0103] The newly added memory capacity quantifies the amount of memory occupied by the newly added computing data and is the basic basis for allocating available memory resources. The comparison between the newly added memory capacity and the available memory capacity of conventional devices is the key condition for triggering the excess data screening mechanism. Tracking the data processing progress of conventional devices can effectively avoid overloading or idleness of conventional devices. Load balancing is achieved by dynamically adjusting the division of newly added computing data, thereby maximizing the utilization of the memory resources of conventional devices and improving computing efficiency.

[0104] [Eighth embodiment]

[0105] See also Figure 5 In a specific embodiment, the theoretical load of each computing device is determined based on the pre-allocation information, and the pre-allocation information is adjusted based on the theoretical load and the average access hit rate of the tensor data block, specifically including:

[0106] S510, correcting the theoretical load according to the average access hit rate and communication efficiency to obtain the actual load of each computing device memory;

[0107] S520, obtaining the normal operating load of the computing device, and determining whether the actual load is less than or equal to the normal operating load;

[0108] S530: If yes, allocate the newly added calculation data according to the pre-allocation information;

[0109] S540. If not, record the computing device whose actual load is greater than the normal operating load as an overloaded device, and record the computing device whose actual load is less than the normal operating load as an idle device;

[0110] S550, obtaining the overload memory and overload data block corresponding to the newly added calculation data in the overload device;

[0111] S560: Migrate the overloaded data block to the idle device based on the distance between the overloaded device and the idle device and the overloaded memory.

[0112] In steps S510 to S540, communication efficiency refers to the ratio of the actual amount of data transmitted per unit time between computing devices to the theoretical amount of data transmitted under ideal conditions. Communication efficiency is usually affected by factors such as network bandwidth and device interface performance. Actual load is the real-time load of the computing device obtained by correcting the theoretical load using communication efficiency and average access hit rate. Normal operating load refers to the maximum load threshold that the computing device can withstand for a long time under normal and stable operating conditions. An overloaded device refers to a computing device whose actual load exceeds the normal operating load, and an idle device refers to a computing device whose actual load is lower than the normal operating load. The actual load is calculated as follows:

[0113] Actual load = theoretical load × (1 + average access hit rate + communication efficiency).

[0114] The average access hit rate and communication efficiency range from 0% to 1%.

[0115] It should be noted that when the average access hit rate and communication efficiency are low, the theoretical load of the computing device will be lower than the actual load. Therefore, the average access hit rate and communication efficiency need to be considered when calculating the actual load. After data is allocated according to the pre-allocation information of the newly added computing data, although the memory usage in each computing device is less than or equal to the normal operating memory, due to the low communication efficiency or the tensor data blocks of the newly added computing data need to be frequently accessed in the computing device, although the memory usage is not exceeded, the actual computing load may exceed the normal operating load. Therefore, it is necessary to confirm whether the pre-allocation information of the newly added computing data is reasonable based on the actual load and the normal operating load.

[0116] In step S550 and step S560, the overloaded data block refers to the tensor data block in the newly added computing data that causes the actual load of the computing device to exceed the normal operating load. The overloaded memory refers to the newly added memory capacity occupied by the overloaded data block. The device distance is a logical indicator used to measure the communication efficiency or resource sharing degree between computing devices in tensor parallel computing. The device distance is usually proportional to the communication efficiency and resource sharing degree.

[0117] It should be noted that idle devices that are close to the overloaded device and have sufficient load capacity are given priority to migrate overloaded data blocks. The close distance between devices means lower network latency and higher bandwidth utilization, which can significantly reduce data migration time and network congestion during long-distance migration. Sufficient load capacity can avoid secondary overload caused by insufficient computing load of the target idle device.

[0118] By correcting the theoretical load based on the average access hit rate and communication efficiency, we obtain an actual load that is closer to the actual operating status of the computing device. By comparing the actual load with the normal operating load, we can determine whether the actual load is within the normal operating load of the computing device, thereby effectively avoiding the decline in computing efficiency or equipment failure caused by improper allocation of data resources. The identification of overloaded devices and the determination of overloaded data blocks provide clear goals and directions for the adjustment and migration of data resources, thereby effectively reducing the probability of computing device failure and improving the stability and reliability of computing devices. Selecting idle devices based on device distance can reduce data transmission costs and improve communication efficiency.

[0119] Ninth embodiment

[0120] See also Figure 6In one embodiment of the present invention, a computing resource allocation system 100 based on data dimension conversion is also provided. The computing resource allocation method described in the above embodiment is applied to the allocation system 100. The allocation system 100 includes: a storage module 110, which is used to store the minimum alignment granularity and current computing data of all computing devices; a processing module 120, which is used to convert current computing data of different dimensions into a one-dimensional tensor; a computing module 130, which is used to calculate the available memory capacity of the locked device; and an allocation module 140, which is used to process the pre-allocation information of the newly added computing data. The allocation system has all the technical features of the above computing resource allocation method, which will not be described in detail here.

[0121] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A computing resource allocation method based on data dimension conversion, characterized in that: The computing resource allocation method comprises: Obtaining a minimum alignment granularity of all computing devices in the work library, and converting current computing data of different dimensions into one-dimensional tensors according to the minimum alignment granularity; Dynamically partitioning the memory space in the computing device according to the memory footprint of each tensor data block in the one-dimensional tensor and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor; Planning the fixed computing data according to the predetermined storage layout, marking the computing device that computes the fixed computing data as a locked device, and calculating the available memory capacity of the locked device; When the working library receives the newly added calculation data, the pre-allocation information of the newly added calculation data is determined according to the calculation status of the fixed calculation data and the available memory capacity; determining a theoretical load of each of the computing devices according to the pre-allocation information, and adjusting the pre-allocation information according to the theoretical load and an average access hit rate of the tensor data block; Correcting the theoretical load according to the average access hit rate and communication efficiency to obtain the actual load of each computing device memory; Obtaining the normal operating load of the computing device, and determining whether the actual load is less than or equal to the normal operating load; If so, allocating the newly added computing data according to the pre-allocation information; If not, the computing device whose actual load is greater than the normal operating load is recorded as an overloaded device, and the computing device whose actual load is less than the normal operating load is recorded as an idle device; Obtaining the overload memory and overload data block corresponding to the newly added calculation data in the overload device; The overloaded data block is migrated to the idle device according to the device distance between the overloaded device and the idle device and the overloaded memory.

2. The computing resource allocation method according to claim 1, wherein: The step of obtaining the minimum alignment granularity of all computing devices in the working library and converting current computing data of different dimensions into one-dimensional tensors according to the minimum alignment granularity specifically includes: Obtaining the memory alignment granularity of each computing device, and filtering to obtain the minimum alignment granularity corresponding to the working library; Dividing the current calculation data on different dimensions in tensor parallel according to the minimum alignment granularity to obtain a plurality of tensor data blocks; The multiple tensor data blocks are spliced ​​together to convert the current calculation data into the one-dimensional tensor.

3. The computing resource allocation method according to claim 2, wherein: Dynamically partitioning the memory space in the computing device according to the memory occupancy of each tensor data block in the one-dimensional tensor and the minimum alignment granularity to obtain a predetermined storage layout of the one-dimensional tensor specifically includes: Filtering the tensor data blocks according to the memory occupancy to obtain fully loaded data blocks and missing data blocks; Allocating the fully loaded data blocks to each of the computing devices according to the memory occupancy; Calculating missing memory of each missing data block, and merging the missing data blocks according to the missing memory to obtain a merged data block; The merged data blocks are allocated to the respective computing devices according to the memory occupancy, and the predetermined storage layout is obtained according to the distribution of the merged data blocks and the fully loaded data blocks.

4. The computing resource allocation method according to claim 3, wherein: The step of planning the fixed computing data according to the predetermined storage layout, recording the computing device for computing the fixed computing data as a locked device, and calculating the available memory capacity of the locked device specifically includes: Recording the memory usage of the fixed computing data as a target usage, and calculating the memory margin of each computing device according to the predetermined storage layout; Obtaining a processing plan for the fixed computing data, and when the fixed computing data is planned to be processed by a single locking device, calculating the available memory capacity according to the target occupancy and the memory margin; When the fixed calculation data is planned to be processed by a plurality of the locking devices, an average occupancy is calculated based on the target occupancy and the target number of the locking devices; Calculating the theoretical occupancy corresponding to the reduction of the locked devices according to the target number and the target occupancy; The available memory capacity of the locking device is calculated according to the average occupancy, the theoretical occupancy and the memory margin.

5. The computing resource allocation method according to claim 4, characterized in that: Calculating the available memory capacity of the locking device according to the average occupancy, the theoretical occupancy, and the memory margin specifically includes: When the memory margin of each locked device is greater than or equal to the average occupancy and less than each theoretical occupancy, determining the available memory capacity according to the average occupancy and the memory margin; When the memory remaining amount is greater than or equal to the theoretical occupied amount, judging whether the theoretical occupied amount is reasonable according to the theoretical occupied amount and the number of locked devices corresponding to the theoretical occupied amount; If so, calculating the available memory capacity according to the theoretical occupancy and the memory margin; If not, a reasonable occupancy is determined according to the theoretical occupancy and the memory margin, and the available memory capacity is calculated according to the reasonable occupancy and the memory margin.

6. The computing resource allocation method according to claim 4, characterized in that: When the working library receives the newly added computing data, determining the pre-allocation information of the newly added computing data according to the computing status of the fixed computing data and the available memory capacity specifically includes: Recording the computing devices other than the locked device as regular devices; When the fixed calculation data has not been calculated, allocating the newly added calculation data according to the available memory capacity corresponding to the conventional device to obtain the pre-allocation information; After the fixed calculation data starts to be calculated, determining the allocable margin of the locking device according to the completion degree of the fixed calculation data; The newly added computing data is allocated according to the available memory capacity and the allocatable margin corresponding to the conventional device to obtain the pre-allocation information.

7. The computing resource allocation method according to claim 6, characterized in that: When the fixed calculation data is not calculated, allocating the newly added calculation data according to the available memory capacity corresponding to the conventional device to obtain the pre-allocation information specifically includes: Acquire the newly added memory capacity corresponding to the newly added computing data, and when the newly added memory capacity is less than or equal to the available memory capacity, allocate the newly added computing data to the conventional device; When the newly added memory capacity is greater than the available memory capacity, filtering the newly added calculation data according to the available memory capacity to obtain excess calculation data; The excess calculation data is divided into corresponding conventional devices according to the data processing progress of the conventional devices.

8. A computing resource allocation system based on data dimension conversion, characterized in that: The computing resource allocation method according to any one of claims 1 to 7 is applied to the allocation system, the allocation system comprising: a storage module, configured to store the minimum alignment granularity and the current calculation data of all the computing devices; a processing module, configured to convert the current calculation data of different dimensions into the one-dimensional tensor; a calculation module, configured to calculate the available memory capacity of the locking device; An allocation module is used to process the pre-allocation information of the newly added calculation data.

Images