Dynamic bandwidth allocation method and device, medium and product

Abstract

The invention discloses a dynamic bandwidth allocation method and device, a medium and a product, and relates to the technical field of model training. The dynamic bandwidth allocation method comprises the following steps: acquiring gradient tensor metadata of each computational node in a cross-modal model; performing global sorting on the gradient importance score in the gradient tensor metadata of each computational node, determining a gradient scheduling priority, and determining a bandwidth upper limit threshold matched with the gradient tensor according to the gradient importance score in the gradient tensor metadata of each computational node and the total bandwidth to be allocated; and generating a decision scheduling instruction according to the gradient scheduling priority and the bandwidth upper limit threshold matched with the gradient tensor. According to the technical scheme of the embodiment of the invention, dynamic bandwidth resource allocation can be carried out in the model training process, the model training process is accelerated, and efficient utilization of network resources is realized.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and in particular to a dynamic bandwidth allocation method, device, medium, and product. Background Technology

[0002] In cross-modal training, all gradient tensors are treated as equally important. However, the contribution of gradients to model convergence varies drastically across different modalities and even between different layers of the same model. Allocating equal bandwidth to unimportant gradients as to critical gradients results in a severe waste of valuable network resources. In heterogeneous environments with limited bandwidth (such as edge computing), indiscriminately transmitting all gradients simultaneously can easily lead to network bottlenecks, as all gradient synchronization requires waiting, thus slowing down the entire training process. Furthermore, static allocation strategies in cross-modal training cannot be adjusted according to training needs, resulting in poor dynamic adaptability. Summary of the Invention

[0003] This invention provides a dynamic bandwidth allocation method, device, medium, and product to solve the problem that static bandwidth resource allocation strategies cannot allocate bandwidth resources on demand during cross-modal model training.

[0004] According to one aspect of the present invention, a dynamic bandwidth allocation method is provided, comprising:

[0005] Obtain the gradient tensor metadata of each computation node in the cross-modal model;

[0006] The gradient importance score in the gradient tensor metadata of each computing node is globally sorted to determine the gradient scheduling priority. Based on the gradient importance score in the gradient tensor metadata of each computing node and the total bandwidth to be allocated, the upper limit threshold of the bandwidth matching the gradient tensor is determined.

[0007] Decision scheduling instructions are generated based on gradient scheduling priority and bandwidth upper limit thresholds that match the gradient tensor.

[0008] According to another aspect of the present invention, a dynamic bandwidth allocation device is provided, comprising:

[0009] The data acquisition module is used to acquire gradient tensor metadata for each computation node in the cross-modal model;

[0010] The bandwidth and scheduling priority allocation module is used to globally sort the gradient importance scores in the gradient tensor metadata of each computing node, determine the gradient scheduling priority, and determine the upper limit threshold of bandwidth matching the gradient tensor based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated.

[0011] The scheduling instruction generation module is used to generate decision scheduling instructions based on gradient scheduling priority and bandwidth upper limit threshold matching the gradient tensor.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] At least one processor; and a memory communicatively connected to said at least one processor;

[0014] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the dynamic bandwidth allocation method according to any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the dynamic bandwidth allocation method according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the dynamic bandwidth allocation method described in any embodiment of the present invention.

[0017] The technical solution of this invention obtains the gradient tensor metadata of each computing node in a cross-modal model, then globally sorts the gradient importance scores in the gradient tensor metadata of each computing node to determine the gradient scheduling priority. Based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated, a bandwidth upper limit threshold matching the gradient tensor is determined. Finally, a decision scheduling instruction is generated based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor. In this solution, the network bandwidth allocation strategy can be dynamically adjusted in real time based on the gradient importance scores, ensuring priority transmission of important gradients. Simultaneously, the model convergence is accelerated based on the priority transmission order of gradients and bandwidth allocation, thereby reducing the total model training time. This solves the problem of static bandwidth resource allocation strategies during cross-modal model training, which cannot allocate bandwidth resources on demand. It enables dynamic bandwidth resource allocation during model training, speeding up the model training process and achieving efficient utilization of network resources.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a dynamic bandwidth allocation method for model training provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart of a dynamic bandwidth allocation method for model training provided in Embodiment 2 of the present invention;

[0022] Figure 3 A schematic diagram illustrating the key steps in bandwidth allocation for training dynamic networks for cross-modal models;

[0023] Figure 4 This is a schematic diagram of the structure of a dynamic bandwidth allocation device provided in Embodiment 3 of the present invention;

[0024] Figure 5 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "current," "target," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Example 1

[0028] Figure 1This is a flowchart of a dynamic bandwidth allocation method for model training provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where bandwidth resources are dynamically adapted during cross-modal model training. This method can be executed by a dynamic bandwidth allocation device for model training, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0029] Step 110: Obtain the gradient tensor metadata of each computation node in the cross-modal model.

[0030] In this context, a cross-modal model can be an intelligent system in the field of artificial intelligence used to process and fuse multiple data modalities (such as text, images, video, audio, etc.). For example, a cross-modal model has multiple computation nodes, each responsible for computing a local gradient tensor; thousands of gradient tensors can form a gradient. Gradient tensor metadata can be structured data used in deep learning to describe tensor gradient information. Gradient tensor metadata may include, but is not limited to, the gradient update target node and gradient importance scores.

[0031] Optionally, each computing node is responsible for calculating the gradient importance score for the locally computed gradient tensor. The gradient importance score can be used to describe the importance of the gradient tensor to model convergence during model training. For example, in the early stages of model training, the importance of all parameters is likely the same; therefore, the gradient importance scores corresponding to the gradient tensors can be consistent in the early stages of model training. However, in the later stages of model training, the low-level features of some modalities have stabilized, and their gradient importance decreases, while the gradient importance of higher-level fusion layers increases. At this time, the gradient importance score of the stabilized gradient tensor is lowered, while the gradient importance score of the gradient tensor under the fusion layer is raised. That is, the gradient importance score can dynamically and accurately assess the importance of each gradient tensor to model convergence during a specific training period. This embodiment of the invention does not limit the adjustment value of the gradient importance score.

[0032] In this embodiment of the invention, gradient tensor metadata to be transmitted on each computing node in a cross-modal model can be collected by a centralized or distributed controller.

[0033] Step 120: Perform global sorting of gradient importance scores in the gradient tensor metadata of each computing node to determine gradient scheduling priority, and determine the upper limit threshold of bandwidth matching the gradient tensor based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated.

[0034] Here, gradient scheduling priority can be the priority of gradient tensor transmission. Total bandwidth to be allocated can be the total bandwidth that needs to be allocated. Bandwidth upper limit threshold can be the maximum bandwidth for gradient tensor transmission.

[0035] In this embodiment of the invention, gradient importance scores can be parsed from the gradient tensor metadata of each computing node. The global gradient tensors are then sorted according to their gradient tensor importance scores from high to low, rather than just the internal sorting within the computing nodes. Based on the sorting results of the gradient tensors, the gradient scheduling priority is determined. Furthermore, according to the gradient importance scores in the gradient tensor metadata of each computing node, the bandwidth allocation weight of each gradient tensor is determined. The product of the bandwidth allocation weight of each gradient tensor and the total bandwidth to be allocated is used as the upper limit threshold for matching the corresponding gradient tensor.

[0036] Step 130: Generate decision scheduling instructions based on gradient scheduling priority and bandwidth upper limit threshold matching the gradient tensor.

[0037] Among them, the decision scheduling instruction can be an instruction that specifies the transmission priority of each gradient stream and the upper limit of the bandwidth matched by the gradient tensor in the current time slice.

[0038] In this embodiment of the invention, a decision scheduling instruction can be created based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor, and then the gradient can be transmitted to the gradient update destination node, i.e. the gradient update node, based on the decision scheduling instruction.

[0039] The technical solution of this invention obtains the gradient tensor metadata of each computing node in a cross-modal model, then globally sorts the gradient importance scores in the gradient tensor metadata of each computing node to determine the gradient scheduling priority. Based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated, a bandwidth upper limit threshold matching the gradient tensor is determined. Finally, a decision scheduling instruction is generated based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor. In this solution, the network bandwidth allocation strategy can be dynamically adjusted in real time based on the gradient importance scores, ensuring priority transmission of important gradients. Simultaneously, the model convergence is accelerated based on the priority transmission order of gradients and bandwidth allocation, thereby reducing the total model training time. This solves the problem of static bandwidth resource allocation strategies during cross-modal model training, which cannot allocate bandwidth resources on demand. It enables dynamic bandwidth resource allocation during model training, speeding up the model training process and achieving efficient utilization of network resources.

[0040] Example 2

[0041] Figure 2This is a flowchart of a dynamic bandwidth allocation method for model training provided in Embodiment 2 of the present invention. This embodiment is a specific embodiment based on the above embodiment, and provides specific optional implementation methods for obtaining gradient tensor metadata of each computing node in a cross-modal model. Figure 2 As shown, the method includes:

[0042] Step 210: Calculate the gradient importance score of the gradient tensor through each computing node according to the gradient importance evaluation strategy.

[0043] The gradient importance evaluation strategy can be a pre-defined rule for calculating gradient importance. This strategy can include at least one of the following: gradient norm evaluation strategy, loss function sensitivity evaluation strategy, and cross-modal prior evaluation strategy. The gradient norm evaluation strategy can be a strategy that calculates the gradient importance score based on the L2 norm. The loss function sensitivity evaluation strategy can be a strategy that evaluates the gradient importance score based on the sensitivity of the loss function, that is, it uses the partial derivative of the loss function with respect to the gradient tensor to quantify the gradient importance. The loss function is used to measure the impact of small changes in the gradient tensor on the model loss; the higher the sensitivity of the loss function, the greater the importance of the gradient tensor. The cross-modal prior evaluation strategy can be a strategy that evaluates the gradient importance score using the gradient tensor corresponding to the source label and the weights corresponding to the source label.

[0044] In this embodiment of the invention, each computing node can compute the gradient importance score of its local gradient tensor in parallel based on at least one gradient importance evaluation strategy.

[0045] In an optional embodiment of the present invention, calculating the gradient importance score of the gradient tensor according to the gradient importance evaluation strategy through each computing node may include: calculating multiple initial gradient importance scores through the current computing node when the gradient importance evaluation strategy of the current computing node is not unique; and calculating the gradient importance score of the gradient tensor based on the multiple initial gradient importance scores and the evaluation strategy weights.

[0046] The initial gradient importance score can be an unfused gradient importance score calculated based on multiple gradient importance evaluation strategies when the gradient importance evaluation strategy of the current computing node is not unique. The evaluation strategy weight can be the weight coefficient of the gradient importance evaluation strategy.

[0047] In this embodiment of the invention, if the gradient importance evaluation strategy of the current computing node is not unique, the current computing node can calculate multiple initial gradient importance scores for each gradient tensor according to multiple gradient importance evaluation strategies. Then, the initial gradient importance scores are weighted with the evaluation strategy weights of the gradient importance evaluation strategies used in their calculation to obtain the gradient importance score for each gradient tensor. By using multiple gradient importance evaluation strategies, the importance of gradient tensors to model convergence during model training is evaluated. Furthermore, by fusing multi-dimensional initial gradient importance scores according to the evaluation strategy weights, the scientific rigor of the gradient importance scores is enhanced to meet the needs of different model training scenarios.

[0048] Step 220: Use the gradient importance score of the gradient tensor and the gradient update target node as the gradient tensor metadata.

[0049] In this context, the gradient update target node is the node whose gradient needs to be updated.

[0050] In this embodiment of the invention, the gradient importance score of the gradient tensor and the gradient update target node corresponding to the gradient tensor can be used as gradient tensor metadata to provide a data foundation for the subsequent automated and scientific dynamic allocation of bandwidth resources.

[0051] In an optional embodiment of the present invention, the cross-modal prior evaluation strategy may include: determining a gradient importance score of the gradient tensor based on the gradient norm and the mapping basis weights of the source labels of the gradient tensor.

[0052] Here, the gradient norm can be the L2 norm of the gradient tensor. The source label can be used to characterize the source of the gradient tensor. For example, the source label can be, but is not limited to, the third layer of the image encoder, or the text-image fusion layer. The mapping base weights can be weight coefficients that have a mapping relationship with the source label of the gradient tensor.

[0053] In this embodiment of the invention, the gradient norm of the gradient tensor can be calculated first, and the mapping basis weights with the source labels of the gradient tensor can be determined through prior knowledge or real-time analysis results. Then, the gradient norm of the gradient tensor is multiplied by the mapping basis weights of the corresponding source labels to obtain the gradient importance score of the gradient tensor. By assigning different basis weights to gradient tensors with different source labels, the semantic importance of gradient tensors can be distinguished, enabling the communication strategy to perceive the semantics of gradient tensors during model training and allocate communication resources to key components (such as image feature extractors and text feature extractors) at different stages of model training.

[0054] Step 230: Perform global sorting of gradient importance scores in the gradient tensor metadata of each computing node to determine gradient scheduling priority, and determine the upper limit threshold of bandwidth matching the gradient tensor based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated.

[0055] In an optional embodiment of the present invention, the gradient importance scores in the gradient tensor metadata of each computing node are globally sorted to determine the gradient scheduling priority. This may include: determining the priority division ratio; globally sorting the gradient importance scores in the gradient tensor metadata of each computing node in descending order to obtain gradient importance score sorting data; and determining the gradient scheduling priority based on the priority division ratio and the gradient importance score sorting data.

[0056] The priority division ratio can be used to specify the number of gradient tensors under different transmission priorities. The gradient importance score ranking data can be the result of ranking gradient importance scores from high to low.

[0057] In this embodiment of the invention, a pre-configured priority division ratio can be obtained to sort the gradient importance scores in the gradient tensor metadata of all computing nodes in descending order, i.e., to achieve global sorting of gradient importance scores. Then, according to the priority division ratio, the gradient importance score sorted data is divided into an array, and the gradient tensors corresponding to the gradient importance score sorted data in the top-ranked group are assigned higher priorities (gradient tensors corresponding to gradient importance score sorted data within the same group have the same priority, with the priority of the top-ranked group being higher than that of the bottom-ranked group), thus obtaining the gradient scheduling priority. Through the priority division ratio and the gradient importance score sorted data, a reasonable division of gradient tensor transmission priorities is achieved, and the priority division ratio can be dynamically adjusted as needed to meet different model training requirements.

[0058] Step 240: Generate decision scheduling instructions based on gradient scheduling priority and bandwidth upper limit threshold matching the gradient tensor.

[0059] In an optional embodiment of the invention, before generating decision scheduling instructions based on gradient scheduling priority and a bandwidth upper limit threshold matching the gradient tensor, the invention may further include: compressing the gradient tensor through each computing node, and / or overlapping communication and computation when computing the gradient tensor.

[0060] Compressing gradient tensors can reduce the amount of data transmitted by quantization (e.g., converting 32-bit floating-point numbers to 16-bit floating-point numbers) or sparsification (transmitting only gradients with large absolute values). Overlapping communication and computation allows the gradient tensors of the previous batch of data to be synchronized when the gradient tensors of the next batch of data are generated, thus hiding communication delays.

[0061] In this embodiment of the invention, after each computing node calculates the gradient tensor, it can also compress the gradient tensor and / or overlap communication and computation during the gradient tensor calculation. By using these techniques simultaneously, a synergistic effect is created, further unlocking the model's performance potential.

[0062] In an optional embodiment of the present invention, after generating a decision scheduling instruction based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor, the method may further include: when it is determined, based on the instruction execution result of the decision scheduling instruction, that the cross-modal model does not meet the model update termination condition, updating the cross-modal model and returning to perform the operation of obtaining the gradient tensor metadata of each computing node in the cross-modal model.

[0063] The instruction execution result can be the execution result of the decision scheduling instruction, reflecting the gradient tensor update status of the gradient update target node. The model update termination condition can be the condition for the model to end training. The model update termination condition can include, but is not limited to, the condition that the model prediction error is within the allowable error range.

[0064] In this embodiment of the invention, if the gradient tensor of the gradient update target node is successfully updated based on the execution result of the decision scheduling instruction, it is further determined whether the model prediction capability of the cross-modal model meets the model update termination condition. If the cross-modal model does not meet the model update termination condition, the next round of training of the cross-modal model is performed, and each computing node in the cross-modal model recalculates the gradient tensor to update the gradient tensor metadata. Then, the operation of obtaining the gradient tensor metadata of each computing node in the cross-modal model is returned until the cross-modal model meets the model update termination condition. Since the gradient importance evaluation is performed dynamically in each iteration, the adaptive training state of the multimodal model is ensured.

[0065] Figure 3 A schematic diagram illustrating the key steps in bandwidth allocation for training dynamic networks for cross-modal models, as shown below. Figure 3 As shown, after the compute nodes of the cross-modal model perform local training, they calculate the gradient importance score of each gradient tensor, package the gradient tensor and its metadata, and send them to the bandwidth allocation unit. The bandwidth allocation unit then sorts all gradients to be transmitted according to the global gradient importance score, executes a dynamic bandwidth allocation strategy, allocates high bandwidth to gradient tensors with high importance scores, prioritizes the transmission of high-priority gradient tensors at the network link layer, and performs gradient aggregation and model updates. Then, at the start of a new round of model training, the gradient importance score is re-evaluated based on the new model, and the process returns to the compute nodes to perform local training.

[0066] The bandwidth allocation unit is a centralized or distributed controller that collects gradient tensor information (gradient tensors to be transmitted and their metadata) planned to be sent by all computing nodes and performs global sorting based on gradient importance scores, rather than just internal node sorting, to ensure that the most critical gradient tensors always have the highest transmission priority. The specific steps are as follows: the physical network bandwidth is divided into multiple virtual channels. The top K% of gradient tensors by gradient importance score are placed in the "high-speed channel," enjoying guaranteed high bandwidth, while the remaining gradient tensors are transmitted in the "normal channel." The weight of bandwidth allocation for each gradient tensor is proportional to its gradient importance score; that is, gradient tensors with higher gradient importance scores receive a larger share of bandwidth.

[0067] Each computing node uses local cross-modal data to perform forward computation and backward propagation, generating a set of gradient tensors {g1, g2, ..., gn}. Each computing node computes gradient importance scores in parallel, and the gradient importance scores of the gradient tensors are I_g1, I_g2, ..., I_gn, respectively.

[0068] The compute node sends the metadata of the gradient tensor to the bandwidth allocation unit. The gradient tensor itself is temporarily stored in the send buffer.

[0069] The bandwidth allocation unit sends decision scheduling instructions to network switches and nodes based on the gradient importance score ranking data and gradient importance assessment strategy, explicitly specifying the transmission priority and bandwidth limit of each gradient stream in the current time slice.

[0070] Network switches and node operating systems adjust their queue scheduling algorithms (such as weighted fair queues) according to decision scheduling instructions to prioritize the transmission of gradient tensors with high gradient importance scores.

[0071] The gradient tensors are aggregated at the gradient update target node, allowing the cross-modal model to be updated. A new round of model training begins.

[0072] By prioritizing the transmission of gradient tensors most critical to cross-modal model updates, the "quality" of each gradient synchronization is improved. Cross-modal models achieve more efficient updates with the same number of iterations, reaching the target accuracy with fewer iterations and directly reducing total training time. Allocating valuable bandwidth to the most cost-effective gradients avoids network congestion and reduces overall communication latency. This effect is particularly significant in bandwidth-constrained heterogeneous environments, breaking down the barriers between communication and computation. It enables communication strategies to be aware of training semantics and automatically identify key components at different training stages, allocating communication resources accordingly.

[0073] The technical solution of this invention calculates the gradient importance score of the gradient tensor according to a gradient importance evaluation strategy at each computing node. The gradient importance score of the gradient tensor and the target node for gradient updates are then used as gradient tensor metadata. The gradient importance scores in the gradient tensor metadata of each computing node are globally sorted to determine gradient scheduling priority. Based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated, a bandwidth upper limit threshold matching the gradient tensor is determined. Furthermore, a decision scheduling instruction is generated based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor. In this solution, the network bandwidth allocation strategy can be dynamically adjusted in real time based on the gradient importance score, ensuring priority transmission of important gradients. Simultaneously, the priority transmission order of gradients and bandwidth allocation accelerate model convergence, reducing the total model training time. This solves the problem of static bandwidth resource allocation strategies in cross-modal model training, which cannot allocate bandwidth resources on demand. It enables dynamic bandwidth resource allocation during model training, accelerating the model training process and achieving efficient utilization of network resources.

[0074] Example 3

[0075] Figure 4 This is a schematic diagram of a dynamic bandwidth allocation device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes:

[0076] The data acquisition module 310 is used to acquire the gradient tensor metadata of each computation node in the cross-modal model;

[0077] The bandwidth and scheduling priority allocation module 320 is used to globally sort the gradient importance scores in the gradient tensor metadata of each computing node, determine the gradient scheduling priority, and determine the upper limit threshold of bandwidth matching the gradient tensor based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated.

[0078] The scheduling instruction generation module 330 is used to generate decision scheduling instructions based on gradient scheduling priority and bandwidth upper limit threshold matching the gradient tensor.

[0079] The technical solution of this invention obtains the gradient tensor metadata of each computing node in a cross-modal model, then globally sorts the gradient importance scores in the gradient tensor metadata of each computing node to determine the gradient scheduling priority. Based on the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated, a bandwidth upper limit threshold matching the gradient tensor is determined. Finally, a decision scheduling instruction is generated based on the gradient scheduling priority and the bandwidth upper limit threshold matching the gradient tensor. In this solution, the network bandwidth allocation strategy can be dynamically adjusted in real time based on the gradient importance scores, ensuring priority transmission of important gradients. Simultaneously, the model convergence is accelerated based on the priority transmission order of gradients and bandwidth allocation, thereby reducing the total model training time. This solves the problem of static bandwidth resource allocation strategies during cross-modal model training, which cannot allocate bandwidth resources on demand. It enables dynamic bandwidth resource allocation during model training, speeding up the model training process and achieving efficient utilization of network resources.

[0080] Optionally, the data acquisition module 310 is used to calculate the gradient importance score of the gradient tensor according to the gradient importance evaluation strategy through each of the computing nodes; and to use the gradient importance score of the gradient tensor and the gradient update target node as the gradient tensor metadata; wherein the gradient importance evaluation strategy includes at least one of the gradient norm evaluation strategy, loss function sensitivity evaluation strategy and cross-modal prior evaluation strategy.

[0081] Optionally, the data acquisition module 310 includes a gradient importance score calculation unit, used to calculate multiple initial gradient importance scores through the current computing node when the gradient importance evaluation strategy of the current computing node is not unique; and to calculate the gradient importance score of the gradient tensor based on the multiple initial gradient importance scores and the evaluation strategy weights.

[0082] Optionally, a cross-modal prior evaluation strategy includes: determining a gradient importance score for the gradient tensor based on the gradient norm and the mapping basis weights of the source labels of the gradient tensor.

[0083] Optionally, the bandwidth and scheduling priority allocation module 320 includes a scheduling priority determination unit, used to determine the priority division ratio; to globally sort the gradient importance scores in the gradient tensor metadata of each computing node in descending order to obtain gradient importance score sorting data; and to determine the gradient scheduling priority based on the priority division ratio and the gradient importance score sorting data.

[0084] Optionally, the dynamic bandwidth allocation device further includes a communication technology overlay module for compressing the gradient tensor through each of the computing nodes, and / or for overlapping communication and computation when computing the gradient tensor.

[0085] Optionally, the dynamic bandwidth allocation device further includes a data update module, which is used to update the cross-modal model when it is determined, based on the instruction execution result of the decision scheduling instruction, that the cross-modal model does not meet the model update termination condition, and return to perform the operation of obtaining the gradient tensor metadata of each computing node in the cross-modal model.

[0086] The dynamic bandwidth allocation device provided in the embodiments of the present invention can execute the dynamic bandwidth allocation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0087] Example 4

[0088] Figure 5 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0089] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as ROM 12, RAM 13, etc., communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An I / O interface 15 is also connected to the bus 14. The ROM 12 is a read-only memory, the RAM 13 is a random access memory, and the I / O interface 15 is an input / output interface.

[0090] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0091] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as dynamic bandwidth allocation methods.

[0092] In some embodiments, the dynamic bandwidth allocation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the dynamic bandwidth allocation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the dynamic bandwidth allocation method by any other suitable means (e.g., by means of firmware).

[0093] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0097] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0098] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS servers, such as high management difficulty and weak business scalability.

[0099] This application also discloses a computer program product, which includes a computer program that, when executed by a processor, implements the dynamic bandwidth allocation method provided in any embodiment of this application. This program product shares the same inventive concept as the dynamic bandwidth allocation method disclosed in the embodiments of this application, and therefore will not be described in detail here.

[0100] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0101] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A dynamic bandwidth allocation method, characterized by, The method comprises the following steps: obtaining gradient tensor metadata of each computing node in the cross-modal model; globally sorting the gradient importance scores in the gradient tensor metadata of each computing node, determining a gradient scheduling priority, and determining a bandwidth upper limit threshold matched with the gradient tensor according to the gradient importance scores in the gradient tensor metadata of each computing node and the total bandwidth to be allocated; generating a decision scheduling instruction according to the gradient scheduling priority and the bandwidth upper limit threshold matched with the gradient tensor.

2. The method of claim 1, wherein, The method of obtaining gradient tensor metadata of each computing node in the cross-modal model comprises the following steps: calculating, by each computing node, a gradient importance score of a gradient tensor according to a gradient importance evaluation strategy; taking the gradient importance score of the gradient tensor and a gradient update destination node as the gradient tensor metadata; wherein the gradient importance evaluation strategy comprises at least one of a gradient norm evaluation strategy, a loss function sensitivity evaluation strategy, and a cross-modal prior evaluation strategy.

3. The method of claim 2, wherein, The method of calculating, by each computing node, a gradient importance score of a gradient tensor according to a gradient importance evaluation strategy comprises the following steps: when the gradient importance evaluation strategy of the current computing node is not unique, calculating, by the current computing node, a plurality of initial gradient importance scores; calculating the gradient importance score of the gradient tensor according to the plurality of initial gradient importance scores and an evaluation strategy weight.

4. The method of claim 2, wherein, The cross-modal prior evaluation strategy comprises determining the gradient importance score of the gradient tensor according to a gradient norm and a mapping basis weight of a source label of the gradient tensor.

5. The method of claim 1, wherein, The method of globally sorting the gradient importance scores in the gradient tensor metadata of each computing node, determining a gradient scheduling priority, comprises the following steps: determining a priority division ratio; globally sorting the gradient importance scores in the gradient tensor metadata of each computing node in descending order to obtain gradient importance score sorting data; determining the gradient scheduling priority according to the priority division ratio and the gradient importance score sorting data.

6. The method of claim 1, wherein, Before generating the decision scheduling instruction according to the gradient scheduling priority and the bandwidth upper limit threshold matched with the gradient tensor, the method further comprises the following steps: compressing the gradient tensor by each computing node, and / or overlapping communication and calculation when calculating the gradient tensor.

7. The method of claim 1, wherein, After generating the decision scheduling instruction according to the gradient scheduling priority and the bandwidth upper limit threshold matched with the gradient tensor, the method further comprises the following steps: when it is determined according to an instruction execution result of the decision scheduling instruction that the cross-modal model does not meet a model update end condition, updating the cross-modal model and returning to perform the operation of obtaining gradient tensor metadata of each computing node in the cross-modal model.

8. An electronic device, comprising: The electronic device comprises: at least one processor and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the dynamic bandwidth allocation method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the dynamic bandwidth allocation method of any one of claims 1-7 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by the processor, implements the dynamic bandwidth allocation method according to any one of claims 1-7.

Images