A method for managing a request message in a network computing, a switch, a device and a medium

Abstract

This invention proposes an on-network computing request message management method, switch, device, and medium, relating to the field of network communication technology. The method includes: allocating corresponding cache space based on read request messages sent by the GPU; generating a state table corresponding to the read request message and refreshing the state table to a first state; scanning the cache space according to a preset time; if no feedback message corresponding to the read request message is found, refreshing the state table to a second state; the second state is different from the first state; generating a corresponding aging opcode based on the second state and clearing the cache space; feeding back the aging opcode to the GPU so that the GPU releases memory associated with the read request message. This invention can achieve timely processing of message-related storage space, saving system resources.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and more specifically, to an online computing request message management method, switch, device, and medium. Background Technology

[0002] AIGC (AI Generated Content) refers to content generated by artificial intelligence. To generate this complex and high-fidelity content, it is usually necessary to train large-scale deep learning models, which contain hundreds of millions or even trillions of parameters. Training such large models often requires GPU clusters, which interconnect thousands of GPUs to form a massive parallel computing network.

[0003] In the field of GPU interconnect networks, many manufacturers have begun to adopt SHARP (Scalable Hierachial Aggregation and Reduction Protocol) to implement aggregated communication. This method offloads some computing tasks to the switch chip that interconnects with the GPU to obtain local data, and then aggregates the local data on multiple nodes into a global result through All Reduce.

[0004] The root GPU initiates a data read request to the switch, which forwards the request to other GPUs. The switch then collects feedback data from these other GPUs, calculates the global result, and sends the global result back to the root GPU. However, during All Reduce, if either the GPU or the switch fails to respond, the other will remain in a waiting state for an extended period, resulting in wasted resources. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an on-network computing request message management method, switch, device and medium to solve the problem that existing message requests may occupy memory space for a long time, resulting in resource waste.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide an on-network computing request message management method, applied to a switching device, the method comprising: Allocate corresponding cache space based on the read request message sent by the GPU; Generate a status table corresponding to the read request message, and refresh the status table to the first status; The cache space is scanned according to a preset time. If no feedback message corresponding to the read request message is found, the status table is refreshed to the second status. The second status is different from the first status. Based on the second state, a corresponding aging opcode is generated and the cache space is cleared. The aging opcode is then fed back to the GPU so that the GPU releases the memory associated with the read request message.

[0007] In an optional implementation, the step of generating a status table corresponding to the read request message and refreshing the status table to the first state includes: Generate a request status valid status table and a request status aging status table corresponding to the read request message; Refresh both the request status valid status table and the request status aging status table to the first status.

[0008] In an optional implementation, the step of scanning the cache space according to a preset time, and refreshing the status table to the second state if no feedback message corresponding to the read request message is found, includes: The cache space is scanned for the first time according to the first scan time generated by the preset timer. If the first round of scanning does not detect the feedback message, the request status validity status table is refreshed to the second status. The cache space is scanned a second time according to the second scan time generated by the preset timer. If the second round of scanning does not detect the feedback message, the request status aging status table is refreshed to the second status.

[0009] In an optional implementation, the step of generating a corresponding aging opcode based on the second state and clearing the cache space includes: If the request status aging status table is refreshed to the second status, then the aging operation code is generated and the cache space is cleared.

[0010] In an optional implementation, the step of allocating corresponding cache space based on the read request message sent by the GPU includes: Determine the root GPU that sent the read request message, and determine the sub-GPU that processes the read request message; The corresponding cache space is allocated according to the sub-GPU and the read request message.

[0011] In an optional implementation, the step of refreshing the status table to the second state if no feedback message corresponding to the read request message is found includes: If all the message information returned by the sub-GPUs according to the read request message is collected, then the feedback message is generated according to the message information; If the feedback message is not generated within the preset time, the status table will be refreshed to the second status.

[0012] In an optional implementation, the method further includes: If the feedback message is generated at the preset time, the status table is refreshed to the first status. A global reduction result is generated based on the feedback message, and a result feedback instruction is generated based on the first state. The global reduction result is then fed back to the root GPU based on the result feedback instruction.

[0013] Secondly, embodiments of the present invention provide a switch, comprising: The allocation module is used to allocate corresponding cache space according to the read request message sent by the GPU; The refresh module is used to generate a status table corresponding to the read request message and refresh the status table to the first status. An aging module is used to scan the cache space according to a preset time. If no feedback message corresponding to the read request message is found, the status table is refreshed to a second status. The second status is different from the first status. The sending module is used to generate a corresponding aging opcode based on the second state and clear the cache space, and feed the aging opcode back to the GPU so that the GPU releases the memory associated with the read request message.

[0014] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the on-network computing request message management method described in the first aspect.

[0015] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the on-network computing request message management method described in the first aspect.

[0016] This invention provides an online computing request message management method, switch, device, and medium. By allocating corresponding cache space for read request messages and constructing corresponding status tables for them, the cache space is periodically scanned. If no feedback message corresponding to the read request message is found, the cache space is cleared and the related memory is released. This enables timely processing of message-related storage space and saves system resources.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown; Figure 2 The diagram illustrates a flowchart of an online computing request message management method provided by an embodiment of the present invention. Figure 3 This diagram illustrates the principle of an online computing request message management method provided by an embodiment of the present invention. Figure 4 A functional block diagram of a switch provided by an embodiment of the present invention is shown.

[0020] icon: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 400 - Switch; 401 - Distribution module; 402 - Refresh module; 403 - Aging module; 404 - Transmitting module. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] Please refer to Figure 1 , Figure 1 This is a block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0025] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0026] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0027] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0028] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0029] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating an on-network computing request message management method provided in this embodiment. The method is applied to a switching device and includes: S201. Allocate the corresponding cache space according to the read request message sent by the GPU.

[0030] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the principle of an on-network computing request message management method provided in this embodiment.

[0031] The GPU sends a Load Data Request (LDR) to the switch. Upon receiving the LDR, the switch allocates a buffer index (used to store subsequent LDR_DATA messages from member GPUs), edits the message by adding the buffer index to the message header, and then broadcasts the edited message. This approach can be applied to AllReduce scenarios; therefore, the switch typically broadcasts the LDR instruction to multiple other GPUs. Thus, the GPU sending the load data request can be defined as the Root GPU, and the switch will also broadcast the LDR instruction to the Root GPU.

[0032] S202. Generate a status table corresponding to the read request message, and refresh the status table to the first status.

[0033] The status table is generated after the switch has allocated the cache space. It is used to record the cache status in the cache space. When the cache space has just been allocated, there is no data stored in the cache space. At this time, the status of the cache space is refreshed to the first status.

[0034] S203. Scan the cache space according to a preset time. If no feedback message corresponding to the read request message is found, refresh the status table to the second status. The second status is different from the first status.

[0035] When the switch is in an All Reduce scenario, if other GPUs do not provide feedback related to the data read request, the switch will remain in a waiting state. To avoid this, a timer can be set to scan the cache space at a specific interval. If feedback packets from other GPUs have been received in the cache space, they can be processed. If no feedback packets from other GPUs are found in the cache space, the corresponding state table is refreshed to the second state to distinguish it from the first state when the state table was first created.

[0036] S204. Generate a corresponding aging opcode according to the second state and clear the cache space. Feed back the aging opcode to the GPU so that the GPU releases the memory associated with the read request message.

[0037] When the cache space is determined to be in the second state at the preset time, the process corresponding to the read request message can be identified as the aging process. At this time, the cache space can be cleared, and then an aging opcode can be generated for the GPU to release the memory associated with the read request message, thus preventing the memory from being occupied for a long time.

[0038] This embodiment allocates corresponding cache space for read request messages and builds a corresponding status table for them. Then, by periodically scanning the cache space, if no feedback message corresponding to the read request message is found, the cache space is cleared and the related memory is released. This enables timely processing of the storage space related to the message and saves system resources.

[0039] In one implementation, the step of generating a status table corresponding to the read request message and refreshing the status table to a first state includes: Generate a request status valid status table and a request status aging status table corresponding to the read request message; Refresh both the request status valid status table and the request status aging status table to the first status.

[0040] After allocating the buffer space, the switch generates two entries to record the LDR Valid State and LDR Aging State, respectively. These two entries are defined as the Request State Valid State Table and the Request State Aging State Table.

[0041] The request status valid status table and the request status aging status table are indexed using a Buffer Index, and the initial status of both tables is set to the first state, such as 1.

[0042] This embodiment configures two request status valid status tables and request status aging status tables for the cache space to record the cache status of the cache space, so as to clean up the cache space that does not meet the requirements in a timely manner, thereby saving system resources.

[0043] In one implementation, the step of scanning the cache space according to a preset time and refreshing the status table to a second state if no feedback message corresponding to the read request message is found includes: The cache space is scanned for the first time according to the first scan time generated by the preset timer. If the first round of scanning does not detect the feedback message, the request status validity status table is refreshed to the second status. The cache space is scanned a second time according to the second scan time generated by the preset timer. If the feedback message is not detected in the second round of scanning, the request status aging status table is refreshed to the second status.

[0044] During scanning, all Buffer Indices can be scanned to index both the request state active state table and the request state aging state table. The first scan time and the second scan time can be the same or different.

[0045] If no LDR_DATA message is received from other GPUs based on the read request message during the first round of scanning, the status of the request status valid status table is refreshed to the second status, such as refreshing to 0. If no LDR_DATA message is received from other GPUs based on the read request message during the second round of scanning, the status of the request status aging status table is refreshed to the second status, such as refreshing to 0.

[0046] If the request status aging status table is refreshed to the second status, then the aging operation code is generated and the cache space is cleared.

[0047] After the status of the request status aging table is refreshed to 0, that is, LDR Valid is cleared to 0, the switch will edit the read request message and generate an LDR_RETURN message representing the aging opcode. The LDR_RETURN message is sent to the root GPU, and the cache space is cleared for use by other processes.

[0048] This embodiment uses a timer to scan the request status valid status table and the request status aging status table respectively, and updates the status table through the aging mechanism. This achieves timely processing of message feedback information and timely cache clearing of messages that have not been responded to for a long time, thus saving system resources.

[0049] In one implementation, the step of allocating corresponding cache space based on the read request message sent by the GPU includes: Determine the root GPU that sent the read request message, and determine the sub-GPU that processes the read request message; The corresponding cache space is allocated according to the sub-GPU and the read request message.

[0050] like Figure 3 As shown, after receiving a read request packet from the root GPU, the switch stores the read request packet and allocates buffer space. Then, it broadcasts the read request packet to other GPUs. Each GPU processes the read request packet accordingly and sends back the processing result. Different GPUs can allocate corresponding buffer space for their processing results. Finally, the switch performs a reduction calculation on all the processing results and sends the reduction calculation result back to the root GPU.

[0051] This embodiment allocates corresponding cache space according to the sub-GPU and read request message, realizes the orderly storage and management of feedback information of each sub-GPU, and improves the management efficiency of network request message.

[0052] In one implementation, the step of refreshing the status table to the second state if no feedback message corresponding to the read request message is found includes: If all the message information returned by the sub-GPUs according to the read request message is collected, then the feedback message is generated according to the message information; If the feedback message is not generated within the preset time, the status table will be refreshed to the second status.

[0053] In an All Reduce scenario, each sub-GPU needs to process relevant message information and return the processing result to the switch. If some sub-GPUs fail to return a result, the switch cannot perform reduction calculations. A feedback message indicates that all sub-GPUs have returned a result. Therefore, the switch typically only performs reduction calculations upon receiving a feedback message. If no feedback message is received within a preset time, the state table can be refreshed to the second state, causing the switch to stop waiting and perform subsequent operations such as releasing cache space.

[0054] If the feedback message is generated at the preset time, the status table is refreshed to the first status. A global reduction result is generated based on the feedback message, and a result feedback instruction is generated based on the first state. The global reduction result is then fed back to the root GPU based on the result feedback instruction.

[0055] If a feedback message is generated normally within the preset time, it means that the processing results of all sub-GPUs have been collected. At this time, the switch can perform reduction calculation to obtain the global reduction result, and then send the global reduction result to the root GPU.

[0056] This embodiment monitors the feedback messages periodically to determine whether the switch's cache space has collected all the processing results fed back by the sub-GPUs within a preset time. If the collection is complete, the processing results are promptly reduced and calculated and fed back to the root GPU. If the collection is not complete, the status table is updated in a timely manner so that the switch can release the cache space in a timely manner and avoid memory resources being occupied for a long time.

[0057] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a switch is given below. Please refer to [link / reference]. Figure 4 , Figure 4 This is a functional block diagram of a switch provided in an embodiment of the present invention. It should be noted that the switch provided in this embodiment has the same basic principle and technical effects as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The switch 400 includes: The allocation module 401 is used to allocate corresponding cache space according to the read request message sent by the GPU; The refresh module 402 is used to generate a status table corresponding to the read request message and refresh the status table to the first status. The aging module 403 is used to scan the cache space according to a preset time. If no feedback message corresponding to the read request message is found, the status table is refreshed to the second status. The second status is different from the first status. The sending module 404 is used to generate a corresponding aging opcode according to the second state and clear the cache space, and feed the aging opcode back to the GPU so that the GPU releases the memory associated with the read request message.

[0058] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be used by... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0059] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0060] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0061] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for managing on-network computing request messages, characterized in that, Applied to a switch, the method includes: Allocate corresponding cache space based on the read request message sent by the GPU; Generate a status table corresponding to the read request message, and refresh the status table to the first status; The cache space is scanned according to a preset time. If no feedback message corresponding to the read request message is found, the status table is refreshed to the second status. The second status is different from the first status. Based on the second state, a corresponding aging opcode is generated and the cache space is cleared. The aging opcode is then fed back to the GPU so that the GPU releases the memory associated with the read request message.

2. The method for managing on-network computing request messages according to claim 1, characterized in that, The step of generating a status table corresponding to the read request message and refreshing the status table to the first state includes: Generate a request status valid status table and a request status aging status table corresponding to the read request message; Refresh both the request status valid status table and the request status aging status table to the first status.

3. The method for managing on-network computing request messages according to claim 2, characterized in that, The step of scanning the cache space according to a preset time, and refreshing the status table to the second state if no feedback message corresponding to the read request message is found, includes: The cache space is scanned for the first time according to the first scan time generated by the preset timer. If the first round of scanning does not detect the feedback message, the request status validity status table is refreshed to the second status. The cache space is scanned a second time according to the second scan time generated by the preset timer. If the feedback message is not detected in the second round of scanning, the request status aging status table is refreshed to the second status.

4. The method for managing on-network computing request messages according to claim 3, characterized in that, The step of generating the corresponding aging opcode based on the second state and clearing the cache space includes: If the request status aging status table is refreshed to the second status, then the aging operation code is generated and the cache space is cleared.

5. The method for managing on-network computing request messages according to claim 1, characterized in that, The step of allocating corresponding cache space based on the read request message sent by the GPU includes: Determine the root GPU that sent the read request message, and determine the sub-GPU that processes the read request message; The corresponding cache space is allocated according to the sub-GPU and the read request message.

6. The method for managing on-network computing request messages according to claim 5, characterized in that, The step of refreshing the status table to the second state if no feedback message corresponding to the read request message is found includes: If all the message information returned by the sub-GPUs according to the read request message is collected, then the feedback message is generated according to the message information; If the feedback message is not generated within the preset time, the status table will be refreshed to the second status.

7. The method for managing on-network computing request messages according to claim 6, characterized in that, The method further includes: If the feedback message is generated at the preset time, the status table is refreshed to the first status. A global reduction result is generated based on the feedback message, and a result feedback instruction is generated based on the first state. The global reduction result is then fed back to the root GPU based on the result feedback instruction.

8. A switch, characterized in that, include: The allocation module is used to allocate corresponding cache space according to the read request message sent by the GPU; The refresh module is used to generate a status table corresponding to the read request message and refresh the status table to the first status. An aging module is used to scan the cache space according to a preset time. If no feedback message corresponding to the read request message is found, the status table is refreshed to a second status. The second status is different from the first status. The sending module is used to generate a corresponding aging opcode based on the second state and clear the cache space, and feed the aging opcode back to the GPU so that the GPU releases the memory associated with the read request message.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the on-network computing request message management method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the online computing request message management method as described in any one of claims 1-7.

Images