Switch resource recovery method and system

Abstract

The invention provides a method and system for recycling resources of a switch, the switch is connected with at least one computing node, the at least one computing node runs a plurality of processes, the plurality of processes form a communication domain to communicate with each other, and the method comprises the following steps: the switch receives data sent by the plurality of processes and processes the data; a first node in the at least one computing node determines that a first process in the plurality of processes exits abnormally and sends an exception notification message to the switch, the exception notification message is used for indicating that the first process exits abnormally, and the switch obtains the exception notification message from the first node and sends the exception notification message to the first node; and releasing resources required for processing the data according to the exception notification message. According to the method, the switch can release the resources required for processing the data sent between the processes when the first process abnormally exits, meanwhile, additional configuration on the switch can be avoided, and the universality of the switch is improved.

Description

Technical Field

[0001] The present application relates to the field of storage, and more specifically, to a method and system for reclaiming switch resources. Background Art

[0002] A process is a computer program running on a data set. It is the basic unit for resource allocation and the foundation of the operating system structure. Multiple processes running on at least one computing node can form a communication domain and communicate with each other.

[0003] A switch is a network device used for forwarding electrical (optical) signals. It can provide an exclusive electrical signal path for any two network nodes connected to the switch. As an example, the switch can connect to at least one of the above-mentioned computing nodes and forward data sent between multiple processes running in the at least one computing node. The switch can not only forward data sent between multiple processes, but also undertake the task of processing data sent between the multiple processes, that is, the switch has the function of in-network computing (INC). When a process in the communication domain exits abnormally, the INC program in the switch will also exit abnormally. However, when the INC program in the switch exits abnormally, the storage and other resources applied for in the switch cannot be released. Over time, the resources that are not cleaned up in time in the switch will cause the switch's resources to be exhausted, resulting in the loss of the switch's in-network computing INC capability.

[0004] In a related technical solution, an additional process is run in the switch. The process needs to send packets from the switch to the process and needs to process the messages fed back by the process. On the one hand, this requires strong customization of the switch and has very poor versatility. On the other hand, when the switch polls and sends packets to the process, it needs to occupy a certain amount of network bandwidth, which has a performance impact on other network communications.

[0005] Therefore, how to enable the switch to reclaim the resources required for processing data sent between processes when a process in the communication domain exits abnormally while improving the versatility of the switch has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present application provides a method and system for reclaiming switch resources. The method can enable the switch to reclaim resources required for processing data sent between processes when a process in a communication domain exits abnormally, while also improving the versatility of the switch.

[0007] In a first aspect, a method for reclaiming switch resources is provided, wherein the switch is connected to at least one computing node, the at least one computing node runs multiple processes, the multiple processes constitute a communication domain for communicating with each other, the method comprising: the switch receives data sent by the multiple processes and processes the data; a first node among the at least one computing node determines that a first process among the multiple processes exits abnormally, and sends an abnormal notification message to the switch, the abnormal notification message is used to indicate that the first process exits abnormally; the switch obtains the abnormal notification message from the first node, and releases resources required for processing the data according to the abnormal notification message.

[0008] In the above technical solution, after the first node monitors that the first process has abnormally exited, it sends an abnormal notification message to the switch to indicate that the first process has abnormally exited. After receiving the abnormal notification message sent by the first node, the switch can recycle the resources required for processing the data sent between processes in the switch. In this way, the switch can not only release the resources required for processing the data sent between processes in the switch after the first process has abnormally exited, but also because the switch determines that the first process has abnormally exited based on the abnormal notification message sent on the first node, there is no need for the switch to send additional packets to the first process to monitor the status of the first process, thereby avoiding additional configuration of the switch, making the switch more versatile.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the switch sending the abnormal notification message to other processes in the multiple processes, so that the other processes stop communicating with the first process according to the abnormal notification message.

[0010] In the above technical solution, when the first process exits abnormally, the switch will send an abnormal notification message indicating that the first process exits abnormally to other processes in the multiple processes, so that the other processes in the multiple processes will stop communicating with the first process, thereby avoiding unnecessary communication overhead.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the switch obtains multiple exception notification messages from a first node among the at least one computing node, and the method also includes: the switch sends the multiple exception notification messages to other processes among the multiple processes respectively, so that the other processes stop communicating with the first process respectively according to the exception notification messages they have received.

[0012] In the above technical solution, when the daemon process determines that the first process has exited abnormally, it can send messages indicating that the first process has exited abnormally to the remaining multiple processes in the communication domain. Since there are many processes in the communication domain, the switch only needs to correctly receive and parse a message indicating that the first process has exited abnormally to release the resources on the switch normally, so it will tolerate a certain packet loss rate. And as the node scale increases, a higher packet loss rate is allowed.

[0013] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the daemon process in the first node determines that the first process exits abnormally, and the daemon process is used to monitor whether the first process exits abnormally; the daemon process sends the abnormal notification message to the switch.

[0014] In combination with the first aspect, in some implementations of the first aspect, the switch receives the abnormal notification message sent by the daemon process in the first node.

[0015] In combination with the first aspect, in some implementations of the first aspect, the exception notification message reuses a notification message sent by the first process when the first process exits normally.

[0016] In the above technical solution, the above abnormal exit message can reuse the normal exit message, that is, the abnormal notification message reuses the notification message sent by the first process when the first process normally exits. In this way, the abnormal exit message can be parsed without additional configuration or modification of the switch, thereby further improving the versatility of the switch.

[0017] In combination with the first aspect, in some implementations of the first aspect, the switch releases resources required for the data in a garbage collection manner according to the exception notification message.

[0018] In combination with the first aspect, in some implementations of the first aspect, the communication domain is a message passing interface MPI communication domain.

[0019] In combination with the first aspect, in some implementations of the first aspect, the daemon process obtains a return value of an MPI function of the first process during its execution; the daemon process determines that the first process exits abnormally based on the return value as an error code.

[0020] In a second aspect, a system for switch resource recovery is provided, the system comprising a switch and at least one computing node, the at least one computing node running multiple processes, the multiple processes constituting a communication domain for communicating with each other, wherein the switch is used to receive data sent by the multiple processes and process the data; the first node is used to determine that a first process among the multiple processes has exited abnormally, and send an abnormal notification message to the switch, the abnormal notification message being used to indicate that the first process has exited abnormally; the switch is also used to obtain the abnormal notification message from the first node, and release resources required for processing the data according to the abnormal notification message.

[0021] In combination with the second aspect, in certain implementations of the second aspect, the switch is also used to obtain multiple exception notification messages from the first node, and send the multiple exception notification messages to other processes in the multiple processes respectively, so that the other processes can stop communicating with the first process respectively according to the exception notification messages they have received.

[0022] In combination with the second aspect, in some implementations of the second aspect, the exception notification message reuses a notification message sent by the first process when the first process exits normally.

[0023] In combination with the second aspect, in certain implementations of the second aspect, a daemon process is running in the first node, and the first node is specifically used for: the daemon process obtains the return value of the MPI function of the first process during its running; the daemon process determines that the first process exits abnormally based on the return value as an error code.

[0024] In combination with the second aspect, in some implementations of the second aspect, the communication domain is a message passing interface MPI communication domain.

[0025] In a third aspect, a switch is provided, comprising a processor and a memory; the processor is used to execute instructions stored in the memory of the switch, so that the switch executes the steps and methods executed by the switch in the first aspect or any possible implementation of the first aspect.

[0026] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0027] In a fourth aspect, a computing node is provided, comprising a processor and a memory; the processor is used to execute instructions stored in the memory of the computing node, so that the computing node executes the steps or methods performed by the computing node in the first aspect or any possible implementation of the first aspect.

[0028] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0029] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing node and a switch, the computing node and the switch execute the method in the first aspect and any one of the implementations of the first aspect.

[0030] In a sixth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing node and a switch, the computing node and the switch execute the method in the first aspect and any one of the implementations of the first aspect.

[0031] By way of example, these computer-readable storages include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.

[0032] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic block diagram of a scenario applicable to an embodiment of the present application.

[0034] Figure 2 It is a schematic flow chart of a method for switch resource recovery provided in an embodiment of the present application.

[0035] Figure 3 It is a schematic block diagram provided by an embodiment of the present application, in which a daemon process 1 sends a message to process 130 and process 140 indicating that process 120 has exited abnormally.

[0036] Figure 4 It is a schematic block diagram of a switch resource recovery system 400 provided in an embodiment of the present application.

[0037] Figure 5 It is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0039] The present application will present various aspects, embodiments or features around a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0040] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0041] In the embodiments of the present application, "corresponding (corresponding, relevant)" and "corresponding (corresponding)" can sometimes be used interchangeably. It should be pointed out that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0042] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0043] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0044] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0045] With the increase of network bandwidth, the reduction of latency and the emergence of technologies such as remote direct memory access (RDMA), researchers have begun to try to offload some computing operations in scientific computing tasks from the central processing unit (CPU) to specific network communication devices (such as switches), so as to further reduce communication overhead and CPU load while efficiently utilizing the high forwarding capabilities of network devices. Therefore, how to fully utilize the computing power of network devices to achieve communication optimization and reduce communication costs has become one of the research hotspots in the current high-performance computing field.

[0046] The message passing interface (MPI) is the de facto standard for programming in the field of high-performance computing (HPC). Many HPC applications are written using MPI. MPI can also be understood as a common parallel programming model in the design of parallel scientific applications, or a common parallel computing application interface in high-performance computing software, often programmed in non-shared memory environments such as supercomputers and computer clusters. It defines a set of communication protocols that developers can use to perform parallel computing on different hardware and operating systems. The main purpose of MPI is to pass messages in a distributed system so that different processors can communicate with each other and coordinate their work.

[0047] In MPI programs, an individual that participates in communication independently is called a process. In MPI programs, an ordered set consisting of some or all processes is called a process group. Each process in a process group is assigned a sequence number in the group, called a process rank, and the value range of the process number starts from 0. MPI's inter-process communication is achieved through message passing. A message refers to a data exchange between processes. In MPI, the processes that send and receive messages need to perform synchronization operations to ensure the correct delivery of messages.

[0048] The main features of MPI include:

[0049] 1. Parallelism: MPI allows multiple programs to be executed simultaneously on multiple processors and supports distributed memory systems.

[0050] 2. Portability: MPI is a standard that can be used on a variety of different hardware and operating systems, which allows developers to easily migrate programs from one platform to another.

[0051] 3. Flexibility: MPI provides rich message passing capabilities, which allows for easy development and debugging on clusters of different sizes.

[0052] 4. Efficiency: MPI uses efficient communication algorithms and data structures, which can greatly improve the performance of distributed systems.

[0053] An important feature of MPI is in-network computing (INC). As the name suggests, INC refers to offloading the computing part of MPI in the communication process to the switch for execution. At this time, the switch can not only forward messages, but also undertake simple computing work.

[0054] Figure 1 is a schematic block diagram of a scenario applicable to an embodiment of the present application. Figure 1 As shown, the scenario may include a switch 110 , a process 120 , a process 130 , and a process 140 .

[0055] It should be understood that the embodiment of the present application does not specifically limit the number of processes in the above scenario. For the convenience of description, Figure 1 The following is an explanation using three processes as examples.

[0056] See also Figure 1, the processes 120, 130 and 140 communicate with each other based on the MPI protocol, and therefore, the processes 120, 130 and 140 may also be referred to as the MPI processes 120, 130 and 140. Specifically, the processes 120, 130 and 140 cannot communicate directly with each other, and when the processes 120, 130 and 140 need to send data, the data needs to be sent to the switch 110, and the switch 110 forwards the data.

[0057] The switch 110 is a switch with INC function, that is, the switch 110 can not only forward the data to be sent between each process, but also undertake the calculation work. For example, after the switch 110 receives the data to be sent between the process 120, the process 130 and the process 140, it caches the data to be calculated on the network and performs corresponding calculations, and sends the final calculation results to the process 120, the process 130 and the process 140. In this way, by using the switch 110 to participate in the calculation, the amount of forwarded data and the amount of calculation of each process can be reduced, so as to achieve the effect of saving bandwidth and reducing latency, which is also helpful to improve the computing power of the process.

[0058] It should be noted that the above-mentioned process 120, process 130 and process 140 can be run on one server, or can also be run on different servers respectively, and the embodiment of the present application does not make any specific limitation on this.

[0059] When process 120, process 130 and process 140 all exit normally (for example, the program ends, or the system function exit() is called), the INC program in switch 110 also exits normally, and switch 110 releases the storage and other resources it applied for. Specifically, any one of process 120, process 130 and process 140 will send a normal exit message to switch 110, and switch 110 can release the storage and other resources it applied for according to the normal exit message.

[0060] When any of the processes 120, 130, and 140 exits abnormally (for example, the user stops the program using "Ctrl+C", the user terminates the process using the kill command, or the process exits abnormally due to its own program bug), the INC program in the switch 110 will also exit abnormally. However, when the INC program in the switch 110 exits abnormally, the storage and other resources applied for in the switch 110 cannot be released. Over time, the resources in the switch 110 that are not cleaned up in time will cause the resources of the switch 110 to be exhausted, resulting in the loss of the INC computing capability of the switch 110 on the network, which must be solved by restarting the switch 110.

[0061] Therefore, it is crucial to establish a resource recovery mechanism for the switch in the event of abnormal exit of INC.

[0062] In a related technical solution, an additional process is run in the switch. The process needs to send packets from the switch to the process on the server side and needs to process the messages fed back by the process on the server side. On the one hand, this requires strong customization of the switch and has very poor versatility. On the other hand, when the switch polls and sends packets to the server, it needs to occupy a certain amount of network bandwidth, which has a performance impact on other network communications.

[0063] In view of this, an embodiment of the present application provides a method for recovering switch resources, which can recover the resources applied for in-network computing INC in the switch when the process exits abnormally, and at the same time avoid additional configuration of the switch, thereby improving the versatility of the switch.

[0064] Below Figure 1 Take the scenario shown in the figure as an example, combined with Figure 2 A method for reclaiming switch resources provided in an embodiment of the present application is described in detail. It should be understood that Figure 2 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figure 2 The specific numerical values ​​or specific scenarios shown in the examples. Figure 2 The examples given below are obviously susceptible to various equivalent modifications or changes, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0065] Figure 2 FIG. 1 is a schematic flow chart of a method for recycling switch resources provided in an embodiment of the present application. Figure 2 As shown, the method may include steps 210-230, and steps 210-230 are described in detail below.

[0066] Step 210: The daemon process monitors whether the first process exits abnormally or normally.

[0067] In the embodiment of the present application, an additional daemon process may be run in the server where each process (also referred to as an MPI process) is located. Each daemon process is used to monitor whether the corresponding process (also referred to as an MPI process) exits abnormally or normally.

[0068] For example, daemon process 1 may be additionally run in the server where process 120 is located, and daemon process 1 is used to monitor whether process 120 exits abnormally or normally. Here, process 120 corresponds to the first process mentioned above, and daemon process 1 corresponds to the daemon process of the first process mentioned above. For another example, daemon process 2 may be additionally run in the server where process 130 is located, and daemon process 2 is used to monitor whether process 130 exits abnormally or normally. Process 130 corresponds to the first process mentioned above, and daemon process 2 corresponds to the daemon process of the first process mentioned above. For another example, daemon process 3 may be additionally run in the server where process 140 is located, and daemon process 3 is used to monitor whether process 140 exits abnormally or normally. Here, process 140 corresponds to the first process mentioned above, and daemon process 3 corresponds to the daemon process of the first process mentioned above.

[0069] For example, the daemon process is daemon process 1, and the first process is process 120. In a possible implementation, since process 120 returns a return value of an MPI function during operation, daemon process 1 obtains the return value of the MPI function and determines whether process 120 exits abnormally or normally based on the return value of the MPI function.

[0070] It should be understood that since the process 120 and other processes in the communication domain (e.g., process 130, process 140) communicate based on the MPI protocol, in MPI, the function names are in the form of MPI_Xxxx, such as MPI_Send, MPI_Type_commit, MPI_COMM_WORLD, etc. The return value of these functions is usually an integer value, indicating whether the operation is successful or not. If the return value is MPI_SUCCESS (0), it means that the operation is successful, and the process 120 is in the normal exit situation; otherwise, if the return value is a non-zero value, it indicates the error code of the operation, and the process 120 is in the abnormal exit situation.

[0071] It should be noted that different MPI functions may have different return values ​​and error codes. Therefore, when using a specific MPI function, you need to consult the corresponding documentation to understand its specific return value and error code meaning.

[0072] Step 220: When the daemon process determines that the first process exits abnormally, it sends an abnormal exit message to the switch.

[0073] In the embodiment of the present application, assuming that the daemon process obtains a non-zero MPI return value returned by the first process, the daemon process can determine that the first process is abnormally exited. The embodiment of the present application does not specifically limit the reason for the abnormal exit of the first process. For example, the first process may be abnormally exited due to a program bug of the first process caused by the user using "Ctrl+C" to stop the first process, or due to the user using the kill command to terminate the first process, or due to the server where the first process is located being down (failure caused by hardware failure, software problem, network problem or power failure, etc.), resulting in the abnormal exit of the first process, etc.

[0074] As an example, when the daemon process determines that the first process exits abnormally, it may send an abnormal exit message to the switch.

[0075] Step 230: After obtaining the above abnormal exit message, the switch releases the resources applied for the in-network computing INC.

[0076] In the embodiment of the present application, when the daemon process detects that the first process has exited abnormally, the daemon process will send a message indicating that the first process has exited abnormally to other processes in the communication domain, so as to notify other processes in the communication domain that the first process has exited abnormally. After receiving the message indicating that the first process has exited abnormally, the other processes in the communication domain will stop communicating with the first process.

[0077] Specifically, when the daemon process sends a message indicating that the first process has abnormally exited to other processes in the communication domain, the daemon process will send a message indicating that the first process has abnormally exited to the switch, and the switch will forward the message indicating that the first process has abnormally exited to other processes in the communication domain. When the switch forwards the message indicating that the first process has abnormally exited sent by the daemon process, it will parse the message indicating that the first process has abnormally exited. When it is determined that the first process has abnormally exited according to the result of the parsing, the switch will release the resources applied for the INC.

[0078] The above abnormal exit message may also be referred to as an abnormal notification message, and the abnormal notification message is used to indicate that the first process has abnormally exited.

[0079] In a possible implementation, the switch may release the resources applied for the INC by garbage collection. Specifically, the switch automatically identifies and releases the memory space that is no longer used by the garbage collection mechanism.

[0080] It should be understood that the resources requested in the switch for the in-network computing INC may be the processes in the communication domain processed by the switch (e.g., Figure 1 The resources required for data sent by process 120, process 130, process 140) include but are not limited to: storage resources, computing resources, etc.

[0081] In an embodiment of the present application, the daemon process may send a message indicating the abnormal exit of the first process to one of the other processes in the communication domain, so that one of the other processes may stop communicating with the first process, or the daemon process may send a message indicating the abnormal exit of the first process to multiple processes in the other processes in the communication domain, so that multiple processes in the other processes may stop communicating with the first process respectively. The embodiment of the present application does not make any specific limitation on this.

[0082] In the above technical solution, when the daemon process determines that the first process exits abnormally, it can send a message indicating that the first process exits abnormally to each of the remaining processes in the communication domain. Since there are many processes in the communication domain, the switch only needs to correctly receive and parse a message indicating that the first process exits abnormally to release the resources on the switch normally, so a certain packet loss rate will be tolerated. And as the node scale increases, a higher packet loss rate is allowed.

[0083] For ease of description, the following description is made by taking the daemon process as daemon process 1 and the first process as process 120 as an example.

[0084] An example, such as Figure 3 As shown, when the daemon process 1 determines that the process 120 exits abnormally, it can send a message indicating that the process 120 exits abnormally to the process 130 through the switch 110. After receiving the message indicating the abnormal exit from the daemon process 1, the switch 110 can send the message indicating the abnormal exit to the process 130. After receiving the message indicating that the process 120 exits abnormally, the process 130 stops communicating with the process 120.

[0085] Another example, Figure 3 As shown, when daemon process 1 determines that process 120 exits abnormally, it can send messages indicating that process 120 exits abnormally to process 130 and process 140 through switch 110. After receiving two abnormal exit messages from daemon process 1, switch 110 can send the two abnormal exit messages to process 130 and process 140 respectively. After process 130 receives a message indicating that process 120 exits abnormally, it will stop communicating with process 120. After process 140 receives a message indicating that process 120 exits abnormally, it will also stop communicating with process 120.

[0086] In an embodiment of the present application, as an example, the above abnormal exit message can reuse the normal exit message, that is, the abnormal notification message reuses the notification message sent by the first process when the first process normally exits. In this way, the abnormal exit message can be parsed without additional configuration or modification of the switch, thereby further improving the versatility of the switch.

[0087] In the above technical solution, by running an additional daemon process on the server (also called a computing node), the daemon process is used to monitor whether the process running on the server exits abnormally. When the process running on the server exits abnormally, the switch recovers the resources applied for the in-network computing INC in the switch. In this way, there is no need to start an additional process on the switch to monitor whether the process running on the server exits abnormally, thereby avoiding additional configuration of the switch, making the switch more versatile.

[0088] Combination of the above Figures 1 to 3 , describes in detail the method provided by the embodiment of the present application, and will be combined with Figure 4-Figure 5 , describes in detail the embodiment of the device of the present application. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0089] Figure 4 4 is a schematic block diagram of a switch resource recovery system 400 provided in an embodiment of the present application. The system 400 can be implemented by software, hardware, or a combination of both. The system 400 provided in an embodiment of the present application can implement the embodiment of the present application. Figure 2 The method steps in .

[0090] The system 400 includes a switch 410 and at least one computing node 420. Specifically, the switch 410 may correspond to Figure 1 The at least one computing node 420 may run multiple processes, and the multiple processes may constitute a communication domain to communicate with each other. For example, the multiple processes may correspond to Figure 1 Process 120-process 140 in.

[0091] As an example, the switch 410 is used to receive data sent by the multiple processes and process the data; the computing node 420 is used to determine that the first process among the multiple processes has exited abnormally, and send an exception notification message to the switch, where the exception notification message is used to indicate that the first process has exited abnormally; the switch is also used to obtain the exception notification message from the computing node 420, and release the resources required to process the data according to the exception notification message.

[0092] Optionally, the switch 410 is also used to obtain multiple exception notification messages from the computing node 420, and send the multiple exception notification messages to other processes in the multiple processes respectively, so that the other processes can stop communicating with the first process respectively according to the exception notification messages they receive.

[0093] Optionally, the exception notification message reuses a notification message sent by the first process when the first process exits normally.

[0094] Optionally, a daemon process is running in the first node, and the computing node 420 is specifically used for: the daemon process obtains the return value of the MPI function of the first process during its running; the daemon process determines that the first process exits abnormally based on the return value as an error code.

[0095] Optionally, the communication domain is a message passing interface MPI communication domain.

[0096] In addition, the system embodiment and the method embodiment provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment above, which will not be repeated here.

[0097] The method provided in the embodiment of the present application can be performed by a computing device, which can also be referred to as a computer system. It includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a processing unit, a memory and a memory control unit, and then the function and structure of the hardware are described in detail. The operating system is any one or more computer operating systems that implement business processing through a process, for example, a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software. In addition, optionally, the computer system is a handheld device such as a smart phone, or a terminal device such as a personal computer, and the present application is not particularly limited, as long as the method provided by the embodiment of the present application can be used. The execution subject of the method provided in the embodiment of the present application can be a computing device, or a functional module in a computing device that can call a program and execute a program.

[0098] Combine the following Figure 5 , a computing device provided in an embodiment of the present application is described in detail. It should be understood that Figure 5 The computing device shown may correspond to the switch 410 mentioned above, or may correspond to the computing node 420 mentioned above.

[0099] Figure 51 is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application. The computing device 1500 may be a server or a computer or other device with computing capabilities. Figure 5 The computing device 1500 shown includes at least one processor 1510 and a memory 1520 .

[0100] It should be understood that the present application does not limit the number of processors and memories in the computing device 1500 .

[0101] The processor 1510 executes the instructions in the memory 1520, so that the computing device 1500 implements the method provided by the present application. Alternatively, the processor 1510 executes the instructions in the memory 1520, so that the computing device 1500 implements the functional modules provided by the present application, thereby implementing the method provided by the present application.

[0102] Optionally, the computing device 1500 further includes a communication interface 1530. The communication interface 1530 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1500 and other devices or a communication network.

[0103] Optionally, the computing device 1500 further includes a system bus 1540, wherein the processor 1510, the memory 1520, and the communication interface 1530 are respectively connected to the system bus 1540. The processor 1510 can access the memory 1520 through the system bus 1540. For example, the processor 1510 can read and write data or execute code in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0104] In a possible implementation, the function of the processor 1510 is mainly to interpret the instructions (or codes) of the computer program and process the data in the computer software. The instructions of the computer program and the data in the computer software can be stored in the memory 1520 or the cache 1516.

[0105] Optionally, the processor 1510 may be an integrated circuit chip with signal processing capabilities. As an example and not a limitation, the processor 1510 is a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Among them, the general-purpose processor is a microprocessor, etc. For example, the processor 1510 is a central processing unit (CPU).

[0106] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514 .

[0107] Optionally, the processing unit 1512 is also called a core or kernel, which is the most important component of the processor. The processing unit 1512 is manufactured from single crystal silicon using a certain production process, and all calculations, command reception, command storage, and data processing of the processor are performed by the core. The processing units run program instructions independently and use the ability of parallel computing to speed up the running speed of the program. Various processing units have a fixed logical structure. For example, the processing unit includes logical units such as a first-level cache, a second-level cache, an execution unit, an instruction-level unit, and a bus interface.

[0108] In one implementation example, the memory control unit 1514 is used to control data interaction between the memory 1520 and the processing unit 1512. Specifically, the memory control unit 1514 receives a memory access request from the processing unit 1512, and controls access to the memory based on the memory access request. As an example and not a limitation, the memory control unit is a device such as a memory management unit (MMU).

[0109] In one implementation example, each memory control unit 1514 addresses the memory 1520 through the system bus. And an arbiter ( Figure 5 ), which is responsible for handling and coordinating competing accesses of multiple processing units 1512.

[0110] In an implementation example, the processing unit 1512 and the memory control unit 1514 are connected to each other through connection lines inside the chip, such as address lines, so as to achieve communication between the processing unit 1512 and the memory control unit 1514.

[0111] Optionally, each processor 1510 also includes a cache 1516, wherein the cache is a buffer for data exchange (called cache). When the processing unit 1512 wants to read data, it will first search for the required data from the cache. If it is found, it will be executed directly. If it is not found, it will be searched from the memory. Since the running speed of the cache is much faster than the memory, the role of the cache is to help the processing unit 1512 run faster.

[0112] The memory 1520 can provide a running space for the processes in the computing device 1500. For example, the computer program (specifically, the program code) used to generate the process is stored in the memory 1520. After the computer program is executed by the processor to generate the process, the processor allocates a corresponding storage space for the process in the memory 1520. Furthermore, the above storage space further includes a text segment, an initialized data segment, a bit initialized data segment, a stack segment, a heap segment, etc. The memory 1520 stores the data generated during the running of the process in the storage space corresponding to the above process, such as intermediate data, process data, etc.

[0113] Optionally, the storage is also called memory, and its function is to temporarily store the operation data in the processor 1510 and the data exchanged with the external storage such as the hard disk. As long as the computer is running, the processor 1510 will transfer the data to be calculated to the memory for calculation, and when the calculation is completed, the processing unit 1512 will transmit the result.

[0114] As an example and not limitation, memory 1520 is a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory is a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory is a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory 1520 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0115] The structure of the computing device 1500 listed above is only an example, and the present application is not limited thereto. The computing device 1500 of the embodiment of the present application includes various hardware in the computer system in the prior art. For example, the computing device 1500 also includes other memories besides the memory 1520, such as disk storage, etc. Those skilled in the art should understand that the computing device 1500 may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned computing device 1500 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 1500 may also include only the devices necessary to implement the embodiment of the present application, and does not necessarily include Figure 5 All devices shown in .

[0116] In this embodiment, a computer program product including instructions is also provided, and the computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device is caused to execute the method provided above, or the computing device is caused to implement the functions of the system provided above.

[0117] In this embodiment, a computer-readable storage medium is also provided. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions. When the instructions in the computer-readable storage medium are executed on a computing device, the computing device executes the method provided above.

[0118] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0124] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for recycling switch resources, characterized in that: The switch is connected to at least one computing node, the at least one computing node runs multiple processes, the multiple processes constitute a communication domain to communicate with each other, and the method includes: The switch receives data sent by the multiple processes and processes the data; A first node among the at least one computing node determines that a first process among the multiple processes exits abnormally, and sends an abnormality notification message to the switch, where the abnormality notification message is used to indicate that the first process exits abnormally; The switch obtains the abnormal notification message from the first node, and releases resources required for processing the data according to the abnormal notification message.

2. The method according to claim 1, characterized in that The switch obtains a plurality of the abnormal notification messages from the first node, and the method further includes: The switch sends the plurality of abnormal notification messages to other processes in the plurality of processes respectively, so that the other processes respectively stop communicating with the first process according to the abnormal notification messages received by the other processes.

3. The method according to claim 1 or 2, characterized in that: The abnormal notification message reuses the notification message sent by the first process when the first process exits normally.

4. The method according to any one of claims 1 to 3, characterized in that The first node of the at least one computing node determines that a first process of the multiple processes exits abnormally, including: The daemon process in the first node obtains the return value of the MPI function of the first process during its execution; The daemon process determines that the first process exits abnormally according to the return value being an error code.

5. The method according to any one of claims 1 to 4, characterized in that The communication domain is a message passing interface MPI communication domain.

6. A switch resource recovery system, characterized in that: The system comprises the switch and at least one computing node, the at least one computing node is connected, the at least one computing node comprises a first node, the at least one computing node runs a plurality of processes, the plurality of processes constitute a communication domain to communicate with each other, wherein, The switch is used to receive data sent by the multiple processes and process the data; The first node is used to determine that a first process among the multiple processes exits abnormally, and send an abnormal notification message to the switch, where the abnormal notification message is used to indicate that the first process exits abnormally; The switch is further configured to obtain the abnormal notification message from the first node, and release resources required for processing the data according to the abnormal notification message.

7. The system according to claim 6, characterized in that The switch is further configured to obtain the plurality of abnormal notification messages from the first node, and send the plurality of abnormal notification messages to other processes in the plurality of processes respectively, so that the other processes respectively stop communicating with the first process according to the abnormal notification messages received by each of them.

8. The system according to claim 6 or 7, characterized in that: The abnormal notification message reuses the notification message sent by the first process when the first process exits normally.

9. The system according to any one of claims 6 to 8, characterized in that A daemon process is running in the first node, and the first node is specifically used for: The daemon process obtains the return value of the MPI function of the first process during its operation; The daemon process determines that the first process exits abnormally according to the return value being an error code.

10. The system according to any one of claims 6 to 9, characterized in that The communication domain is a message passing interface MPI communication domain.

11. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 5.

Images