Method and system for recycling switch resources

Abstract

The application provides a method and system for recycling resources of a switch, the switch being connected to at least one computing node, the at least one computing node running a plurality of processes, the plurality of processes constituting a communication domain to communicate with each other, the method comprising: the switch receiving data sent by the plurality of processes and processing the data; a first node in the at least one computing node determining that a first process in the plurality of processes abnormally exits and sending an abnormal notification message to the switch, the abnormal notification message being used to indicate that the first process abnormally exits, the switch obtaining the abnormal notification message from the first node and releasing resources required for processing the data according to the abnormal notification message. The method can make the switch release the resources required for processing the data sent between the processes when the first process abnormally exits, and meanwhile, additional configuration of the switch can be avoided, and the versatility of the switch is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of storage, and more particularly, to a method and system for switch resource recycling. BACKGROUND

[0002] A process is a running activity of a program on a certain data set in a computer, is a basic unit of resource allocation of a system, and is a basis of an operating system structure. A plurality of processes running in at least one computing node can constitute a communication domain and communicate with each other.

[0003] A switch is a network device for forwarding electric (optical) signals, which can provide an exclusive electric signal path for any two network nodes connected to the switch. As an example, the switch can be connected to the at least one computing node and forward data transmitted between a plurality of processes running in the at least one computing node. The switch not only forwards data transmitted between the plurality of processes, but also undertakes processing tasks of the data transmitted between the plurality of processes, that is, the switch has an in-network computing (INC) function. When a certain process in the communication domain abnormally exits, the INC program in the switch also abnormally exits. However, when the INC program in the switch abnormally exits, the storage and other resources applied in the switch cannot be released. Over time, the resources in the switch that are not cleaned up in time will cause the resources of the switch to be exhausted, resulting in the loss of the in-network computing INC capability of the switch.

[0004] In a related technical solution, an additional process is run in the switch, which needs to send a packet to the process from the switch and needs to process the message fed back by the process. On the one hand, this requires strong customization configuration of the switch, and the universality is very poor. On the other hand, when the switch polls the packet to the process, it needs to occupy a certain network bandwidth, which has a performance impact on other network communications.

[0005] Therefore, how to recycle the resources required for processing data transmitted between processes when a certain process in the communication domain abnormally exits, while also improving the universality of the switch, has become a technical problem to be solved urgently. SUMMARY

[0006] The present application provides a method and system for switch resource recycling, which can recycle the resources required for processing data transmitted between processes when a certain process in the communication domain abnormally exits, while also improving the universality of the switch.

[0007] In a first aspect, a method for recycling resources of a switch is provided. The switch is connected to at least one computing node, and the at least one computing node runs a plurality of processes. The plurality of processes constitute a communication domain to communicate with each other. The method comprises: receiving, by the switch, data sent by the plurality of processes, and processing the data; determining, by a first node of the at least one computing node, that a first process of the plurality of processes abnormally exits, and sending, by the first node, an abnormal notification message to the switch. The abnormal notification message is used to indicate that the first process abnormally exits; and obtaining, by the switch, the abnormal notification message from the first node, and releasing resources required for processing the data according to the abnormal notification message.

[0008] In the above technical solution, after the first node monitors that the first process abnormally exits, the first node sends an abnormal notification message to the switch to indicate that the first process abnormally exits. After the switch receives the abnormal notification message sent by the first node, the switch can recycle resources required for processing data sent between processes. In this way, the switch can not only release resources required for processing data sent between processes after the first process abnormally exits, but also does not need to send a packet to the first process to monitor the state of the first process, thereby avoiding additional configuration of the switch, and making the switch more versatile.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further comprises: sending, by the switch, the abnormal notification message to other processes of the plurality of processes, so that the other processes stop communication with the first process according to the abnormal notification message.

[0010] In the above technical solution, in the case where the first process abnormally exits, the switch sends an abnormal notification message indicating that the first process abnormally exits to other processes of the plurality of processes. In this way, the other processes of the plurality of processes stop communication with the first process, thereby avoiding unnecessary communication overhead.

[0011] In combination with the first aspect, in some implementations of the first aspect, the switch obtains a plurality of abnormal notification messages from the first node of the at least one computing node. The method further comprises: sending, by the switch, the plurality of abnormal notification messages to other processes of the plurality of processes, respectively, so that the other processes stop communication with the first process according to the respective received abnormal notification messages.

[0012] In the technical solution, when the daemon process determines that the first process is abnormally exited, the daemon process can send a message indicating that the first process is abnormally exited to the rest of the plurality of processes in the communication domain. Since there are many processes in the communication domain, the switch only needs to correctly receive and analyze one message indicating that the first process is abnormally exited to normally release the resources on the switch, and thus can tolerate a certain packet loss rate. Moreover, as the node scale increases, a higher packet loss rate is allowed.

[0013] With reference to the first aspect, in some implementations of the first aspect, the method further includes: a daemon process in the first node determining that the first process is abnormally exited, the daemon process being configured to monitor whether the first process is abnormally exited; and the daemon process sending the abnormal notification message to the switch.

[0014] With reference to the first aspect, in some implementations of the first aspect, the switch receives the abnormal notification message sent by the daemon process in the first node.

[0015] With reference to the first aspect, in some implementations of the first aspect, the abnormal notification message multiplexes a notification message sent by the first process when the first process is normally exited.

[0016] In the technical solution, the packet of the abnormal exit can multiplex the packet of the normal exit, that is, the abnormal notification message multiplexes the notification message sent by the first process when the first process is normally exited. In this way, the packet of the abnormal exit can be analyzed without additional configuration or modification of the switch, thereby further improving the versatility of the switch.

[0017] With reference to the first aspect, in some implementations of the first aspect, the switch releases the resources required by the data in a garbage collection manner according to the abnormal notification message.

[0018] With reference to the first aspect, in some implementations of the first aspect, the communication domain is a message passing interface (MPI) communication domain.

[0019] With reference to the first aspect, in some implementations of the first aspect, the daemon process obtains a return value of an MPI function in a running process of the first process; and the daemon process determines that the first process is abnormally exited according to the return value as an error code.

[0020] In a second aspect, a system for recycling resources of a switch is provided. The system includes a switch and at least one computing node. The at least one computing node runs a plurality of processes that constitute a communication domain to communicate with each other. The switch is configured to receive data sent by the plurality of processes and process the data. A first node is configured to determine that a first process of the plurality of processes abnormally exits and send an abnormal notification message to the switch. The abnormal notification message is used to indicate that the first process abnormally exits. The switch is further configured to obtain the abnormal notification message from the first node and release resources required for processing the data according to the abnormal notification message.

[0021] With reference to the second aspect, in some implementations of the second aspect, the switch is further configured to obtain a plurality of the abnormal notification messages from the first node and send the plurality of the abnormal notification messages to other processes of the plurality of processes respectively, so that the other processes stop communication with the first process according to the respective received abnormal notification messages.

[0022] With reference to the second aspect, in some implementations of the second aspect, the abnormal notification message multiplexes a notification message sent by the first process when the first process normally exits.

[0023] With reference to the second aspect, in some implementations of the second aspect, the first node runs a daemon process. The first node is specifically configured to: obtain, by the daemon process, a return value of an MPI function of the first process during running of the first process; and determine, by the daemon process, that the first process abnormally exits according to the return value as an error code.

[0024] With reference to the second aspect, in some implementations of the second aspect, the communication domain is a message passing interface (MPI) communication domain.

[0025] In a third aspect, a switch is provided. The switch includes a processor and a memory. The processor is configured to execute instructions stored in the memory of the switch, so that the switch performs steps and methods performed by the switch in the first aspect or any possible implementation of the first aspect.

[0026] Optionally, the processor can be a general-purpose processor and can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, or the like. When implemented by software, the processor can be a general-purpose processor that reads software codes stored in a memory to implement the processor. The memory can be integrated in the processor or located outside the processor and exist independently.

[0027] In a fourth aspect, a computing node is provided, comprising a processor and a memory; the processor is configured to execute instructions stored in the memory of the computing node, so that the computing node performs the steps of the method performed by the computing node in the first aspect or any possible implementation of the first aspect.

[0028] Optionally, the processor can be a general-purpose processor, which can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in the memory. The memory can be integrated in the processor or exist independently outside the processor.

[0029] In a fifth aspect, a computer program product containing instructions is provided, which, when executed by a computing node and a switch, causes the computing node and the switch to perform the method in the first aspect and any possible implementation of the first aspect.

[0030] In a sixth aspect, a computer-readable storage medium is provided, which includes computer program instructions, and when the computer program instructions are executed by a computing node and a switch, the computing node and the switch perform the method in the first aspect and any possible implementation of the first aspect.

[0031] As an example, the computer-readable storage includes, but is not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and a hard drive.

[0032] Optionally, as an implementation, the storage medium can be a non-volatile storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a schematic block diagram of a scenario applicable to the embodiments of the present application.

[0034] Figure 2 is a schematic flow chart of a method for recycling resources of a switch provided by the embodiments of the present application.

[0035] Figure 3 is a schematic block diagram of a daemon process 1 sending a message indicating that the process 120 is abnormally exited to the process 130 and the process 140 provided by the embodiments of the present application.

[0036] Figure 4 is a schematic block diagram of a system 400 for switch resource recycling provided by an embodiment of the present application.

[0037] Figure 5 is an architecture schematic diagram of a computing device 1500 provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the present application will be described below with reference to the drawings.

[0039] The present application will present various aspects, embodiments or features around a system including a plurality of devices, components, modules, etc. It should be understood and appreciated that each system can include additional devices, components, modules, etc., and / or can not include all of the devices, components, modules, etc. discussed in connection with the drawings. Furthermore, combinations of these solutions can also be used.

[0040] In addition, in the embodiments of the present application, the words "example", "for example", etc. are used to mean serving as an example, instance or illustration. Any embodiment or design solution described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the word "example" is used to present the concept in a specific manner.

[0041] In the embodiments of the present application, "corresponding" and "relevant" can be used interchangeably at times. It should be noted that when the distinction is not emphasized, the meanings expressed are consistent.

[0042] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the network architecture evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0043] In the present specification, the reference to "one embodiment" or "some embodiments" and the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments" and the like in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise be specifically noted. The terms "comprising", "including", "having" and their variants mean "including but not limited to", unless otherwise be specifically noted.

[0044] In this application, "at least one" means one or more, "multiple" means two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent: including the case of A alone, A and B exist at the same time, and B alone, where A, B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple.

[0045] With the increase of network bandwidth, the decrease of delay and the emergence of technologies such as remote direct memory access (RDMA), researchers have begun to try to offload part of the computing operations in scientific computing tasks from the central processing unit (CPU) to a specific network communication device (such as a switch), so that the network device can be used efficiently while further reducing communication overhead and CPU load. Therefore, how to make full use of the computing power of the network device to realize communication optimization and reduce communication cost has become one of the research hotspots in the field of high-performance computing.

[0046] Message passing interface (MPI) is the de facto standard for programming in the field of high-performance computing (HPC), and many HPC applications are written using MPI. MPI can also be understood as a commonly used parallel programming model for parallel scientific application design, or a commonly used parallel computing application programming interface in high-performance computing software, often used in supercomputers, computer clusters, and other non-shared memory environment programming. It defines a set of communication protocols that developers can use for parallel computing on different hardware and operating systems. The main purpose of MPI is to perform message passing in a distributed system, allowing different processors to communicate and coordinate with each other.

[0047] In an MPI program, an individual that independently participates in communication is called a process. In an MPI program, an ordered set consisting of some or all processes is called a process group. Each process in a process group is assigned a sequence number in the group, called the process rank, which starts from 0. Inter-process communication in MPI is achieved through message passing. A message refers to a data exchange between processes. In MPI, the processes that send and receive messages need to perform synchronization operations to ensure correct message delivery.

[0048] The main features of MPI include:

[0049] 1. Parallelism: MPI allows multiple programs to be executed simultaneously on multiple processors and supports distributed memory systems.

[0050] 2. Portability: MPI is a standard that can be used on various hardware and operating systems, making it easy for developers to migrate programs from one platform to another.

[0051] 3. Flexibility: MPI provides rich message passing capabilities, making it easy to develop and debug on clusters of different sizes.

[0052] 4. Efficiency: MPI uses efficient communication algorithms and data structures to greatly improve the performance of distributed systems.

[0053] An important feature of MPI is in-network computing (INC). INC, as the name implies, refers to offloading the computing part of MPI during communication to the switch for execution. At this time, the switch not only forwards packets, but also performs simple computing work.

[0054] Figure 1 is a schematic diagram of a scenario applicable to embodiments of the present application. As Figure 1 shown, the scenario can include a switch 110, a process 120, a process 130, and a process 140.

[0055] It should be understood that the number of processes in the above scenario is not specifically limited by embodiments of the present application. For ease of description, Figure 1 three processes are used as an example in the description.

[0056] Referring to Figure 1Process 120, process 130 and process 140 communicate with each other based on the MPI protocol, and thus, process 120, process 130 and process 140 can also be referred to as MPI process 120, MPI process 130 and MPI process 140. Specifically, process 120, process 130 and process 140 cannot directly communicate with each other, and when process 120, process 130 and process 140 need to send data to each other, the data needs to be sent to switch 110 for forwarding.

[0057] Switch 110 is a switch with INC function, that is, switch 110 not only forwards data that needs to be sent between processes, but also undertakes computing work. For example, after receiving data that needs to be sent between process 120, process 130 and process 140, switch 110 caches the data that needs to be calculated on the network and performs corresponding calculation, and sends the final calculation result to process 120, process 130 and process 140. In this way, by participating in the calculation with switch 110, the amount of data to be forwarded and the amount of calculation of each process can be reduced, achieving the effect of saving bandwidth and reducing latency, and also helping to improve the computing power of the process.

[0058] It should be noted that process 120, process 130 and process 140 can run on one server, or can also run on different servers, and the embodiments of the present application do not make specific limitation thereon.

[0059] When process 120, process 130 and process 140 all normally exit (for example, the program ends or the system function exit() is called), the INC program in switch 110 also normally exits, and switch 110 releases the storage and other resources it applies for. Specifically, any one of process 120, process 130 and process 140 sends a normal exit message to switch 110, and switch 110 can release the storage and other resources it applies for according to the normal exit message.

[0060] When any one of process 120, process 130 and process 140 abnormally exits (for example, the user stops the program using "Ctrl+C", the user terminates the process using the kill command, or the process abnormally exits due to its own program bug), the INC program in switch 110 also abnormally exits. However, when the INC program in switch 110 abnormally exits, the storage and other resources applied for in switch 110 cannot be released. Over time, the resources in switch 110 that are not cleaned up in time will cause the resources of switch 110 to be exhausted, resulting in the loss of the in-network computing INC capability of switch 110, which must be solved by restarting switch 110.

[0061] Therefore, it is important to establish a resource recycling mechanism of the switch under the abnormal exit of the in-network computing INC.

[0062] In a related technical solution, an additional process is run in the switch, which needs to send a packet to the process on the server side and process the message fed back by the process on the server side. On the one hand, this requires strong customization of the switch, and the universality is very poor. On the other hand, when the switch sends a packet to the server, it needs to occupy a certain network bandwidth, which has a performance impact on other network communications.

[0063] Therefore, the embodiments of the present application provide a method for recycling resources of a switch, which can recycle resources of the switch for in-network computing INC in the case of abnormal exit of a process, and can also avoid additional configuration of the switch, thereby improving the universality of the switch.

[0064] The following will take the scenario shown in FIG. 1 as an example, and combine the description of the method for recycling resources of a switch provided by the embodiments of the present application. Figure 1 The following will take the scenario shown in FIG. 1 as an example, and combine the description of the method for recycling resources of a switch provided by the embodiments of the present application. Figure 2 The method for recycling resources of a switch provided by the embodiments of the present application will be described in detail. It should be understood that the examples described below are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific values or specific scenarios shown in the examples. Figure 2 The following will take the scenario shown in FIG. 1 as an example, and combine the description of the method for recycling resources of a switch provided by the embodiments of the present application. Figure 2 The following will take the scenario shown in FIG. 1 as an example, and combine the description of the method for recycling resources of a switch provided by the embodiments of the present application. Figure 2 The following will take the scenario shown in FIG. 1 as an example, and combine the description of the method for recycling resources of a switch provided by the embodiments of the present application.

[0065] Figure 2 FIG. 2 is a schematic flowchart of the method for recycling resources of a switch provided by the embodiments of the present application. As shown in FIG. 2, the method can include steps 210-230, which will be described in detail below. Figure 2

[0066] Step 210: The daemon process monitors whether the first process is an abnormal exit or a normal exit.

[0067] In the embodiments of the present application, an additional daemon process can be run in each server where the process (which can also be referred to as an MPI process) is located. Each daemon process is used to monitor whether the corresponding process (which can also be referred to as an MPI process) is an abnormal exit or a normal exit.

[0068] ​For example, a daemon process 1 can be additionally run in the server where the process 120 is located, the daemon process 1 being used to monitor whether the process 120 is abnormally exited or normally exited, the process 120 here corresponding to the first process above, the daemon process 1 here corresponding to the daemon process of the first process above. For another example, a daemon process 2 can be additionally run in the server where the process 130 is located, the daemon process 2 being used to monitor whether the process 130 is abnormally exited or normally exited, the process 130 here corresponding to the first process above, the daemon process 2 here corresponding to the daemon process of the first process above. For another example, a daemon process 3 can be additionally run in the server where the process 140 is located, the daemon process 3 being used to monitor whether the process 140 is abnormally exited or normally exited, the process 140 here corresponding to the first process above, the daemon process 3 here corresponding to the daemon process of the first process above.

[0069] For example, taking the daemon process as the daemon process 1 and the first process as the process 120 as an example. In one possible implementation, since the process 120 will return a return value of an MPI function during running, the daemon process 1 will obtain the return value of the MPI function, and determine whether the process 120 is abnormally exited or normally exited according to the value of the return value of the MPI function.

[0070] It should be understood that since the process 120 and other processes (for example, the process 130, the process 140) in the communication domain communicate based on the MPI protocol, in the MPI, the function names all adopt the form of MPI_Xxxx, for example, MPI_Send, MPI_Type_commit, MPI_COMM_WORLD, and the like. The return value of these functions is usually an integer value, indicating whether the operation is successful. If the return value is MPI_SUCCESS (0), it indicates that the operation is successful, and in this case, the process 120 belongs to the normally exited case; otherwise, if the return value is a non-0 value, it indicates an error code of the operation, and in this case, the process 120 belongs to the abnormally exited case.

[0071] It should be noted that different MPI functions can have different return values and error codes. Therefore, when a specific MPI function is used, the corresponding document needs to be consulted to understand the specific return value and error code meanings.

[0072] Step 220: The daemon process sends an abnormally exited message to the switch in the case of determining that the first process is abnormally exited.

[0073] In the embodiments of the present application, the daemon process can determine that the first process is in the abnormal exit condition, assuming that the daemon process obtains the non-0 MPI return value returned by the first process. The embodiments of the present application do not make specific limitation on the cause of the abnormal exit of the first process. For example, the first process can be stopped by the user using "Ctrl+C", or terminated by the user using the kill command, or abnormally exited due to its own program bug, or abnormally exited due to the failure of the server where the first process is located (failure caused by hardware failure, software problem, network problem, or power failure, etc.), and so on.

[0074] As an example, the daemon process can send the abnormal exit message to the switch upon determining that the first process is in the abnormal exit condition.

[0075] Step 230: The switch releases the resources applied for the in-network computing INC upon obtaining the abnormal exit message.

[0076] In the embodiments of the present application, the daemon process sends the abnormal exit message of the first process to other processes in the communication domain upon monitoring that the first process is in the abnormal exit condition, so as to notify the other processes in the communication domain of the abnormal exit of the first process. The other processes in the communication domain stop the communication with the first process upon receiving the abnormal exit message of the first process.

[0077] Specifically, the daemon process sends the abnormal exit message of the first process to other processes in the communication domain, and the daemon process sends the abnormal exit message indicating the abnormal exit of the first process to the switch, and the switch forwards the abnormal exit message to the other processes in the communication domain. The switch analyzes the abnormal exit message upon forwarding the abnormal exit message indicating the abnormal exit of the first process, and releases the resources applied for the INC upon determining that the first process is in the abnormal exit condition.

[0078] The abnormal exit message can also be referred to as an abnormal notification message, which is used to indicate the abnormal exit of the first process.

[0079] In a possible implementation manner, the switch can release the resources applied for the INC by means of garbage collection. Specifically, the switch automatically identifies and releases the memory space that is no longer used by means of the garbage collection mechanism.

[0080] It should be understood that the resources applied for the in-network computing INC in the switch can be the resources required by the switch to process the data sent by the processes in the communication domain (for example, the processes 120, 130, and 140 in FIG. 1), including but not limited to storage resources, computing resources, and the like. Figure 1 It should be understood that the resources applied for the in-network computing INC in the switch can be the resources required by the switch to process the data sent by the processes in the communication domain (for example, the processes 120, 130, and 140 in FIG. 1), including but not limited to storage resources, computing resources, and the like.

[0081] In the embodiment of the present application, the daemon sends a message indicating that the first process has exited abnormally to one of the other processes in the communication domain, so that the one of the other processes stops communication with the first process, or the daemon sends a message indicating that the first process has exited abnormally to a plurality of the other processes in the communication domain, so that the plurality of the other processes stop communication with the first process, and the embodiment of the present application does not make a specific limitation in this regard.

[0082] In the above technical solution, when the daemon determines that the first process has exited abnormally, the daemon sends a message indicating that the first process has exited abnormally to each of the remaining processes in the communication domain. Since there are many processes in the communication domain, the switch only needs to correctly receive and analyze one message indicating that the first process has exited abnormally to normally release the resources on the switch, and thus can tolerate a certain packet loss rate. Moreover, as the size of the node increases, a higher packet loss rate is allowed.

[0083] For ease of description, the daemon is taken as daemon 1 and the first process is taken as process 120 in the following description.

[0084] For example, as shown in FIG. 1, when the daemon 1 determines that the process 120 has exited abnormally, the daemon 1 can send a message indicating that the process 120 has exited abnormally to the process 130 through the switch 110. Figure 3 After the switch 110 receives the message from the daemon 1, the switch 110 sends the message to the process 130. After the process 130 receives the message indicating that the process 120 has exited abnormally, the process 130 stops communication with the process 120.

[0085] For another example, as shown in FIG. 2, when the daemon 1 determines that the process 120 has exited abnormally, the daemon 1 can send a message indicating that the process 120 has exited abnormally to the process 130 and the process 140 through the switch 110. Figure 3 After the switch 110 receives the two messages from the daemon 1, the switch 110 sends the two messages to the process 130 and the process 140, respectively. After the process 130 receives the message indicating that the process 120 has exited abnormally, the process 130 stops communication with the process 120. After the process 140 receives the message indicating that the process 120 has exited abnormally, the process 140 also stops communication with the process 120.

[0086] In one embodiment of this application, the abnormal exit message can reuse the normal exit message; that is, the abnormal notification message reuses the notification message sent by the first process when the first process exits normally. In this way, the abnormal exit message can be parsed without additional configuration or modification to the switch, thereby further improving the versatility of the switch.

[0087] In the above technical solution, an additional daemon process runs on the server (also known as the computing node). This daemon process monitors whether processes running on the server exit abnormally. If a process running on the server exits abnormally, the switch reclaims the resources allocated by the switch for the on-network computing INC. This eliminates the need to start a separate process on the switch to monitor whether processes running on the server exit abnormally, thus avoiding additional configuration of the switch and increasing its versatility.

[0088] The above text combined Figures 1 to 3 The method provided in the embodiments of this application is described in detail below. Figures 4-5 The embodiments of the apparatus of this application are described in detail below. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to the foregoing method embodiments.

[0089] Figure 4 This is a schematic block diagram of a switch resource reclamation system 400 provided in an embodiment of this application. The system 400 can be implemented through software, hardware, or a combination of both. The system 400 provided in this embodiment can implement the embodiments of this application. Figure 2 The methods and steps in the text.

[0090] The system 400 includes a switch 410 and at least one computing node 420. Specifically, the switch 410 may correspond to... Figure 1 The switch 110 in the middle. The at least one computing node 420 can run multiple processes, which constitute a communication domain to communicate with each other, for example, multiple processes can correspond to Figure 1 Processes 120-140 in the process.

[0091] As an example, the switch 410 is used to receive data sent by the plurality of processes and process the data; the computing node 420 is used to determine that the first process among the plurality of processes has exited abnormally and send an abnormal notification message to the switch, the abnormal notification message being used to indicate that the first process has exited abnormally; the switch is also used to obtain the abnormal notification message from the computing node 420 and release the resources required to process the data according to the abnormal notification message.

[0092] Optionally, the switch 410 is further configured to acquire a plurality of the abnormal notification messages from the computing node 420, and send the plurality of the abnormal notification messages to other processes in the plurality of processes respectively, so that the other processes stop communication with the first process according to the respective received abnormal notification messages.

[0093] Optionally, the abnormal notification message is multiplexed with a notification message sent by the first process when the first process normally exits.

[0094] Optionally, the first node runs a daemon process, and the computing node 420 is specifically configured to: acquire, by the daemon process, a return value of an MPI function in a running process of the first process; and determine, by the daemon process, the abnormal exit of the first process according to the return value as an error code.

[0095] Optionally, the communication domain is a message passing interface (MPI) communication domain.

[0096] In addition, the system embodiment and the method embodiment provided by the above-described embodiments belong to the same concept, and the specific implementation process is described in the method embodiments above, which will not be described here.

[0097] The method provided by the embodiments of the present application can be executed by a computing device, which can also be referred to as a computer system. The computer system includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a processing unit, a memory, and a memory control unit. The functions and structures of the hardware are described in detail later. The operating system is any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer includes application programs such as a browser, an address book, word processing software, and instant messaging software. Optionally, the computer system is a handheld device such as a smartphone or a terminal device such as a personal computer, and the present application is not particularly limited as long as it can execute the method provided by the embodiments of the present application. The execution subject of the method provided by the embodiments of the present application can be a computing device, or a functional module in the computing device that can call and execute a program.

[0098] The following describes the embodiments of the present application in conjunction with Figure 5 A computing device provided by the embodiments of the present application is described in detail. It should be understood that Figure 5 The computing device shown can correspond to the switch 410 or the computing node 420 described above.

[0099] Figure 5Figure 1 is a schematic diagram of an architecture of a computing device 1500 according to an embodiment of the present application. The computing device 1500 can be a server or a computer or other device having computing capability. Figure 5 The computing device 1500 shown includes at least one processor 1510 and memory 1520.

[0100] It should be understood that the number of processors and memories in the computing device 1500 is not limited.

[0101] The processor 1510 executes instructions in the memory 1520 so that the computing device 1500 implements the method provided by the present application. Alternatively, the processor 1510 executes instructions in the memory 1520 so that the computing device 1500 implements the various functional modules provided by the present application, thereby implementing the method provided by the present application.

[0102] Optionally, the computing device 1500 further includes a communication interface 1530. The communication interface 1530 uses a transceiving module such as, but not limited to, a network interface card, a transceiver, to implement communication between the computing device 1500 and other devices or communication networks.

[0103] Optionally, the computing device 1500 further includes a system bus 1540, wherein the processor 1510, the memory 1520 and the communication interface 1530 are respectively connected with the system bus 1540. The processor 1510 can access the memory 1520 through the system bus 1540, for example, the processor 1510 can read and write data in the memory 1520 or execute code in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0104] In one possible implementation, the function of the processor 1510 is mainly to interpret the instructions (or code) of the computer program and process the data in the computer software. The instructions of the computer program and the data in the computer software can be saved in the memory 1520 or the cache 1516.

[0105] Optionally, the processor 1510 is a chip that has a processing capability of signals. As an example but not limitation, the processor 1510 is a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Among them, the general processor is a microprocessor and the like. For example, the processor 1510 is a central processing unit (CPU).

[0106] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514.

[0107] Optionally, the processing unit 1512 is also called a core or a kernel, which is the most important component of the processor. The processing unit 1512 is manufactured by single crystal silicon with a certain production process, and all the calculations, command acceptance, command storage and data processing of the processor are executed by the core. The processing units respectively independently run program instructions, and use the parallel computing capability to speed up the program running. Various processing units have fixed logic structures, for example, the processing unit includes logic units such as a first level cache, a second level cache, an execution unit, an instruction level unit and a bus interface.

[0108] In one implementation, the memory control unit 1514 is used to control the data interaction between the memory 1520 and the processing unit 1512. Specifically, the memory control unit 1514 receives a memory access request from the processing unit 1512, and controls the access to the memory based on the memory access request. As an example but not limitation, the memory control unit is a memory management unit (MMU) and the like.

[0109] In one implementation, each memory control unit 1514 addresses the memory 1520 through a system bus. And an arbiter (not shown in the figure) is configured in the system bus, which is responsible for processing and coordinating the competitive access of multiple processing units 1512. Figure 5

[0110] In one implementation, the processing unit 1512 and the memory control unit 1514 are communicatively connected through the internal connection line of the chip, such as the address line, so as to realize the communication between the processing unit 1512 and the memory control unit 1514.

[0111] ​Optionally, each processor 1510 also includes a cache 1516, which provides temporary storage of often-used data and instructions. Recent use of data can be stored in the cache 1516 for quick future use. The cache 1516 also decreases the number of times information is loaded from the main memory 1520 into the processor 1510, thus improving the processor's performance.

[0112] The memory 1520 can provide a space for processes in the computing device 1500 to run, for example, the memory 1520 stores computer programs (in particular, the codes of the programs) for generating the processes. After the computer programs are run by the processor to generate the processes, the processor allocates corresponding storage spaces in the memory 1520 for the processes. Further, the storage spaces further include a text segment, an initialized data segment, a bit initialized data segment, a stack segment, a heap segment, and the like. The memory 1520 stores data generated during the running of the processes, for example, intermediate data, or process data, and the like, in the storage spaces corresponding to the processes.

[0113] Optionally, the memory is also referred to as the internal memory, which is used to temporarily store the data for operation in the processor 1510, and exchange the data with the external memory such as the hard disk. As long as the computer is running, the processor 1510 will call the data needed for operation to the internal memory for operation, and the processing unit 1512 will transmit the results after the operation is completed.

[0114] By way of example, and not limitation, memory 1520 is volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory is random access memory (RAM), which acts as external cache. By way of example, and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). The memory 1520 of the system and method described herein are intended to include, without being limited to, these and any other suitable types of memory.

[0115] The structure of the computing device 1500 listed above is only exemplary, and the present application is not limited thereto. The computing device 1500 of the embodiments of the present application includes various hardware in the prior art computer system, for example, the computing device 1500 also includes other memories in addition to the memory 1520, such as disk memories and the like. Those skilled in the art should understand that the computing device 1500 can also include other devices necessary for normal operation. Meanwhile, according to specific needs, those skilled in the art should understand that the above-mentioned computing device 1500 can also include hardware devices for realizing other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 1500 can also only include devices necessary for realizing the embodiments of the present application, and it is not necessary to include all the devices shown in the prior art. Figure 5 The computing device 1500 of the embodiments of the present application can also include other devices necessary for normal operation. Meanwhile, according to specific needs, those skilled in the art should understand that the above-mentioned computing device 1500 can also include hardware devices for realizing other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 1500 can also only include devices necessary for realizing the embodiments of the present application, and it is not necessary to include all the devices shown in the prior art.

[0116] In the embodiment, a computer program product including instructions is also provided. The computer program product can be a software or program product including instructions, which can be run on a computing device or stored in any available medium. When it is run on the computing device, it causes the computing device to perform the method provided above, or causes the computing device to realize the functions of the system provided above.

[0117] In the embodiment, a computer readable storage medium is also provided. The computer readable storage medium can be any available medium that the computing device can store or a data storage device such as a data center including one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium includes instructions, which, when executed on the computing device, cause the computing device to perform the method provided above.

[0118] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0123] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0124] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for switch resource recycling, the method comprising: The switch is connected with at least one computing node, the at least one computing node runs a plurality of processes, the plurality of processes constitute a communication domain to communicate with each other, and the method comprises: The switch receives data sent by the plurality of processes and processes the data; A first node in the at least one computing node determines that a first process in the plurality of processes abnormally exits, and sends an abnormal notification message to the switch, the abnormal notification message being used to indicate that the first process abnormally exits; The switch acquires the abnormal notification message from the first node, and releases resources required for processing the data according to the abnormal notification message.

2. The method of claim 1, wherein, The switch acquires a plurality of abnormal notification messages from the first node, and the method further comprises: The switch sends the plurality of abnormal notification messages to other processes in the plurality of processes respectively, so that the other processes stop communication with the first process according to the respective received abnormal notification messages.

3. The method of claim 1, wherein, The abnormal notification message multiplexes a notification message sent by the first process when the first process normally exits.

4. The method according to any one of claims 1 to 3, characterized in that, The first node in the at least one computing node determines that a first process in the plurality of processes abnormally exits, and comprises: A daemon process in the first node acquires a return value of an MPI function in a running process of the first process; The daemon process determines that the first process abnormally exits according to the return value as an error code.

5. The method according to any one of claims 1 to 3, characterized in that, The communication domain is a message passing interface (MPI) communication domain.

6. A system for switch resource recycling, the system comprising: The system comprises the switch and at least one computing node, the switch is connected with the at least one computing node, the at least one computing node comprises a first node, the at least one computing node runs a plurality of processes, the plurality of processes constitute a communication domain to communicate with each other, and wherein, The switch is configured to receive data sent by the plurality of processes and process the data; The first node is configured to determine that a first process in the plurality of processes abnormally exits, and send an abnormal notification message to the switch, the abnormal notification message being used to indicate that the first process abnormally exits; The switch is further configured to acquire the abnormal notification message from the first node, and release resources required for processing the data according to the abnormal notification message.

7. The system of claim 6, wherein: The switch is further configured to acquire a plurality of abnormal notification messages from the first node, and send the plurality of abnormal notification messages to other processes in the plurality of processes respectively, so that the other processes stop communication with the first process according to the respective received abnormal notification messages.

8. The system of claim 6, wherein, The abnormal notification message multiplexes a notification message sent by the first process when the first process normally exits.

9. The system of any one of claims 6 to 8, wherein, The first node runs a daemon process, and the first node is specifically configured to: The daemon process acquires a return value of an MPI function in a running process of the first process; The daemon process determines that the first process abnormally exits according to the return value as an error code.

10. The system of any one of claims 6 to 8, wherein, The communication domain is a message passing interface (MPI) communication domain.

11. A computer program product comprising instructions, characterized in that, When the instructions are executed by the cluster of computing devices, the cluster of computing devices is caused to perform the method of any of claims 1-5.

12. A computer-readable storage medium, characterized in that, A computer program product comprising computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1-5.

Images