Communication method and related equipment

Abstract

The invention discloses a communication method and related equipment, and is applied to the field of communication. The method comprises the following steps: acquiring a first serial number of a first network card in a server and a second serial number of the first network card in a communication plane; and calculating a phase deflection value of the communication plane according to the first serial number. A first communication partner serial number is calculated according to the second serial number, the first communication partner serial number is the serial number of the communication object of the first network card in the Kth algorithm step on the communication plane, and the first communication partner serial number corresponds to the second network card. And combining the first communication partner serial number with the phase deflection value to obtain a second communication partner serial number which is the serial number of the third network card on the communication plane. And replacing the communication object of the first network card in the Kth algorithm step with the third network card from the second network card. According to the method, the flow of constructing the phase deflection communication flow is added, so that the flow among the nodes of different communication planes generates phase difference, the flow of the whole network is balanced, and the communication performance is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a communication method and related equipment. Background Art

[0002] In AI scenarios, the volume of computing is growing, and the demand for computing cluster networking is also increasing. The mainstream computing networking form on the market is the fat-tree topology, but when Fat-tree is applied to ultra-large-scale networking solutions, the number of Fat-tree layers will increase. In other words, a large number of switches for core layer communications need to be added, resulting in excessively high networking costs. In ultra-large-scale networking scenarios, the scalability and networking costs of mesh networking are far superior to those of fat-tree. Therefore, mesh networking is an effective alternative.

[0003] Mesh networking is a type of networking with a mesh-like topology, including full mesh networking and Dragonfly networking. In actual applications, it is found that in the case of parallel communication of multiple network cards, symmetric collective communication in Mesh networking shows longer communication time than in Fat-tree topology. For example, in a Dragonfly network with 8 servers and 64 network cards (8 network cards per server), actual tests and comparisons with Fat-tree topology of the same scale show that the communication time of the All-Reduce operator in symmetric collective communication has increased by more than 50%, and the communication time of the All-to-all operator has increased by more than 1 times.

[0004] It can be seen from this that the performance degradation of symmetric collective communications in Mesh networking scenarios is quite serious, and improvements and innovative solutions are urgently needed. Summary of the invention

[0005] The present application provides a communication method and related equipment, which can reduce the probability of link congestion between nodes and improve communication performance.

[0006] The first aspect of the present application provides a communication method that can be applied to a communication device. The method includes: obtaining a first serial number, the first serial number is the serial number of the first network card in the target server, the target server includes N network cards, each of the N network cards corresponds to a communication plane, wherein the first network card corresponds to the target communication plane, the target communication plane includes M network cards, the M network cards belong to different servers, the serial numbers of the M network cards in their respective servers are the same as the first serial number, the M network cards communicate through a target communication algorithm, and the target communication algorithm includes multiple algorithm steps; obtaining a second serial number, the second serial number is the serial number of the first network card in the target communication plane; based on the first The phase deviation value of the target communication plane is calculated based on the second serial number, the number of network cards included in the target server, and the number of network cards included in the target communication plane; the first communication partner serial number is calculated based on the second serial number, the first communication partner serial number is the serial number of the communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner serial number corresponds to the second network card; the first communication partner serial number and the phase deviation value are combined to obtain the second communication partner serial number, the second communication partner serial number is the serial number of the third network card in the target communication plane; the communication object of the first network card in the Kth algorithm step is replaced by the third network card from the second network card.

[0007] The N network cards of the target server correspond to a communication plane respectively, so the number of network cards included in the target server is the number of communication planes, and multiple communication planes can communicate in parallel.

[0008] The first network card corresponds to the target communication plane. The serial numbers of the network cards in the target communication plane in their respective servers are the same as the first serial number, and the first serial number is the serial number of the target communication plane. The network cards in the same communication plane can communicate with each other. There are multiple target communication algorithms used for communication, and different communication algorithms include different numbers of algorithm steps.

[0009] In the first aspect of the present application, the phase deviation value of the communication plane is first calculated based on the number of network cards in the communication plane, the total number of communication planes and the serial number of the communication plane, and then the communication object of the network card is recalculated based on the phase deviation value of the communication plane, so that the traffic in different communication planes between two nodes has a phase difference, thereby balancing the traffic of the entire network, reducing the bandwidth load of a single link, improving the bandwidth utilization of the entire network, reducing the probability of congestion, and improving communication performance.

[0010] In a possible implementation manner of the first aspect, the target communication algorithm is a Having-Doubling algorithm, and the plurality of algorithm steps is log2M algorithm steps.

[0011] In this possible implementation, the target communication algorithm is limited to the Having-Doubling algorithm, which improves the practicability of the solution.

[0012] In a possible implementation of the first aspect, before the step of obtaining the first serial number, the method further includes: performing collective communication on the N network cards using a Reduce-Scatter operator. After the step of replacing the communication object of the first network card in the Kth algorithm step from the second network card to the third network card, the method further includes: performing collective communication on the N network cards using an All-Gather operator.

[0013] When the target communication algorithm is the Having-Doubling algorithm, before inter-node communication, the Reduce-Scatter operator needs to be used to perform collective communication on the network cards in the target server. At the end of the communication process, the All-Gather operator needs to be used to perform collective communication on the network cards in the target server.

[0014] In a possible implementation of the first aspect, the above step: calculating the first communication partner sequence number based on the second sequence number includes: calculating based on the following formula: first communication partner sequence number = second sequence number ^(1<<((K+first sequence number)%log2M)), where ^ represents a bitwise exclusive OR operation, << represents a left shift operation, and % represents a remainder operation.

[0015] When the target communication algorithm is the Having-Doubling algorithm, the total number of algorithm steps is log2M steps, where M is the number of network cards included in the target communication plane. The first communication partner sequence number is then calculated based on the first sequence number of the first network card in the target server, the second sequence number in the target communication plane, the number of algorithm steps K, and the total number of algorithm steps.

[0016] In this possible implementation, the method for calculating the first communication partner sequence number is limited, thereby improving the practicability of the solution.

[0017] In a possible implementation manner of the first aspect, the target communication algorithm is a Pairwise algorithm, and the multiple algorithm steps are M-1 algorithm steps.

[0018] In this possible implementation, the target communication algorithm is limited to the Pairwise algorithm, which expands the application scenarios of the solution and improves the practicability of the solution.

[0019] In a possible implementation manner of the first aspect, before the above step of obtaining the first sequence number, the method further includes: using an All-to-all operator to perform collective communication on the N network cards.

[0020] When the target communication algorithm is the Pairwise algorithm, before inter-node communication, you need to use the All-to-all operator to perform collective communication on the network cards in the target server.

[0021] In a possible implementation of the first aspect, the first communication partner sequence number includes a third communication partner sequence number and a fourth communication partner sequence number, and the above step: calculating the first communication partner sequence number based on the second sequence number includes: calculating the third communication partner sequence number and the fourth communication partner sequence number based on the second sequence number; the third communication partner sequence number and the fourth communication partner sequence number are calculated based on the following formula: third communication partner sequence number = (second sequence number + MK) % M, fourth communication partner sequence number = (second sequence number + K) % M, wherein % represents remainder.

[0022] When the target communication algorithm is the Pairwise algorithm, the total number of algorithm steps is M-1. Under the Pairwise algorithm, the first network card has two communication objects in each algorithm step, so there are two communication partner numbers, namely the third communication partner number and the fourth communication partner number. The third communication partner number and the fourth communication partner number are calculated based on the second number of the first network card in the target communication plane, the number of network cards included in the target communication plane, and the number of algorithm steps K.

[0023] In this possible implementation, the method for calculating the first communication partner sequence number is limited, thereby improving the practicability of the solution.

[0024] The second aspect of the present application provides a communication device, including an acquisition unit, a calculation unit, a combination unit, and a replacement unit. The acquisition unit is used to acquire a first serial number, the first serial number is the serial number of the first network card in the target server, the target server includes N network cards, each of the N network cards corresponds to a communication plane, wherein the first network card corresponds to the target communication plane, the target communication plane includes M network cards, the M network cards belong to different servers, the serial numbers of the M network cards in their respective servers are the same as the first serial number, the M network cards communicate with each other through a target communication algorithm, and the target communication algorithm includes multiple algorithm steps; the acquisition unit is also used to acquire a second serial number, the second serial number is the serial number of the first network card in the target communication plane; the calculation unit is used to calculate the serial number based on the first serial number, The phase deviation value of the target communication plane is calculated based on the number of network cards included in the target server and the number of network cards included in the target communication plane; the calculation unit is also used to calculate the first communication partner sequence number based on the second sequence number, the first communication partner sequence number is the sequence number of the communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner sequence number corresponds to the second network card; the combination unit is used to combine the first communication partner sequence number and the phase deviation value to obtain the second communication partner sequence number, and the second communication partner sequence number is the sequence number of the third network card in the target communication plane; the replacement unit is used to replace the communication object of the first network card in the Kth algorithm step from the second network card to the third network card.

[0025] In a possible implementation manner of the second aspect, the target communication algorithm is a Having-Doubling algorithm, and the multiple algorithm steps are log2M algorithm steps.

[0026] In a possible implementation manner of the second aspect, the communication device further includes a communication unit, which is used to use a Reduce-Scatter operator to perform collective communication on the N network cards; the communication unit is also used to use an All-Gather operator to perform collective communication on the N network cards.

[0027] In a possible implementation of the second aspect, the calculation unit is specifically used to calculate the first communication partner sequence number based on the following formula: first communication partner sequence number = second sequence number ^(1<<((K+first sequence number)%log2M)), where ^ represents a bitwise exclusive OR operation, << represents a left shift operation, and % represents a remainder operation.

[0028] In a possible implementation manner of the second aspect, the target communication algorithm is a Pairwise algorithm, and the multiple algorithm steps are M-1 algorithm steps.

[0029] In a possible implementation manner of the second aspect, the communication unit is further configured to use an All-to-all operator to perform collective communication on the N network cards.

[0030] In a possible implementation of the second aspect, the first communication partner sequence number includes a third communication partner sequence number and a fourth communication partner sequence number, and the calculation unit is specifically used to calculate the third communication partner sequence number and the fourth communication partner sequence number based on the second sequence number; the third communication partner sequence number and the fourth communication partner sequence number are calculated based on the following formula: third communication partner sequence number = (second sequence number + MK) % M, fourth communication partner sequence number = (second sequence number + K) % M, wherein % represents remainder.

[0031] The communication device provided in the second aspect of the present application is used to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0032] The third aspect of the present application provides a communication device, including a processor and a memory, wherein the memory is used to store instructions, and the processor is used to obtain the instructions stored in the memory to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] A fourth aspect of the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0034] A fifth aspect of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any possible implementation of the first aspect.

[0035] In a sixth aspect, the present application provides a chip system, which includes at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via lines, and the at least one processor is used to run computer programs or instructions to execute the method described in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1a A schematic diagram of the architecture of a communication system used in an embodiment of the present application;

[0037] Figure 1b Another schematic diagram of the architecture of the communication system used in the embodiment of the present application;

[0038] Figure 2 A schematic diagram of an embodiment of a communication method provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of another embodiment of the communication method provided in the embodiment of the present application;

[0040] Figure 4a A schematic diagram of communication traffic characteristics in an embodiment of the present application;

[0041] Figure 4b Another schematic diagram of the communication traffic characteristics in the embodiment of the present application;

[0042] Figure 5 A schematic diagram of another embodiment of the communication method provided in the embodiment of the present application;

[0043] Figure 6a Another schematic diagram of the communication traffic characteristics in the embodiment of the present application;

[0044] Figure 6b Another schematic diagram of the communication traffic characteristics in the embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0046] Figure 8 Another structural diagram of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0048] The terms "system" and "network" in the specification and claims of the present application and the above-mentioned drawings can be used interchangeably. Unless otherwise specified, ordinal numbers such as "first" and "second" are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of multiple objects. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] To facilitate understanding, the relevant terms and concepts mainly involved in the embodiments of the present application are first introduced below.

[0050] 1. Mesh networking

[0051] Mesh networking is a general term for networking types with a mesh-like topology. Mesh networking includes Full Mesh networking and Dragonfly networking.

[0052] 2. Collective communication

[0053] Collective Communications refers to global communication operations in which all processes in a process group participate.

[0054] 3. Symmetric Collection Communication

[0055] Symmetric collective communication is a type of collective communication with the following characteristics: the communication purpose and behavior are equivalent for each process, and there is no concept of master and slave processes. Typical symmetric collective communications include All-Reduce, All-to-all, All-Gather, and Reduce-Scatter.

[0056] 4. In-phase superposition

[0057] When multiple identical communication flows are superimposed, the features with larger flows in the time domain characteristics will produce larger flows due to the superposition of amplitudes at the same time period, which is called the same-phase superposition phenomenon of multiple identical communication flows.

[0058] 5. Phase deflection

[0059] By adding a deflection factor to the communication algorithm, the time domain features of multiple communication flows generated by the communication algorithm change in phase. That is, the order and interval of the communication flow features are consistent, but the starting points are different. If the communication flow time domain features are plotted with time as the horizontal axis and communication flow as the vertical axis, each feature will have a phase difference similar to a waveform graph. This phenomenon is called phase deflection of the time domain features of multiple communication flows.

[0060] The embodiment of the present application provides a communication method, which can reduce the probability of congestion of communication links between nodes and reduce communication delay. The embodiment of the present application also provides a corresponding device, a computer-readable storage medium, and a computer program product, etc. The following are respectively described.

[0061] See below Figure 1a and Figure 1b , is a schematic diagram of the architecture of the communication system used in the embodiment of the present application.

[0062] The present invention can be applied in the scenario of Mesh networking topology, when multiple network communication planes are in parallel to perform symmetric collective communication, and perform hierarchical communication inside and outside nodes, as described below:

[0063] Figure 1a and Figure 1b All of them are Mesh networking scenarios. Figure 1a It is a communication system under the Full Mesh networking architecture, such as Figure 1a As shown, the communication system includes four servers, namely A0, A1, A2 and A3, each server includes four network cards, and the network cards in the server are numbered from 0 to 3. Figure 1b It is the communication system under the Dragonfly network architecture, such as Figure 1b As shown, the communication system includes 8 servers, namely B0 to B7. Among them, B0 and B1 are connected to switch S0, B2 and B3 are connected to switch S1, B4 and B5 are connected to switch S2, and B6 and B7 are connected to switch S3. Each server includes 4 network cards, and the network cards are numbered 0 to 3 in the server.

[0064] A server is a node. Multiple network cards in a server can communicate with each other through the intra-node interconnection. For example, for server A0, the four network cards in AO can communicate with each other. Network cards with the same number between servers can communicate with each other, and the network cards with the same number form a communication plane. Figure 1a In the communication system shown, the network card No. 1 in A0, the network card No. 1 in A1, the network card No. 1 in A2, and the network card No. 1 in A3 form a communication plane, and the network cards in the same communication plane can communicate with each other. For Full Mesh networking, network cards with the same number communicate directly between nodes, while network cards in Dragonfly networking first communicate with the connected switch, and then communicate with network cards with the same number through the switch. Multiple network cards included in the server can form multiple communication planes, for example Figure 1a In the communication system shown, each server includes four network cards, so four communication planes can be formed. Multiple network cards of a single server can communicate in parallel, that is, multiple communication planes can communicate in parallel.

[0065] Currently, when performing symmetric collective communications, although multiple communication planes can communicate in parallel, the traffic of multiple network cards included in the same server has almost the same characteristics in the time domain, which makes the traffic of different communication planes between nodes form a phase superposition, resulting in peak traffic concentrated on some links, which is prone to network congestion, resulting in high communication time consumption and poor communication performance.

[0066] In view of this, an embodiment of the present application provides a communication method that can be applied to communication equipment. By adding a process for constructing a phase-deflected communication flow, a phase difference is generated in the flow between nodes in different parallel communication planes, thereby balancing the flow of the entire network, reducing the bandwidth load of local links, reducing network congestion, and thereby reducing communication time, improving the utilization rate of the bandwidth of the entire network, and improving communication performance.

[0067] The communication method provided in the embodiment of the present application is mainly applied to symmetric collective communication scenarios in Mesh networking.

[0068] See below Figure 2 , is a schematic diagram of an embodiment of the communication method provided in the embodiment of the present application. Figure 2 As shown, this embodiment includes steps 201 to 206.

[0069] 201. Obtain a first serial number of a first network card.

[0070] The first network card is located in the target server. The communication device obtains the first serial number of the first network card by obtaining an environment variable or reading the network card configuration. The first serial number is the serial number of the first network card in the target server. The target server includes N network cards, and the serial numbers of the N network cards in the target server are numbered sequentially starting from 0. For example, if the target server includes 4 network cards, the serial numbers of the 4 network cards in the target server are 0, 1, 2, and 3 respectively.

[0071] The target server can communicate with the other servers in the communication system. Specifically, the network cards with the same serial number in each server form a communication plane, and the network cards in the same communication plane can communicate with each other. One serial number in the server corresponds to one network card, so the network cards in a communication plane all come from different servers. For example, the communication system includes 4 servers, and each server includes 8 network cards. Then the network card with serial number 0 in each server forms a communication plane, and the network card with serial number 1 forms another communication plane. And so on. Multiple communication planes can communicate in parallel, and each communication plane includes 4 network cards. The network cards included in a communication plane have the same serial number in each server, so the serial number of the network card in the server is also the serial number of the communication plane in many parallel communication planes.

[0072] That is to say, the N network cards included in the target server correspond to N communication planes, and each network card corresponds to a communication plane. Among them, the communication plane corresponding to the first network card is the target communication plane, and the target communication plane includes M network cards, and the M network cards belong to different servers, that is, the communication system includes at least M servers, and the sequence numbers of the M network cards in their respective servers are the same as the first sequence number, and the sequence number of the target communication plane is also the first sequence number. For example, if the sequence number of the first network card in the target server is 2, then the sequence numbers of the remaining network cards in the target communication plane in their respective servers are also 2, and the sequence number of the target communication plane is also 2.

[0073] The M network cards in the target communication plane communicate through a target communication algorithm. The target communication algorithm may be a Having-Doubling algorithm or a Pairwise algorithm, etc. The target communication algorithm includes multiple algorithm steps.

[0074] 202. Obtain a second serial number of the first network card.

[0075] The network cards are numbered in sequence in the communication plane to which they belong, starting from 0. Exemplarily, if the communication plane includes 5 network cards, the sequence numbers of the 5 network cards in the target communication plane are 0, 1, 2, 3 and 4 respectively.

[0076] The target communication plane includes M network cards, and the serial numbers of the M network cards in the target communication plane are 0 to M. The communication device obtains the second serial number of the first network card by reading the network card configuration, etc. The second serial number is the serial number of the first network card in the target communication plane.

[0077] 203. Calculate a phase deflection value of the target communication plane.

[0078] Since one network card corresponds to one communication plane, the communication device can obtain the number of communication planes according to the number of network cards included in the target server. The communication device then obtains the serial number of the target communication plane according to the first serial number of the first network card.

[0079] The communication device uses the serial number of the target communication plane as a deflection factor, and then combines the number of communication planes N and the number of network cards M included in the target communication plane to calculate the phase deflection value θp of the target communication plane. Specifically, the communication device calculates the deflection value of the target communication plane according to the following formula:

[0080] θp=M / N*first serial number

[0081] Among them, θp is rounded up, and the value less than 1 is taken as 1.

[0082] Exemplarily, the number of network cards included in the target communication plane is 4, the number of network cards included in the target server is also 4, and the first serial number of the first network card is 2, then the phase deflection value θp of the target communication plane is 4 / 4*2=2.

[0083] 204. Calculate a first communication partner sequence number, where the first communication partner sequence number is a sequence number of a communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner sequence number corresponds to the second network card.

[0084] The first network card communicates with the remaining network cards in the target communication plane through the target communication algorithm. The communication device calculates the communication object of the first network card at the Kth algorithm step according to the sequence number of the first network card in the target communication plane (i.e., the second sequence number), and the Kth algorithm step is any one of the multiple algorithm steps included in the target communication algorithm. Specifically, the communication device calculates the first communication partner sequence number according to the second sequence number, and the first communication partner sequence number is the sequence number of the communication object of the first network card in the Kth algorithm step in the target communication plane. The first communication partner sequence number corresponds to the second network card and is the sequence number of the second network card in the target communication plane. In other words, the first network card will communicate with the second network card at the Kth algorithm step.

[0085] 205. Obtain a second communication partner sequence number according to the phase deflection value and the first communication partner sequence number, where the second communication partner sequence number is a sequence number of the third network card in the target communication plane.

[0086] The new communication object of the first network card at the Kth algorithm step is obtained by phase shifting the first communication partner sequence number. Specifically, the phase shift value of the target communication plane is combined with the first communication partner sequence number to obtain the second communication partner sequence number, which is the sequence number of the new communication object of the first network card in the target communication plane at the Kth algorithm step. The second communication partner sequence number corresponds to the third network card and is the signal of the third network card in the target communication plane.

[0087] 206. Replace the communication object of the first network card in the Kth algorithm step from the second network card to the third network card.

[0088] The communication device updates the communication object of the first network card at the Kth algorithm step to the third network card.

[0089] In this embodiment, the phase deflection value of the communication plane is first calculated according to the number of network cards in the communication plane, the total number of communication planes and the serial number of the communication plane, and then the communication object of the network card is recalculated according to the phase deflection value of the communication plane, so that the traffic in different communication planes between two nodes has a phase difference, thereby balancing the traffic of the entire network, reducing the bandwidth load of a single communication link, improving the bandwidth utilization of the entire network, reducing the probability of congestion, and improving communication performance.

[0090] Furthermore, this embodiment is applied to Mesh networking scenarios, which can reduce networking costs.

[0091] In the embodiment of the present application, there are multiple target communication algorithms, such as Having-Doubling algorithm or Pairwise algorithm. Different target communication algorithms have different methods for calculating the first communication partner sequence number and the second communication partner sequence number, which are described below respectively.

[0092] See also Figure 3 , which is a schematic diagram of another embodiment of the communication method provided in an embodiment of the present application. Figure 3 In the illustrated embodiment, nodes communicate with each other through the All-Reduce operator, and the communication algorithm used is the Having-Doubling algorithm. Figure 3 The illustrated embodiment includes steps 301 to 309 .

[0093] 301. Perform collective communication on N network cards in the target server.

[0094] Based on the node interconnection characteristics between network cards, the Reduce-Scatter operator is used to perform collective communication on N network cards.

[0095] 302. Obtain a first serial number and a second serial number of a first network card.

[0096] The communication device obtains the first serial number of the first network card in the target server and the second serial number of the first network card in the target communication plane by obtaining environment variables or reading the network card configuration.

[0097] 303. In the All-Reduce operator scenario, the Having-Doubling algorithm is used to perform collective communication on the network cards in the target communication plane.

[0098] The target communication plane includes M network cards, and the Having-Doubling algorithm will generate log2M algorithm steps. For example, if the target communication plane includes 4 network cards, log24=2 algorithm steps will be generated during the communication process.

[0099] 304. Calculate a phase deflection value of the target communication plane.

[0100] In this embodiment, step 304 and Figure 2 Step 203 in the illustrated embodiment is similar and will not be described in detail here.

[0101] 305. Calculate the communication object of the first network card at the Kth algorithm step.

[0102] The communication device calculates the communication object of the first network card in the Kth algorithm step based on the second sequence number of the first network card and the number of algorithm steps. The sequence number of the communication object in the target communication plane is the first communication partner sequence number, and the first communication partner sequence number corresponds to the second network card.

[0103] Specifically, the communication object of the first network card at the Kth algorithm step is calculated based on the following formula:

[0104] First communication partner sequence number = second sequence number ^ (1 << ((K + first sequence number) % log2M))

[0105] Among them, ^ represents the bitwise XOR operation, << represents the left shift operation, and % represents the remainder operation. Both the bitwise XOR operation and the left shift operation are binary operations.

[0106] Exemplarily, the target communication plane includes 4 network cards, the first serial number of the first network card is 2, the second serial number is 3, and the first serial number is the serial number of the target communication plane. The total number of algorithm steps is log24=2, and K is the first algorithm step. Then (K+first serial number)%log2M is (1+2)%2=1, that is, the number of bits to be shifted left is 1. The binary number of 1 shifted left by 1 bit is 10, which is equal to 2 when converted to decimal. Therefore, the serial number of the first communication partner is 3^2, the binary representation of 3 is 11, the binary representation of 2 is 10, and the binary number obtained by 3^2 is 01, which is equal to 1 when converted to decimal. That is, the serial number of the communication object of the first network card in the first algorithm step in the target communication plane is 1, and serial number 1 corresponds to the second network card.

[0107] 306. Calculate the second communication partner sequence number.

[0108] The communication device combines the phase deflection value of the target communication plane with the first communication partner sequence number to obtain the second communication partner sequence number, which is the sequence number of the new communication object of the first network card in the target communication plane at the Kth algorithm step.

[0109] Specifically, the new communication object of the first network card at the Kth algorithm step is calculated based on the following formula:

[0110] Second communication partner number = (first communication partner number + θp) % M

[0111] Among them, θp is the phase deflection value of the target communication plane, M is the number of network cards in the target communication plane, and % is the modulo operation.

[0112] Taking step 305 as an example, the target communication plane includes 4 network cards, the target server also includes 4 network cards, the first serial number of the first network card is 2, the second serial number is 3, and the first communication partner serial number is calculated to be 1. , then the phase deflection value of the target communication plane θp = 4 / 4*2 = 2. Therefore, the second communication partner serial number = (1+2)%4 = 3, that is, the serial number of the new communication object of the first network card in the Kth algorithm step in the target communication plane is 3.

[0113] 307. Update the communication object of the first network card in the Kth algorithm step.

[0114] The communication device searches for a network card with a sequence number equal to the sequence number of the second communication partner on the target communication plane, and determines that the second communication partner sequence number corresponds to the third network card, that is, the third network card is the new communication object of the first network card at the Kth algorithm step. The communication device replaces the communication object of the first network card at the Kth algorithm step from the second network card to the third network card, and communicates with the first network card and the third network card at the Kth algorithm step.

[0115] 308. According to the method described in steps 305 to 307, the communication objects of the N network cards in the target server in each algorithm step are updated, and when executing the algorithm step, the N network cards communicate with the updated communication objects.

[0116] 309. Perform collective communication on N network cards in the target server.

[0117] After N network cards have completed the communication process of all algorithm steps, the All-Gather operator is used to perform collective communication on the N network cards.

[0118] Figure 3 The theoretical communication traffic characteristics before and after updating the communication object in the embodiment shown are as follows: Figure 4a and Figure 4b shown. Figure 4a and Figure 4b In the figure, a communication system including four servers and each server including four network cards is used as an example for explanation. The Having-Doubling algorithm includes two algorithm steps, in which the line segments represent the communication traffic between the network cards.

[0119] See also Figure 4a , Figure 4a is the theoretical communication flow characteristics before replacing the communication object (before constructing the phase deflection process). Figure 4a As shown in the figure, in the first algorithm step, the four network cards of server A all communicate with the four network cards in server B, and the four network cards of server C all communicate with the four network cards in server D. The traffic between nodes is concentrated on the communication link between server A and server B, and the communication link between server C and server D, while the other links are idle. In the second algorithm step, the four network cards of server A all communicate with the four network cards in server C, and the four network cards of server B all communicate with the four network cards in server D. The traffic between nodes is concentrated on the communication link between server A and server C, and the communication link between server B and server D, while the other links are idle. In this case, the traffic is concentrated on some links, which is prone to congestion.

[0120] See also Figure 4b , Figure 4b is the theoretical communication flow characteristic after replacing the communication object (after constructing the phase deflection process). Figure 4b As shown in the figure, after updating the communication object, the communication objects of the server in the two algorithm steps become multiple, which increases the number of links that can be used simultaneously, makes the communication traffic between nodes more balanced, reduces the congestion probability, and is expected to reduce the communication delay by 20% in the All-Reduce collective communication scenario.

[0121] This embodiment describes the process of using the Having-Doubling algorithm to perform inter-node communication in the All-Reduce collective communication scenario, expands the application scenarios of the solution, and improves the feasibility of the solution.

[0122] See below Figure 5 , a schematic diagram of another embodiment of the communication method provided in an embodiment of the present application. Figure 5 In the illustrated embodiment, nodes communicate with each other through an All-to-all operator, and the communication algorithm used is a Pairwise algorithm. Figure 5 The illustrated embodiment includes steps 501 to 508 .

[0123] 501. Perform collective communication on N network cards in the target server.

[0124] Based on the node interconnection characteristics between network cards, the All-to-all operator is first used to perform collective communication on N network cards.

[0125] 502. Obtain a first serial number and a second serial number of a first network card.

[0126] In this embodiment, step 502 and Figure 3 Step 302 in the illustrated embodiment is similar and will not be described in detail here.

[0127] 503. In the All-to-all operator scenario, the Pairwise algorithm is used to perform collective communication on the network cards in the target communication plane.

[0128] The target communication plane includes M network cards, and the Pairwise algorithm will generate M-1 algorithm steps. For example, if the target communication plane includes 4 network cards, 4-1=3 algorithm steps will be generated during the communication process.

[0129] 504. Calculate a phase deflection value of the target communication plane.

[0130] In this embodiment, step 504 and Figure 2 Step 203 in the illustrated embodiment is similar and will not be described in detail here.

[0131] 505. Calculate the communication object of the first network card at the Kth algorithm step.

[0132] In each algorithm step of the Pairwise algorithm, the network card has two communication objects. The communication device calculates the communication object of the first network card in the Kth algorithm step based on the second sequence number of the first network card, and the sequence number of the communication object in the target communication plane is the first communication partner sequence number. There are two communication objects, so the first communication partner sequence number includes the third communication partner sequence number and the fourth communication partner sequence number. The first communication partner sequence number corresponds to the second network card, and the first communication partner sequence number includes the third communication partner sequence number and the fourth communication partner sequence number, so the second network card also includes two network cards, namely the fourth network card and the fifth network card. Among them, the third communication partner sequence number corresponds to the fourth network card, and the fourth communication partner sequence number corresponds to the fifth network card.

[0133] Specifically, the communication object of the first network card at the Kth algorithm step is calculated based on the following formula:

[0134] The third communication partner serial number = (the second serial number + MK) % M

[0135] Fourth communication partner serial number = (second serial number + K) % M

[0136] Among them, % represents the remainder operation.

[0137] Exemplarily, the target communication plane includes 4 network cards, the first serial number of the first network card is 2, the second serial number is 3, and the first serial number is the serial number of the target communication plane. The total number of algorithm steps is 4-1=3, and K is the first algorithm step. Then the serial number of the third communication partner = (3+4-1)%4=2, and the serial number of the fourth communication partner = (3+1)%4=0, that is, the serial numbers of the communication objects of the first network card in the first algorithm step in the target communication plane are 2 and 0, and the serial number 2 corresponds to the fourth network card, and the serial number 3 corresponds to the fifth network card.

[0138] 506. Calculate the second communication partner sequence number.

[0139] The communication device combines the phase deviation value of the target communication plane with the first communication partner sequence number to obtain the second communication partner sequence number, which is the sequence number of the new communication object of the first network card in the target communication plane at the Kth algorithm step. The first communication partner sequence number includes two sequence numbers, so the second communication partner sequence number obtained after adding the phase deviation value also includes two sequence numbers, namely the fifth communication partner sequence number and the sixth communication partner sequence number. Among them, the fifth communication partner sequence number is obtained by combining the third communication partner sequence number with the phase deviation value, and the fifth communication partner sequence number corresponds to the sixth network card. The sixth communication partner sequence number is obtained by combining the fourth communication partner sequence number with the phase deviation value, and the sixth communication partner sequence number corresponds to the seventh network card.

[0140] Specifically, the new communication object of the first network card at the Kth algorithm step is calculated based on the following formula:

[0141] Fifth communication partner number = (third communication partner number + θp) % M

[0142] Sixth communication partner number = (fourth communication partner number + θp) % M

[0143] Among them, % represents the remainder operation.

[0144] Taking step 505 as an example, the target communication plane includes 4 network cards, the target server also includes 4 network cards, the first serial number of the first network card is 2, the second serial number is 3, and the K-th algorithm step calculates that the serial number of the third communication partner is 2, and the serial number of the fourth communication partner is 0. Then the phase deflection value θp of the target communication plane is 4 / 4*2=2. Therefore, the serial number of the fifth communication partner is (3+2)%4=1, and the serial number of the sixth communication partner is (0+2)%4=2, that is, the serial numbers of the new communication objects of the first network card in the target communication plane are 1 and 2 at the K-th algorithm step, and the serial number 1 corresponds to the sixth network card, and the serial number 2 corresponds to the seventh network card.

[0145] It is understandable that the fourth network card and the seventh network card may be the same network card, and the fifth network card and the sixth network card may be the same network card. For example, in the above example, the fourth network card and the seventh network card are the same network card. Alternatively, the fourth network card and the sixth network card may be the same network card, and the fifth network card and the seventh network card may be the same network card. For example, when the first sequence number is 0, the phase deflection value is 0, and the original communication object and the new communication object are the same network card.

[0146] 507. Update the communication object of the first network card in the Kth algorithm step.

[0147] The communication device replaces the communication object of the first network card in the Kth algorithm step from the fourth network card to the sixth network card, and replaces the fifth network card with the seventh network card, and communicates with the sixth network card and the seventh network card in the Kth algorithm step.

[0148] 508. According to the method described in steps 505 to 507, the communication objects of the N network cards in the target server in each algorithm step are updated, and when executing the algorithm step, the N network cards communicate with the updated communication objects.

[0149] Figure 5 The theoretical communication traffic characteristics before and after updating the communication object in the embodiment shown are as follows: Figure 6a and Figure 6b shown. Figure 6a and Figure 6b In the figure, a communication system including 4 servers and each server including 4 network cards is used as an example for explanation. When using the Pairwise algorithm, it includes 3 algorithm steps, and the line segments represent the communication traffic between the network cards.

[0150] See also Figure 6a , Figure 6a is the theoretical communication flow characteristics before replacing the communication object (before constructing the phase deflection process). Figure 6a As shown in the figure, in the three algorithm steps, the communication traffic is concentrated on some links, and some links are idle. In this case, congestion is prone to occur, affecting the communication performance.

[0151] See also Figure 6b , Figure 6b is the theoretical communication flow characteristic after replacing the communication object (after constructing the phase deflection process). Figure 6b As shown in the figure, after updating the communication object, the communication traffic between nodes is more balanced than before replacing the communication object, and the number of links that can be used simultaneously increases, thereby reducing the probability of congestion and improving communication performance. It is expected that the communication delay can be reduced by 20% in the All-to-all collective communication scenario.

[0152] This embodiment describes the process of using the Pairwise algorithm to communicate between nodes in an All-to-all collective communication scenario, expands the application scenarios of the solution, and improves the feasibility of the solution.

[0153] The above describes the embodiments of the present application from the perspective of the method. The following describes the relevant devices in the embodiments of the present application from the perspective of specific device implementation.

[0154] See also Figure 7 , an embodiment of the present application provides a schematic diagram of a communication device 700, wherein the communication device 700 includes an acquisition unit 701, a calculation unit 702, a combination unit 703 and a replacement unit 704.

[0155] The acquisition unit 701 is used to obtain a first serial number, where the first serial number is the serial number of the first network card in the target server. The target server includes N network cards, each of the N network cards corresponds to a communication plane, wherein the first network card corresponds to a target communication plane, the target communication plane includes M network cards, the M network cards belong to different servers, the serial numbers of the M network cards in their respective servers are the same as the first serial number, and the M network cards communicate with each other through a target communication algorithm, and the target communication algorithm includes multiple algorithm steps.

[0156] The acquiring unit 701 is further configured to acquire a second sequence number, where the second sequence number is a sequence number of the first network card in the target communication plane.

[0157] The calculation unit 702 is used to calculate the phase deflection value of the target communication plane based on the first sequence number, the number of network cards included in the target server, and the number of network cards included in the target communication plane.

[0158] The calculation unit 702 is further used to calculate the first communication partner sequence number based on the second sequence number, the first communication partner sequence number is the sequence number of the communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner sequence number corresponds to the second network card.

[0159] The combining unit 703 is used to combine the first communication partner sequence number and the phase deflection value to obtain the second communication partner sequence number, where the second communication partner sequence number is the sequence number of the third network card in the target communication plane.

[0160] The replacement unit 704 is used to replace the communication object of the first network card in the Kth algorithm step from the second network card to the third network card.

[0161] Optionally, the target communication algorithm is a Having-Doubling algorithm, and the number of algorithm steps is log2M algorithm steps.

[0162] Optionally, the communication device 700 further includes a communication unit 705, which is used to perform collective communication on the N network cards using a Reduce-Scatter operator; the communication unit is also used to perform collective communication on the N network cards using an All-Gather operator.

[0163] Optionally, the calculation unit 702 is specifically used to calculate the first communication partner sequence number based on the following formula: first communication partner sequence number = second sequence number ^(1<<((K+first sequence number)%log2M)), where ^ represents a bitwise XOR operation, << represents a left shift operation, and % represents a modulo operation.

[0164] Optionally, the target communication algorithm is a Pairwise algorithm, and the plurality of algorithm steps is M-1 algorithm steps.

[0165] Optionally, the communication unit 705 is further configured to use an All-to-all operator to perform collective communication on N network cards.

[0166] Optionally, the first communication partner sequence number includes a third communication partner sequence number and a fourth communication partner sequence number, and the calculation unit 702 is specifically used to calculate the third communication partner sequence number and the fourth communication partner sequence number based on the second sequence number; the third communication partner sequence number and the fourth communication partner sequence number are calculated based on the following formula: third communication partner sequence number = (second sequence number + MK) % M, fourth communication partner sequence number = (second sequence number + K) % M, where % represents remainder.

[0167] Each unit in the communication device 700 performs the above Figure 2 , Figure 3 as well as Figure 5 The operation of the communication device in the illustrated embodiment will not be described in detail here.

[0168] See below Figure 8 , is a possible structural diagram of a communication device 800 provided in an embodiment of the present application, including a processor 801, a communication interface 802, a memory 803 and a bus 804. The processor 801, the communication interface 802 and the memory 803 are interconnected via the bus 804. In the embodiment of the present application, the processor 801 is used to control and manage the actions of the communication device. For example, the processor 801 is used to execute Figure 2 The steps performed by the communication device in the method embodiment are shown. The communication interface 802 is used to support the communication device to communicate. The memory 803 is used to store program codes and data of the communication device.

[0169] Among them, the processor 801 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the contents disclosed in this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The bus 804 can be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0170] The embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the aforementioned Figure 2 , Figure 3 as well as Figure 5 The method in the illustrated embodiment.

[0171] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the aforementioned Figure 2 , Figure 3 as well as Figure 5 The method in the illustrated embodiment.

[0172] The embodiment of the present application also provides a chip system, the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Figure 2 , Figure 3 as well as Figure 5 The method in the illustrated embodiment.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0174] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0177] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A communication method, characterized in that: include: Obtain a first serial number, where the first serial number is a serial number of a first network card in a target server, where the target server includes N network cards, where each of the N network cards corresponds to a communication plane, where the first network card corresponds to a target communication plane, where the target communication plane includes M network cards, where the M network cards belong to different servers, where the serial numbers of the M network cards in their respective servers are the same as the first serial number, where the M network cards communicate with each other through a target communication algorithm, where the target communication algorithm includes a plurality of algorithm steps; Obtain a second serial number, where the second serial number is the serial number of the first network card in the target communication plane; Calculating a phase deflection value of the target communication plane based on the first sequence number, the number of network cards included in the target server, and the number of network cards included in the target communication plane; Calculate a first communication partner sequence number based on the second sequence number, the first communication partner sequence number being the sequence number of the communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner sequence number corresponds to the second network card; Combining the first communication partner sequence number and the phase deflection value to obtain a second communication partner sequence number, where the second communication partner sequence number is the sequence number of the third network card in the target communication plane; The communication object of the first network card in the Kth algorithm step is replaced by the third network card from the second network card.

2. The method according to claim 1, characterized in that: The target communication algorithm is a Having-Doubling algorithm, and the multiple algorithm steps are log2M algorithm steps.

3. The method according to claim 2, characterized in that Before obtaining the first sequence number, the method further includes: Use the Reduce-Scatter operator to perform collective communication on the N network cards; After the communication object of the first network card in the Kth algorithm step is replaced by the third network card from the second network card, the method further includes: The All-Gather operator is used to perform collective communication on the N network cards.

4. The method according to claim 2 or 3, characterized in that: The calculating the first communication partner sequence number based on the second sequence number comprises: The calculation is based on the following formula: The first communication partner serial number = the second serial number ^(1<<((K+first serial number)%log2M)), where ^ represents a bitwise exclusive OR operation, << represents a left shift operation, and % represents a modulo operation.

5. The method according to claim 1, characterized in that The target communication algorithm is a Pairwise algorithm, and the multiple algorithm steps are M-1 algorithm steps.

6. The method according to claim 5, characterized in that Before obtaining the first sequence number, the method further includes: The All-to-all operator is used to perform collective communication on the N network cards.

7. The method according to claim 5 or 6, characterized in that: The first communication partner sequence number includes a third communication partner sequence number and a fourth communication partner sequence number, and the calculating the first communication partner sequence number based on the second sequence number includes: Calculate a third communication partner sequence number and a fourth communication partner sequence number based on the second sequence number; The third communication partner serial number and the fourth communication partner serial number are calculated based on the following formula: The third communication partner serial number = (the second serial number + MK) % M, the fourth communication partner serial number = (the second serial number + K) % M, wherein % represents the remainder.

8. A communication device, characterized in that: include: an acquiring unit, configured to acquire a first serial number, wherein the first serial number is a serial number of a first network card in a target server, wherein the target server includes N network cards, each of the N network cards corresponds to a communication plane, wherein the first network card corresponds to a target communication plane, the target communication plane includes M network cards, the M network cards belong to different servers, the serial numbers of the M network cards in their respective servers are the same as the first serial number, and the M network cards communicate with each other through a target communication algorithm, and the target communication algorithm includes a plurality of algorithm steps; The acquiring unit is further used to acquire a second serial number, where the second serial number is a serial number of the first network card in the target communication plane; a calculation unit, configured to calculate a phase deflection value of the target communication plane based on the first sequence number, the number of network cards included in the target server, and the number of network cards included in the target communication plane; The calculation unit is further used to calculate a first communication partner sequence number based on the second sequence number, the first communication partner sequence number being a sequence number of a communication object of the first network card in the Kth algorithm step in the target communication plane, and the first communication partner sequence number corresponds to the second network card; a combining unit, configured to combine the first communication partner sequence number and the phase deflection value to obtain a second communication partner sequence number, where the second communication partner sequence number is a sequence number of the third network card in the target communication plane; A replacement unit is used to replace the communication object of the first network card in the Kth algorithm step from the second network card to the third network card.

9. The communication device according to claim 8, characterized in that The target communication algorithm is a Having-Doubling algorithm, and the multiple algorithm steps are log2M algorithm steps.

10. The communication device according to claim 9, characterized in that The communication device further comprises: A communication unit, used for performing collective communication on the N network cards using a Reduce-Scatter operator; The communication unit is also used for: The All-Gather operator is used to perform collective communication on the N network cards.

11. The communication device according to claim 9 or 10, characterized in that: The computing unit is specifically used for: The first communication partner sequence number is calculated based on the following formula: The first communication partner serial number = the second serial number ^(1<<((K+first serial number)%log2M)), where ^ represents a bitwise exclusive OR operation, << represents a left shift operation, and % represents a modulo operation.

12. The communication device according to claim 8, characterized in that The target communication algorithm is a Pairwise algorithm, and the multiple algorithm steps are M-1 algorithm steps.

13. The communication device according to claim 12, characterized in that The communication unit is also used for: The All-to-all operator is used to perform collective communication on the N network cards.

14. The communication device according to claim 12 or 13, characterized in that: The first communication partner sequence number includes a third communication partner sequence number and a fourth communication partner sequence number, and the calculation unit is specifically used for: Calculate a third communication partner sequence number and a fourth communication partner sequence number based on the second sequence number; The third communication partner serial number and the fourth communication partner serial number are calculated based on the following formula: The third communication partner serial number = (the second serial number + MK) % M, the fourth communication partner serial number = (the second serial number + K) % M, wherein % represents the remainder.

15. A communication device, characterized in that: include: Processor and memory; The memory is used to store instructions; The processor is configured to execute instructions stored in the memory to implement the method according to any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

17. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images