Transmission device that pre-discards packets to be transmitted

Abstract

To provide a transmitter and a program for preventing degradation of a processing result in a receiver and suppressing increase in time until a processing result is obtained in the receiver, in preliminarily discarding transmission packets for avoiding congestion.SOLUTION: A transmitter 1 includes: a source unit 10 which generates a plurality of first packets based on information to be transmitted to a receiver 2; a control unit 13 which sets a discard rate R; and a packet discard unit 11 which discards some of the first packets according to the discard rate R and transmits remaining second packets to the receiver. The control unit acquires first information (delay L) corresponding to a receiving period of the receiver 2 receiving the second packets and second information (evaluation value P) indicating performance of a result of processing which is performed by the receiver 2 based on the second packets, from the receiver 2, updates the discard rate R based on the delay L and the evaluation value P, and stores the discard rate R in a storage unit 12, to control the discard rate on the basis of the first information and the second information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a technique for pre-discarding packets transmitted to avoid convergence.

Background Art

[0002] Non-Patent Document 1 discloses a configuration for performing distributed deep learning (DDL) of a neural network (NN). In DDL, the processing performed for learning one NN is divided and executed by a plurality of processing devices. In DDL, a processing device generally requires information obtained by the processing performed by other processing devices in order to execute the processing assigned to itself. For this reason, in DDL, packet transmission and reception are performed between processing devices. In the following description, a processing device that transmits a packet to another processing device is referred to as a "transmitting device", and a processing device that receives a packet from another processing device is referred to as a "receiving device". Note that each of the plurality of processing devices can function as both a "transmitting device" and a "receiving device".

[0003] For example, a packet transmitted from a transmitting device during forward propagation in NN learning includes information indicating the output value of the layer for which the transmitting device is responsible among a plurality of layers of the NN. Further, during backpropagation, a packet transmitted from the transmitting device includes information indicating the error of the layer for which the transmitting device is responsible. Non-Patent Document 1 discloses a configuration in which processing devices are connected by an optical path. Non-Patent Document 2 also discloses a configuration in which, in order to avoid convergence in DDL, a part of the packets to be transmitted to the receiving device is discarded in the transmitting device. In DDL, it is known that even if a part of the information transmitted from the transmitting device to the receiving device is missing, it does not have a great impact on learning.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

[0005] However, if the transmitting device discards a large number of packets to avoid congestion, a lot of information will be missing at the receiving device, and the performance of the processing result at the receiving device may deteriorate. On the other hand, if the packet discard at the transmitting device is not sufficient, congestion cannot be avoided, and the time until the processing result is obtained at the receiving device that receives the packet will be prolonged.

[0006] The present disclosure provides a technique for suppressing deterioration of the processing result at the receiving device and suppressing an increase in the time until the processing result is obtained at the receiving device in a transmitting device that pre-discards packets to be transmitted. [Means for Solving the Problems]

[0007] According to one aspect of the present disclosure, a transmitting device includes a generating means for generating a plurality of first packets based on information to be transmitted to a receiving device, a controlling means for setting a discard rate, a transmitting means for discarding a part of the plurality of first packets according to the discard rate and transmitting the remaining plurality of second packets to the receiving device, an acquiring means for acquiring from the receiving device first information corresponding to a reception period of the plurality of second packets by the receiving device and second information indicating performance of a result of processing performed by the receiving device based on the plurality of second packets, and the controlling means controls the discard rate based on the first information and the second information.

Advantages of the Invention

[0008] According to the present disclosure, in a transmitting device that pre-discards packets to be transmitted, it is possible to suppress degradation of a processing result in a receiving device and suppress an increase in time until the processing result is obtained in the receiving device.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential for the invention. Two or more of the plurality of features described in the embodiments may be arbitrarily combined. Also, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] Hereinafter, embodiments will be described by taking the case of performing DDL by a plurality of processing devices as an example. As described above, in DDL, packets are transmitted and received between a plurality of processing devices. In the present embodiment, a processing device that transmits a packet is referred to as a transmitting device, and a processing device that receives a packet is referred to as a receiving device. Note that each processing device can function as both a transmitting device and a receiving device. Although there are a plurality of pairs of a transmitting device and a receiving device, the processes performed by the transmitting device to transmit a packet to the receiving device are the same in all pairs. Therefore, hereinafter, only one pair of the transmitting device 1 and the receiving device 2 shown in FIG. 1 will be described.

[0012] First, terms used in the present embodiment will be described. Deep learning of NN is executed by forward propagation of the output value of each layer based on teacher data and backward propagation of an error based on the result of the forward propagation. Hereinafter, the process until forward propagation and backward propagation of the error based on the result of the forward propagation are performed to update each weight of NN is referred to as one "stage". In deep learning, this stage is repeatedly performed.

[0013] In each stage, the transmitting device 1 transmits a plurality of packets to the receiving device 2. The plurality of packets include, for example, the output values of the layer for which the transmitting device 1 is responsible among the plurality of layers of NN, and the processing results at the transmitting device 1 such as errors. The processing results are used for the processing at the receiving device 2. In the present embodiment, "delay" means the period from the timing when the receiving device 2 starts receiving the first packet from the transmitting device 1 to the timing when the reception of the last packet is completed in one stage. That is, the "delay" in the present embodiment corresponds to the reception period during which the receiving device 2 receives packets from the transmitting device 1. It is assumed that the amount of data transmitted from the transmitting device 1 to the receiving device 2 is constant in each stage. Since the amount of data transmitted from the transmitting device 1 to the receiving device 2 is constant in each stage, when there is no congestion in the network path from the transmitting device 1 to the receiving device 2, the delay is substantially constant in each stage. On the other hand, when congestion occurs in the network path from the transmitting device 1 to the receiving device 2, the delay becomes larger compared to the case where there is no congestion. Generally, the higher the degree of congestion, the larger the delay.

[0014] In each stage, when the transmission of packets from the transmitting device 1 to the receiving device 2 is completed, the receiving device 2 performs predetermined processing. When the predetermined processing at the receiving device 2 is completed, the performance of the processing result is evaluated to determine an evaluation value. The method for evaluating the performance is arbitrary. For example, in DDL, each receiving device 2 receives an error for each stage. Generally, the error becomes smaller as the number of repetitions of the stage increases. However, when the receiving device 2 has not received sufficient information required for processing, the degree of decrease in the error from the previous stage becomes smaller, or in some cases, it may increase. Therefore, it is possible to evaluate the performance of each stage based on the difference in errors between two consecutive stages and determine an evaluation value.

[0015] The operation of the transmission device 1 and the reception device 2 in one stage will be described below with reference to FIG. 1. The source unit 10 of the transmission device 1 packetizes the information that needs to be transmitted to the reception device 2 among the information obtained from the processing results of the transmission device 1 in a certain stage and outputs it to the packet discard unit 11. The storage unit 12 stores the packet discard rate R. The packet discard unit 11 discards some of the plurality of packets from the source unit 10 based on the discard rate R and transmits the remaining plurality of packets to the reception device 2. For example, when the value of the packet discard rate R is 1% and 100 packets are received from the source unit 10, the packet discard unit 11 discards one of them and transmits the remaining 99 packets to the reception device 2.

[0016] The reception unit 20 of the reception device 2 receives the packets from the transmission device 1 and outputs them to the processing unit 21. The processing unit 21 executes a predetermined process given to the reception device 2 using the packets from the transmission device 1. Further, the reception unit 20 measures the above-described delay L based on the received packets and notifies the transmission device 1. Furthermore, the processing unit 21 determines the above-described performance evaluation value P based on the result of the predetermined process and notifies the transmission device 1. The control unit 13 of the transmission device 1 updates the discard rate R based on the delay L and the evaluation value P acquired from the reception device 2 and stores it in the storage unit 12. This discard rate R is used in the next stage. Note that the control unit 13 controls the overall operation performed by the transmission device 1.

[0017] FIG. 2 is a flowchart of the process executed by the transmission device 1. In the flowchart of FIG. 2, each stage is sequentially denoted as the s-th stage. Note that s is an integer of 0 or more. Also, the discard rate R used in the s-th stage is denoted as R s and the delay L and the evaluation value P notified from the reception device 2 in the s-th stage are denoted as the delay L s and the evaluation value P s . Note that the discard rate R 0 in the first stage, that is, the 0-th stage, is set to 0.

[0018] In S10, the control unit 13 initializes an index s indicating the number of repetitions of the stage to 0. In S11, the packet discard unit 11 discards packets from the source unit 10 at a discard rate R s and transmits the remaining packets to the receiving device 2. In the case of the 0th stage, since the discard rate R s is 0, the packet discard unit 11 transmits all the packets from the source unit 10 to the receiving device 2. In S12, the control unit 13 acquires the delay L s and the evaluation value P s at the s-th stage from the receiving device 2.

[0019] In S13, the control unit 13 determines whether s = 0, that is, whether it is the 0th stage. If it is the 0th stage, in S14, the control unit 13 sets the discard rate R 1 at the next stage, that is, the 1st stage, to a predetermined value R p . On the other hand, if it is not the 0th stage, in S15, the control unit 13 calculates the discard rate R s-1 at the next stage, that is, the (s + 1)-th stage, based on the delay L s-1 and the evaluation value P s at the previous stage and the delay L s and the evaluation value P s+1 at the current stage. In S16, the control unit 13 determines whether the current stage is the final stage. If the current stage is not the final stage, in S17, the control unit 13 increments the index s by 1 and repeats the process from S11. On the other hand, if the current stage is the final stage, the control unit 13 ends the process in FIG. 2.

[0020] FIG. 3 is a flowchart of the process in S15 of FIG. 2. In S20, the control unit 13 compares the evaluation value P s at the s-th stage with the evaluation value P s-1 at the (s - 1)-th stage, and determines whether the evaluation value P s is greater than the evaluation value P s-1Determine whether it is more deteriorated and the degree of deterioration is greater than the first threshold. That is, compare with the performance at the (s - 1)th stage and determine whether the performance at the s-th stage is deteriorated more than the first threshold. If the performance at the s-th stage is deteriorated more than the first threshold compared to the previous stage, the control unit 13 sets the rejection rate R s to be too large. Therefore, in S22, the control unit 13 sets the rejection rate R s+1 at the next (s + 1)th stage to be decreased by a predetermined value R s from the rejection rate R p2 at the current s-th stage.

[0021] On the other hand, if the performance at the s-th stage is not deteriorated more than the first threshold compared to the previous stage, in S21, the control unit 13 compares the delay L s at the s-th stage with the delay L s-1 at the (s - 1)th stage, and determines whether the delay L s has increased compared to the delay L s-1 and the degree of increase is greater than the second threshold. That is, compare with the delay at the (s - 1)th stage and determine whether the delay at the s-th stage has increased more than the second threshold. If the delay at the s-th stage has increased more than the second threshold compared to the previous stage, the control unit 13 sets the rejection rate R s to be too small. Therefore, in S23, the control unit 13 sets the rejection rate R s+1 at the next (s + 1)th stage to be increased by a predetermined value R s from the rejection rate R p1 at the current s-th stage. On the other hand, if the delay at the s-th stage has not increased more than the second threshold compared to the previous stage, the control unit 13 determines that the current rejection rate R s is appropriate. In this case, in S24, the control unit 13 determines to use the current rejection rate R s without change at the (s + 1)th stage.

[0022] By controlling the discard rate R as described above, it is possible to suppress the delay and control the discard rate R so as to suppress a decrease in performance. By suppressing the delay, that is, by suppressing an increase in the reception period of information from the transmission device 1 at the reception device 2, it is possible to suppress an increase in the time until a processing result is obtained at the reception device. Further, by suppressing a decrease in performance, it is possible to suppress deterioration of the processing result at the reception device 2.

[0023] Note that since the discard rate R cannot take a negative value, when the discard rate R s+1 becomes a negative value in S22, the control unit 13 sets the discard rate R s+1 to 0. Also, an upper limit value of the discard rate R is determined, and when the discard rate R s+1 exceeds the upper limit value in S23, the control unit 13 may be configured to set the discard rate R s+1 to the upper limit value. Further, the predetermined values R p 、R p+1 and R p+2 may be the same value or different values.

[0024] <Modification Example> In FIG. 3, when the delay at the s-th stage increases by more than the second threshold value with respect to the previous stage in S21, the control unit 13 increases the discard rate R at the next stage by a predetermined value in S23, and otherwise, in S24, it is determined to use the discard rate R as it is at the next stage without changing it. In this modification example, when it reaches the k-th stage (k is an integer of 2 or more), the discard rate R s at the next (s + 1)-th stage is determined by comparing the current delay L s+1 at the s-th stage with the delays at the past k stages. Note that the processing until the k-th stage is the same as that in FIG. 3.

[0025] FIG. 4 shows a flowchart of the processing performed by the control unit 13 in S15 of FIG. 2 in this modification example. Note that in this modification example, S15 in FIG. 2 is "L s ~L s-k and P s and P s-1 Based on and R s+1It is determined. In FIG. 4, the same processing steps as those in the flowchart of FIG. 3 are assigned the same step numbers. The difference from the flowchart of FIG. 3 is that S21 in FIG. 3 becomes S30. In S30, the control unit 13 delays L s and the delays L s-1 L s-2 ···, L s-k at the past k stages are compared. Then, if the increase amount of the delay L s-1 L s-2 ···, L s-k for at least one of L s is greater than the second threshold value, the control unit 13 increases the discard rate R at the next stage by a predetermined value in S23. On the other hand, if the increase amount of the delay L s-1 L s-2 ···, L s-k for all of L s is not greater than the second threshold value, the control unit 13 determines in S24 to use the discard rate R as it is at the next stage without changing it.

[0026] In this modification example, since the discard rate R is controlled based on the comparison results with the delays L at not only one previous stage but also at a plurality of past stages, the discard rate R can be controlled so as to optimize the delay while suppressing the degradation of performance.

[0027] Although the embodiments have been described by taking the case of performing DDL as an example, the present disclosure can be applied to applications where there is no problem even if some information is missing. Examples of such applications are, for example, the distribution of contents such as videos and music. In the distribution of contents, even if some small amount of information is missing, it does not become impossible to view the contents. In the content distribution, the content can be divided into blocks of a predetermined data amount, and the transmission of one block can be regarded as one stage.

[0028] Note that the transmission device 1 according to the present disclosure can be realized by a computer program that causes the device having one or more processors to function as the transmission device 1 when executed by the one or more processors of the device. The computer program may include program instructions executable by one or more processors. The computer program is stored in a non-transitory computer-readable storage medium or can be distributed via a network.

[0029] With the above configuration, in a transmission device that pre-discards packets to be transmitted, it is possible to suppress deterioration of the processing result in the receiving device and to suppress an increase in the time until the processing result is obtained in the receiving device. Therefore, it becomes possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0030] The invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist of the invention.

Description of Reference Numerals

[0031] 10: Source unit, 11: Packet discard unit, 13: Control unit

Claims

1. generating means for generating a plurality of first packets based on information to be transmitted to a receiving device; control means for setting a discard rate; transmitting means for discarding a part of the plurality of first packets according to the discard rate and transmitting the remaining plurality of second packets to the receiving device; acquiring means for acquiring from the receiving device first information corresponding to a reception period of the plurality of second packets by the receiving device and second information indicating performance of a result of processing performed by the receiving device based on the plurality of second packets; comprising; The control means controls the discard rate based on the first information and the second information. A transmitting device.

2. The transmitting device repeatedly executes a plurality of stages, Each of the plurality of stages includes discarding a part of the plurality of first packets according to the discard rate and transmitting the remaining plurality of second packets to the receiving device, and acquiring the first information and the second information from the receiving device, When the control means indicates that the second information acquired in the first stage is significantly deteriorated compared to the performance indicated by the second information acquired in the second stage immediately preceding the first stage by more than a first threshold, the discard rate in the third stage to be performed next to the first stage is decreased by a first predetermined value from the discard rate in the first stage. The transmitting device according to claim 1.

3. When the control means indicates that the second information acquired in the first stage is not significantly deteriorated compared to the performance indicated by the second information acquired in the second stage by more than the first threshold, and when the first information acquired in the first stage indicates that it has increased by more than a second threshold compared to the reception period indicated by the first information acquired in the second stage, the discard rate in the third stage is increased by a second predetermined value from the discard rate in the first stage. The transmitting device according to claim 2.

4. The control means increases the discard rate in the third stage by a second predetermined value from the discard rate in the first stage when the second information acquired in the first stage indicates that the second information acquired in the second stage has not deteriorated significantly beyond the first threshold, and the first information acquired in the first stage indicates that the first information has increased by more than a second threshold relative to at least one of the reception periods indicated by the plurality of pieces of first information acquired in each of the plurality of stages executed before the first stage. The transmitting device according to claim 2.

5. The control means sets the discard rate in the third stage to be the same as the discard rate in the first stage when the second information acquired in the first stage indicates that the second information acquired in the second stage has not deteriorated significantly beyond the first threshold, and the first information acquired in the first stage indicates that the first information has not increased by more than a second threshold relative to the reception period indicated by the first information acquired in the second stage. The transmitting device according to claim 2.

6. The control means sets the discard rate in the third stage to be the same as the discard rate in the first stage when the second information acquired in the first stage indicates that the second information acquired in the second stage has not deteriorated significantly beyond the first threshold, and the first information acquired in the first stage indicates that the first information has not increased by more than a second threshold relative to all of the reception periods indicated by the plurality of pieces of first information acquired in each of the plurality of stages executed before the first stage. The transmitting device according to claim 2.

7. A computer program that, when executed by one or more processors of a device having one or more processors, causes the device to function as the transmitting device according to any one of claims 1 to 6.

Images