Bandwidth guarantee method and related equipment for edge cloud

Abstract

The embodiment of the present invention provides a bandwidth guarantee method, device, medium and electronic device for edge cloud, the method comprising: the sending end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine respectively according to the bandwidth weights of the first virtual machine and the second virtual machine; the sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting and numerical marking to the switch; when the sending end receives the data packet fed back by the receiving end, the sending end adjusts the sorting and numerical marking of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end. The scheme of the present invention can achieve good bandwidth isolation.

Description

Technical Field

[0001] The present invention relates to the communication field of cloud computing, and in particular to a bandwidth guarantee method, device, computer-readable storage medium and electronic device for edge cloud. Background Art

[0002] Traditional cloud computing has developed into three dimensions: central cloud, edge cloud, and IoT terminals. Among them, edge cloud has become an important development direction for cloud computing services due to its advantages of being close to customers and businesses, low latency, self-organization, and definability. Since an edge cloud data center carries a wide variety of commercial customers and businesses, and their technical requirements and usage methods for the network vary greatly, this has led to a large number of data streams from different senders and with very different characteristics mixed in the edge cloud data center network.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a bandwidth guarantee method, device, computer-readable storage medium and electronic device for edge cloud.

[0005] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0006] According to one aspect of an embodiment of the present invention, a bandwidth guarantee method for an edge cloud is provided, which is applied to a system including at least one sending end, one switch and one receiving end, including:

[0007] The transmitting end respectively sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine;

[0008] The sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting value marking to the switch;

[0009] When the sending end receives the data packet fed back by the receiving end, the sending end adjusts the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end.

[0010] In one embodiment, the sending end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, respectively, including:

[0011] The transmitting end marks, according to the bandwidth weights of the first virtual machine and the second virtual machine, the data packets sent by the first virtual machine and the second virtual machine as the sending time required for each data packet to be normally sent according to the bandwidth allocated to the first virtual machine and the second virtual machine; or

[0012] The sending end marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, and sorts them according to the length of time required for each data packet to be sent normally according to the bandwidth allocated to the first virtual machine and the second virtual machine, that is, the shorter the required sending time, the earlier the sequence number.

[0013] In one embodiment, when the sending end receives the data packet fed back by the receiving end, adjusting the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end includes:

[0014] When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the correct sorting value tag value from the data packet fed back by the receiving end;

[0015] The sending end adjusts the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to extracting the correct sorting value mark value from the data packet fed back by the receiving end.

[0016] In one embodiment, the method further comprises:

[0017] The sending end encapsulates the congestion time of the switch recorded by the sending end into the data packets sent by the first virtual machine and the second virtual machine.

[0018] In one embodiment, the method further comprises:

[0019] When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the latest congestion time of the switch from the data packet fed back by the receiving end;

[0020] The sending end encapsulates the latest congestion time of the switch into a data packet sent by the first virtual machine or the second virtual machine.

[0021] According to one aspect of an embodiment of the present invention, a bandwidth guarantee method for an edge cloud is provided, which is applied to a system including at least one sending end, one switch and one receiving end, including:

[0022] The switch receives data packets of the first virtual machine and the second virtual machine that have completed sorting numerical markings and are sent by the sending end;

[0023] The switch sorts the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking according to the sorting numerical markings, and sends them to the receiving end in sequence according to the sorting;

[0024] The switch marks the corresponding data packet when the sorting value mark of the data packet of the first virtual machine or the second virtual machine is less than or equal to the maximum sorting value mark of the data packets of the first virtual machine or the second virtual machine that have been sent and stored by the switch.

[0025] In one embodiment, the method further comprises:

[0026] When the congestion time of the switch encapsulated in the data packet of the first virtual machine or the second virtual machine is inconsistent with the latest congestion time stored in the switch, the switch marks the corresponding data packet.

[0027] According to one aspect of an embodiment of the present invention, a bandwidth guarantee method for an edge cloud is provided, which is applied to a system including at least one sending end, one switch and one receiving end, including:

[0028] The receiving end receives data packets of the first virtual machine and the second virtual machine sent by the switch;

[0029] The receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch;

[0030] The receiving end sends the feedback data packet to the sending end through the switch.

[0031] In one embodiment, the receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including:

[0032] The receiving end obtains the correct sorting value mark of the data packet from the data packet marked by the switch, and writes the correct sorting value mark of the data packet into the feedback data packet.

[0033] In one embodiment, the receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including:

[0034] The receiving end obtains the latest congestion time of the switch in the data packet marked by the switch, and writes the latest congestion time of the switch into the feedback data packet.

[0035] According to one aspect of an embodiment of the present invention, a bandwidth guarantee device for an edge cloud is provided, which is applied to a system including at least one sending end, one switch and one receiving end, including:

[0036] a marking module, configured to sort and numerically mark the data packets sent by the first virtual machine and the second virtual machine respectively according to the bandwidth weights of the first virtual machine and the second virtual machine;

[0037] A sending module, configured to send the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking to the switch;

[0038] The adjustment module is configured to adjust the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end when the data packet fed back by the receiving end is received.

[0039] According to one aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0040] one or more processors;

[0041] The storage device is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of the above embodiments.

[0042] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0043] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0044] It can achieve good bandwidth isolation, and also has the advantages of fast convergence, improved bandwidth utilization, and being friendly to short flows.

[0045] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings herein are incorporated into and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0047] Figure 1A bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention is schematically shown;

[0048] Figure 2 A bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention is schematically shown;

[0049] Figure 3 A bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention is schematically shown;

[0050] Figure 4 The classification of cloud computing network performance isolation methods in the prior art is shown;

[0051] Figure 5 A system framework diagram of an embodiment of the present disclosure is shown;

[0052] Figure 6 The data packet format of one embodiment of the present disclosure is shown;

[0053] Figure 7 A bandwidth guarantee device for edge cloud provided according to an embodiment of the present invention is shown;

[0054] Figure 8 A schematic diagram of the structure of a computer system of an electronic device suitable for implementing an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0055] The exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concepts of the exemplary embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their detailed description will be omitted.

[0056] Although relative terms such as "upper" and "lower" are used in this specification to describe the relative relationship of one component of the illustration to another component, these terms are used in this specification only for convenience, such as the direction of the examples described in the drawings. It is understood that if the modules of the illustration are flipped upside down, the component described as "upper" will become the component "lower". Other relative terms, such as "high", "low", "top", "bottom", "left", "right", etc., also have similar meanings. When a structure is "on" other structures, it may mean that the structure is formed integrally on the other structure, or that the structure is "directly" disposed on the other structure, or that the structure is "indirectly" disposed on the other structure through another structure.

[0057] The terms "a", "an", and "said" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising" and "having" are used to indicate an open-ended inclusive meaning and mean that additional elements / components / etc. may be present in addition to the listed elements / components / etc.

[0058] Currently, the network transmission bandwidth in the edge cloud cannot be guaranteed, and the transmission bandwidth between different virtual machines will fluctuate greatly over time. Existing solutions either lack scalability or cannot achieve high performance.

[0059] To solve the above problems, the present invention provides a bandwidth guarantee method for edge cloud, which is used to guarantee the network transmission bandwidth in the edge cloud, thereby achieving good bandwidth isolation, and also has the advantages of fast convergence, improved bandwidth utilization and short flow friendliness.

[0060] Figure 1 The present invention schematically illustrates a bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention, which can be applied to a system including at least one transmitting end, one switch and one receiving end. The method provided by the embodiment of the present invention can be executed by any electronic device with computer processing capability, such as a terminal device and / or a server. Figure 1 The edge cloud bandwidth guarantee method is applicable to bandwidth guarantee of data transmission between different terminals in the edge cloud, and may include the following steps:

[0061] Step S102, the sending end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine respectively according to the bandwidth weights of the first virtual machine and the second virtual machine;

[0062] Step S104, the sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting value marking to the switch;

[0063] Step S106: When the sending end receives the data packet fed back by the receiving end, the sending end adjusts the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end.

[0064] In the technical solution of the embodiment of the present invention, a distributed marking method is used to simulate the weighted fair queue (Weighted Fair Queuing, WFQ) at the virtual machine level in the network. The order in which data packets from different virtual machines are scheduled in the switch will be consistent with the WFQ algorithm that enables a separate physical queue for each virtual machine, thereby achieving good bandwidth isolation and having the advantages of fast convergence, improved bandwidth utilization, and being friendly to short flows.

[0065] In step S102, the transmitting end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine respectively according to the bandwidth weights of the first virtual machine and the second virtual machine.

[0066] In this step, the terminal device and / or server at the sending end respectively sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine. In one embodiment, the first virtual machine and the second virtual machine may be located at the sending end, and the receiving end has the third virtual machine and the fourth virtual machine for receiving data corresponding to the first virtual machine and the second virtual machine, that is, the first virtual machine at the sending end sends data to the third virtual machine at the receiving end through the switch, and the second virtual machine at the sending end sends data to the fourth virtual machine at the receiving end through the switch.

[0067] In one embodiment, the sending end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, respectively, including:

[0068] The transmitting end marks, according to the bandwidth weights of the first virtual machine and the second virtual machine, the data packets sent by the first virtual machine and the second virtual machine as the sending time required for each data packet to be normally sent according to the bandwidth allocated to the first virtual machine and the second virtual machine; or

[0069] The sending end marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, and sorts them according to the length of time required for each data packet to be sent normally according to the bandwidth allocated to the first virtual machine and the second virtual machine, that is, the shorter the required sending time, the earlier the sequence number.

[0070] In one embodiment, at the sending end, the Hypervisor (virtual machine monitor, In the present invention, a virtual machine monitor (VMM) is used for developing a virtual machine monitor. At the transmitting end, a data packet sent by a first virtual machine is marked according to the sending time required for normal sending of the bandwidth pre-allocated to the first virtual machine. For example, if the time required for sending the first data packet of the first virtual machine according to the allocated bandwidth (for example, 20 megabytes) is 0.5 seconds, the first data of the first virtual machine is marked as 0.5, and the time required for sending the second data packet of the first virtual machine according to the allocated bandwidth (for example, 20 megabytes) is also 0.5 seconds, the first data of the first virtual machine is marked as 1 (0.5+0.5), and the same applies to other data packets of the first virtual machine. Similarly, at the transmitting end, a data packet sent by a second virtual machine is marked according to the sending time required for normal sending of the bandwidth pre-allocated to the second virtual machine. For example, if the time required for sending the first data packet of the second virtual machine according to the allocated bandwidth (for example, 10 megabytes) is 1 second, the first data of the first virtual machine is marked as 1, and the time required for sending the second data packet of the second virtual machine according to the allocated bandwidth is also 1 second, the second data of the second virtual machine is marked as 2 (1+1), and the same applies to other data packets of the second virtual machine. In one embodiment, the sequence number can also be queued according to the length of time. For example, when the first data packet of the first virtual machine needs 0.5 seconds to send according to the allocated bandwidth (for example, 20 megabytes), the second data packet of the first virtual machine needs 1 second, and the third data packet of the first virtual machine needs 1.5 seconds, the first data packet of the second virtual machine needs 1 second to send according to the allocated bandwidth (for example, 10 megabytes), and the second data packet of the second virtual machine needs 2 seconds to send according to the allocated bandwidth, the first data packet of the first virtual machine can be marked as 1, the second data packet of the first virtual machine can be marked as 2, the first data packet of the second virtual machine can be marked as 2, the third data packet of the first virtual machine can be marked as 3, and the second data packet of the second virtual machine can be marked as 4. The purpose of the sorting numerical marking of the present application is to reflect the amount of data sent according to the bandwidth weight allocated to different virtual machines by marking the data packets. The specific marking method can take a variety of forms, as long as the bandwidth weight allocated to the virtual machine can be reflected when the data packets are queued and sent.

[0071] In step S104, the sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting value marking to the switch.

[0072] In this step, the terminal device and / or server of the sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting value marking to the switch.

[0073] In step S106, when the sending end receives the data packet fed back by the receiving end, the sending end adjusts the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end.

[0074] In this step, when the terminal device and / or server at the sending end receives the data packet fed back by the receiving end, it adjusts the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end.

[0075] In one embodiment, when the sending end receives the data packet fed back by the receiving end, adjusting the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end includes:

[0076] When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the correct sorting value tag value from the data packet fed back by the receiving end;

[0077] The sending end adjusts the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to extracting the correct sorting value mark value from the data packet fed back by the receiving end.

[0078] In one embodiment, the method further comprises:

[0079] The congestion time of the switch recorded by the sending end is encapsulated into the data packets sent by the first virtual machine and the second virtual machine.

[0080] In one embodiment, the method further comprises:

[0081] When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the latest congestion time of the switch from the data packet fed back by the receiving end;

[0082] The sending end encapsulates the latest congestion time of the switch into a data packet sent by the first virtual machine or the second virtual machine.

[0083] Figure 2 The present invention schematically illustrates a bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention, which can be applied to a system including at least one transmitting end, one switch and one receiving end. The method provided by the embodiment of the present invention can be executed by any electronic device with computer processing capability, such as a terminal device and / or a server. Figure 2 The edge cloud bandwidth guarantee method is applicable to bandwidth guarantee of data transmission between different terminals in the edge cloud, and may include the following steps:

[0084] Step S202, the switch receives data packets of the first virtual machine and the second virtual machine that have been marked with sorting values ​​and sent by the sending end;

[0085] Step S204, the switch sorts the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking according to the sorting numerical markings, and sends them to the receiving end in sequence according to the sorting;

[0086] Step S206: When the sorting value mark of the data packet of the first virtual machine or the second virtual machine is less than or equal to the maximum sorting value mark of the data packets of the first virtual machine or the second virtual machine that have been sent and stored in the switch, the switch marks the corresponding data packet.

[0087] In one embodiment, the method further comprises:

[0088] When the congestion time of the switch encapsulated in the data packet of the first virtual machine or the second virtual machine is inconsistent with the latest congestion time stored in the switch, the switch marks the corresponding data packet.

[0089] Figure 3 The present invention schematically illustrates a bandwidth guarantee method for edge cloud according to an exemplary embodiment of the present invention, which can be applied to a system including at least one transmitting end, one switch and one receiving end. The method provided by the embodiment of the present invention can be executed by any electronic device with computer processing capability, such as a terminal device and / or a server. Figure 3 The edge cloud bandwidth guarantee method is applicable to bandwidth guarantee of data transmission between different terminals in the edge cloud, and may include the following steps:

[0090] Step S302, the receiving end receives data packets of the first virtual machine and the second virtual machine sent by the switch;

[0091] Step S304, the receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch;

[0092] Step S306: the receiving end sends the feedback data packet to the sending end through the switch.

[0093] In one embodiment, the receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including:

[0094] The receiving end obtains the correct sorting value mark of the data packet from the data packet marked by the switch, and writes the correct sorting value mark of the data packet into the feedback data packet.

[0095] In one embodiment, the receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including:

[0096] The receiving end obtains the latest congestion time of the switch in the data packet marked by the switch, and writes the latest congestion time of the switch into the feedback data packet.

[0097] Figure 4 The classification of cloud computing network performance isolation methods in the prior art is shown.

[0098] like Figure 4 As shown in the figure, the existing cloud computing network performance (such as bandwidth) isolation methods can be divided into centralized and distributed methods according to the location of decision-making. The centralized method uses a centralized controller to monitor global network information and then make bandwidth allocation decisions; while the distributed method uses the distributed speed limiter to perceive the real-time status of the network and then autonomously divide the bandwidth. Furthermore, in the centralized method, the static division method remains relatively stable in the gap between two controller decisions, while the dynamic allocation method will also change in real time after the controller decision is completed; in the distributed solution, the best-effort bandwidth allocation method maintains fairness between virtual machines as much as possible, while the explicit bandwidth guarantee method determines the specific value of the bandwidth guarantee and provides stable bandwidth guarantee.

[0099] The centralized method directly monitors the resources of the entire network through a controller, and then accurately issues bandwidth allocation instructions to guide network devices. This method is fast and accurate; however, as the network scale gradually expands, global real-time monitoring becomes difficult to implement, and the centralized method faces the problem of poor scalability.

[0100] The distributed method deploys a distributed rate limiter in the Hypervisor to sense the real-time status of the network, and then gradually converges to fair bandwidth allocation according to a certain rate adjustment method. This method has good scalability, but due to its gradual trial characteristics, it is easy to cause performance fluctuations.

[0101] One method of directly providing virtual machine-level traffic control is to directly configure a static speed limit queue in the virtual machine monitor (Hypervisor), but this method cannot utilize idle bandwidth and easily leads to bandwidth waste. In order to solve this problem and provide a certain degree of flexibility in bandwidth allocation, designers have explored the use of centralized solutions to directly divide bandwidth between different virtual machines. In the centralized method, a centralized controller monitors the bandwidth usage of virtual machines in the entire network, and limits the speed of overspeeding virtual machines accordingly when congestion occurs. However, this method will encounter scalability problems in the current ultra-large-scale data centers, and it is difficult for centralized controllers to monitor the network usage of virtual machines in the entire network in real time. Therefore, more and more research has turned to the use of distributed speed limit mechanisms to solve the disordered congestion between virtual machines. However, the existing distributed solutions basically follow the idea of ​​TCP (Transmission Control Protocol) to slowly and gradually test the available bandwidth. Such methods also have the problems of slow convergence and unstable performance.

[0102] In order to avoid the difficult choice between scalability (host-side based method) and high performance (network device based method), the present invention designs a hybrid method that combines network edge marking and switch scheduling - SLIT (core-Stateless network performance Isolation). The core idea of ​​this method is to simulate the weighted fair queue (WFQ) at the virtual machine level in the network by combining a distributed marking and switch-side scheduling. The order in which data packets from different virtual machines are scheduled (sent) will be consistent with the WFQ algorithm that enables the allocation of a separate physical queue to each virtual machine.

[0103] Figure 5 A system framework diagram of an embodiment of the present disclosure is shown. Figure 5 In the specification, VM (Virtual Machine) refers to a virtual machine.

[0104] The system framework of SLIT is as follows Figure 5As shown. The present disclosure develops an intelligent tag module in the Hypervisor, which is responsible for calculating the guaranteed scheduling time GT (Guaranteed scheduling time) of each data packet sent on different switches. In addition, the present disclosure also develops a monitoring and updating module in the switch, which is responsible for detecting abnormal data packets passing through this switch. There is also a label-based sorting module in the switch, which sorts the data packets from small to large based on the GT label of the data packet to help different data flows obtain the required bandwidth. Finally, a feedback module is designed in the receiving end, which is used to continuously collect real-time network information and feed back the information to coordinate different sending ends.

[0105] SLIT uses the INT (in-band network telemetry) function of modern switches to enable the terminal host to perceive the real-time queuing situation in the network (such as switches). In the process of data packets propagating from the sender to the receiver, each switch on the transmission path uses the INT function to insert some metadata to display the current network information in this switch. The metadata in SLIT includes event type (Type), congestion time (CT), guaranteed time (GT) and real-time queue length (qLen).

[0106] 1. Sending end

[0107] Algorithm 1. Packet marking and policy adjustment at the sender

[0108] 1:procedure Packet_labeling(p f )

[0109] 2:for all switch k in Path(f)do

[0110] 3:

[0111] 4:

[0112] 5: end for

[0113] 6: end procedure

[0114] 7:procedure Baseline_adaptation(feedback f)

[0115] 8:CT k ,GT k=get_label(feedback f)

[0116] 9:

[0117] 10:

[0118] 11: end procedure

[0119] At the sending end, SLIT allocates a dedicated queue for each VM in the Hypervisor, and each data packet transmitted from the upper layer is assigned to a different queue accordingly. The key at the sending end is the intelligent labeling module, which adjusts the label (such as GT) of the data packet it sends according to its own recorded status and real-time feedback from the network. For each VM f (f, flow; that is, a communicating virtual machine pair), SLIT records the latest estimated scheduling time in switch k. The most recent congestion time of switch k And the guaranteed transmission time for each data packet in the network is slack f (p f ). It is worth mentioning that slack f (p f ) represents the data packet p f The guaranteed bandwidth is b f The virtual transmission time is slack f (p f )=p f / b f .

[0120] As shown in Algorithm 1, the functions of SLIT at the sender can be divided into two parts: Packet_labeling and Baseline_adaptation. For Packet_labeling, when it labels a packet p f , it will iteratively calculate the guaranteed scheduling time of the data packet, that is, Next, the Packet_labeling module will also send the latest congestion time CT k Encapsulate it into the data packet to help the switch quickly identify abnormal data packets.

[0121] When scheduling packets for a specific switch, the present disclosure follows a scheduling mode similar to UPS (Universal Packet Scheduling), that is, the required scheduling time is calculated based on the resources previously consumed by itself and its guaranteed bandwidth. However, if the real-time network conditions are ignored, serious unfairness may occur. To overcome this problem, the present disclosure introduces a Baseline_adaptation module to continuously adjust the baseline value of the label. When a special feedback packet is received from the receiving end, the Baseline_adaptation component first extracts the real-time congestion time CT k and the guaranteed dispatch time GT k It will then reset the label baseline value or the value of the congestion time, which allows SLIT to guide the data flow to quickly converge to the latest fairness state (Lines 8-10 of Algorithm 1).

[0122] 2. Switch side

[0123] Algorithm 2. Traffic monitoring and updating module on the switch side

[0124] 1:procedure LB_sorting(packet)

[0125] 2:packet=enqueue(packet)

[0126] 3:

[0127] 4:queue.sort()

[0128] 5:

[0129] 6:modify_packet(event_type,MGT,CT)

[0130] 7:end if

[0131] 8:

[0132] 9:modify_packet(event_type,MGT,CT)

[0133] 10:end if

[0134] 11: end procedure

[0135] 12:procedure Update_MGT()

[0136] 13:packet=dequeue()

[0137] 14:if Congested=True then

[0138] 15:

[0139] 16:

[0140] 17:

[0141] 18:end if

[0142] 19:end if

[0143] 20:end procedure

[0144] 21:procedure Update_CT()

[0145] 22:if Congested==True&CT==0then

[0146] 23:CT=now()

[0147] 24:else if Congested==False then

[0148] 25: CT = 0

[0149] 26:end if

[0150] 27:end procedure

[0151] On the switch side, SLIT schedules packets to achieve fair bandwidth allocation and detects abnormal traffic that violates the cooperation model between VMs. Specifically, when packets enter the switch, they are queued in a simple first-in, first-out (FIFO) queue (which can adjust the order of packets in real time based on the GT value). The switch then sorts these packets from small to large according to their GT tags. In addition, to avoid unfairness issues, the switch needs to find and mark abnormal data traffic (GT or CT anomalies).

[0152] As shown in Algorithm 2, the LB_sorting module is responsible for sorting packets according to their GT tag values ​​(lines 2-4). Then, the most challenging part is to detect abnormal data traffic without retaining the state of each VM. In general, the switch in SLIT needs to save two variables, namely MGT (the maximum GT sent stored by the switch) and CT. The switch compares the label of the incoming packet with these two variables to detect abnormal packets. The root cause of long-term unfairness is that the new input flow is marked with outdated information, which interferes with its cooperation with other flows in the same switch. If the tag value of the first packet of a new input flow is the same as the packet with the smallest GT at the head of the queue, and the tag values ​​of the subsequent packets of the flow increase sequentially according to the guaranteed scheduling time, the new input flow will maintain the same scheduling order with WFQ. Therefore, what the switch needs to do is to detect the emergence of new flows and instruct the corresponding terminal hosts to adjust their tag baseline values.

[0153] Specifically, the present invention uses the monotonically increasing characteristic of labels to find abnormal traffic from a bunch of data packets. The principle is: if a flow already has data packets waiting in the switch, the guaranteed scheduling time of the new input data packets belonging to the flow must be greater than packets from the same flow that are already queued in the queue. As a result, packets from any existing flow will always have a greater guaranteed scheduling time (MGT) than the packet at the head of the queue. If a packet happens to be marked as less than the MGT, then the packet must belong to a new flow that has no knowledge of the queue (lines 5-7). In this case, the switch marks the packet with an event type, modifies the packet’s field, and schedule this packet immediately to notify the end host as soon as possible. Therefore, the switch should record the latest MGT and continuously update the MGT value when sending packets from the port. As shown in Algorithm 2, the Update_MGT module continuously updates the maximum guaranteed scheduling time MGT, which records the latest MGT since the last congestion time for each switch port (lines 12-20).

[0154] However, there are still cases where packets are incorrectly marked due to outdated reference values. To solve this problem, the present disclosure can retain another state at the sender, namely, the time (CT) when the switch port changes from an uncongested state to a congested state, to help the switch identify new input data flows. The principle is very simple: if the sender retains an incorrect CT for switch k, k , the sender is likely to be unaware of the current network queuing status. Therefore, congestion time checking is introduced into SLIT. (the congestion time recorded by the sender) is also encapsulated in the packet. When the switch receives the packet, it extracts and compares it with the variable CT recorded by the switch itself. Unlike CT, it means that the sender is not aware of the queuing information in this switch (lines 8-10). In this case, the switch will modify the event type and CT fields in the packet. Similarly, the Update_CT component in the switch will also continuously update the time when the switch port state changes from non-congested to congested state (lines 21-27).

[0155] 3. Receiving end

[0156] Algorithm 3. Generate feedback packet at the receiving end

[0157] 1: procedure Update_SW(p f )

[0158] 2:for all switch k in Path(f)do

[0159] 3:

[0160] 4:

[0161] 5:

[0162] 6:end if

[0163] 7:

[0164] 8:

[0165] 9:end if

[0166] 10: end for

[0167] 11: end procedure

[0168] 12:procedureGenerate_feedback(p f )

[0169] 13:for all switch k in Path(f)do

[0170] 14:

[0171] 15:

[0172] 16:

[0173] 17:

[0174] 18:end if

[0175] 19:

[0176] 20:

[0177] twenty one:

[0178] 22:end if

[0179] 23: end for

[0180] 24:output_feedback(feedback f )

[0181] 25:end procedure

[0182] At the receiving end, SLIT receives packets from the network and collects information to perceive real-time queuing information. In addition, if a special packet marked by the switch (such as a packet with modified GT or CT) arrives, the SLIT module at the receiving end needs to record this information and coordinate the various senders to make adjustments based on the real-time network conditions. As shown in Algorithm 3, the Update_SW (switch) module intercepts all packets from the same server. After that, Update_SW tries to update the congestion time RCT of each switch recorded. k (R, receiver) and the recorded guaranteed scheduling time RGT k Specifically, if the received CT k f The congestion time RCT of the label greater than the existing record k , then SLIT will be RCT k Update to the latest value (lines 4-6). Similarly, SLIT updates its recorded guaranteed scheduling time RGT in a similar way k (Lines 7-9). In this way, SLIT continuously stores the latest information in the network.

[0183] Finally, the Generate_feedback module coordinates the marking behavior of the sender by sending feedback packets to different senders. When a special data packet reflecting a violation of fairness in the switch is received, SLIT uses the Update_SW module to collect the latest information and encapsulate the correct label value to guide the corresponding sender to adjust its own behavior. Specifically, if the event type marked in the data packet is 0x01, it means that this data packet has encountered a congestion time violation in switch k. Then, SLIT will encapsulate the latest label value in the feedback data packet. It is worth mentioning that in general, the sender will use the same network knowledge to mark a group of data packets. This means that if the sender has outdated network knowledge, it will incorrectly mark multiple data packets, and these data packets will continue to encounter fairness violations in the switch. As a result, these data packets may trigger the receiving end to continuously feedback the same network event that violates fairness, but this is unnecessary and will waste a lot of bandwidth. In order to solve the problem of blind triggering, the present disclosure adds an additional threshold To retain the label value of the last congestion time violation. The feedback from the receiver is triggered only when a new congestion time violation event occurs (lines 15-18). Similarly, the guaranteed time violation event is defined as 0x02, and the receiver will also encapsulate the correct guaranteed time into the feedback packet. A group of packets that have been sent but not yet confirmed (in-flight packets) may also encounter consecutive violations of the guaranteed time, which will also cause blind triggering of feedback. To overcome this problem, SLIT uses additional thresholds And only if the label of the incoming packet is greater than It is worth noting that the label values ​​of these data packets increase linearly, so the present disclosure also adds the deviation value Δ G Add to RGT k To avoid unnecessary feedback triggering (line 20), the present disclosure also uses qLen to estimate the appropriate Δ G value.

[0184] It is worth noting that the tagging module does not need to have accurate queueing information in the switch, and it can automatically correct for minor errors in the estimation. Imagine that due to an inaccurate estimation, the label of a flow is mistakenly reduced by 1. In this way, all packets from this flow have a higher priority than they should have and can be scheduled one packet earlier than they would have been. After the first packet is scheduled, all remaining packets from this flow will be queued in the correct position with other flows. Therefore, minor errors in the estimation only matter for a short time, and eventually all flows will converge automatically.

[0185] 4. Data Packet Format

[0186] Figure 6 The data packet format of one embodiment of the present disclosure is shown.

[0187] SLIT relies on data packets to carry bandwidth allocation instructions, so it is important to specify a packet format to carry this information. Figure 6 As shown in Figure 1, SLIT inserts metadata into the packet header of the Virtual Extensible Local Area Network (VxLAN). VxLAN with Generic Protocol Extension (VxLAN-GPE) already supports network devices to insert metadata into VxLAN packets. Figure 6 In the SWID field, the Type field reflects the type of network event encountered by the data packet when entering the switch. The present disclosure summarizes all event types in Table 1 below. The sender and receiver initialize the data packet to 0x00 and 0x03 respectively. The SWID field represents the unique switch ID in the entire cloud data center. The 14-bit field used in the present disclosure can accommodate up to 2 14 = 16438 switches. The CT field is 24 bits, consistent with the timestamp specification of the current INT standard. It carries the sender's estimated latest congestion time and can be modified by the switch based on real-time conditions. Similarly, the GT field is also 24 bits and specifies the maximum guaranteed scheduling time for each congestion point on the path as the priority for packet sorting. Finally, the qLen field is initialized to 0 by the sender and modified by the switches on the path. This field records the real-time queue length within the switch.

[0188] Table 1 Network event types in INT metadata

[0189]

[0190] Compared with the prior art, the present invention has the advantages and effects.

[0191] (1) SLIT is a method to provide VM-level bandwidth isolation in a stateless way in the network core. The stateless network core approach overcomes the queue scarcity problem in commodity switches (usually with one queue) and significantly improves scalability.

[0192] (2) SLIT deploys a network-aware label adjustment algorithm on the host side and an abnormal packet detection mechanism on the switch side. Through the cooperation between the host and the switch, SLIT can quickly perceive the real-time status of the network and respond quickly.

[0193] (3) SLIT can achieve good bandwidth isolation, and also has the advantages of fast convergence, improved bandwidth utilization, and being friendly to short flows.

[0194] The following introduces an embodiment of the device disclosed in the present invention, which can be used to execute the bandwidth guarantee method of the edge cloud described above in the present invention.

[0195] Figure 7 A bandwidth guarantee device 700 for an edge cloud provided according to an embodiment of the present invention is shown.

[0196] like Figure 7 As shown, a bandwidth guarantee device 700 for edge cloud provided according to an embodiment of the present invention is applied to a system including at least one sending end, one switch and one receiving end, and may include:

[0197] The marking module 710 is configured to sort and numerically mark the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine;

[0198] A sending module 720, configured to send the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking to the switch;

[0199] The adjustment module 730 is configured to adjust the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end when the data packet fed back by the receiving end is received.

[0200] Since the various functional modules of the bandwidth assurance device for edge cloud in the exemplary embodiment of the present invention correspond to the steps of the exemplary embodiment of the bandwidth assurance method for edge cloud described above, for details not disclosed in the embodiment of the device of the present invention, please refer to the embodiment of the bandwidth assurance method for edge cloud described above.

[0201] The edge cloud bandwidth guarantee device provided in the embodiment of the present invention can achieve good bandwidth isolation, and also has the advantages of fast convergence, improved bandwidth utilization, and short flow friendliness.

[0202] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of an electronic device suitable for implementing an embodiment of the present invention. Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0203] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0204] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0205] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present application are executed.

[0206] It should be noted that the computer-readable storage medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0207] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0208] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0209] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the edge cloud bandwidth guarantee method described in the above embodiment.

[0210] For example, the electronic device can implement Figure 1-Figure 3 The steps shown in .

[0211] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0212] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0213] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0214] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A bandwidth guarantee method for edge cloud, applied to a system including at least one sending end, one switch and one receiving end, characterized in that: include: The transmitting end respectively sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine; The sending end sends the data packets of the first virtual machine and the second virtual machine that have completed the sorting value marking to the switch; When the sending end receives the data packet fed back by the receiving end, the sending end adjusts the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end; The sending end sorts and numerically marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, including: the sending end sorts and marks the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine according to the sending time required for each data packet to be normally sent using the bandwidth allocated to the first virtual machine and the second virtual machine or the length of the sending time, that is, the shorter the required sending time, the earlier the sequence number.

2. The method according to claim 1, characterized in that When the transmitting end receives the data packet fed back by the receiving end, adjusting the sorting value tag of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end includes: When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the correct sorting value tag value from the data packet fed back by the receiving end; The sending end adjusts the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to extracting the correct sorting value mark value from the data packet fed back by the receiving end.

3. The method according to claim 1, characterized in that Also includes: The sending end encapsulates the congestion time of the switch recorded by the sending end into the data packets sent by the first virtual machine and the second virtual machine.

4. The method according to claim 3, characterized in that Also includes: When the transmitting end receives the data packet fed back by the receiving end, the transmitting end extracts the latest congestion time of the switch from the data packet fed back by the receiving end; The sending end encapsulates the latest congestion time of the switch into a data packet sent by the first virtual machine or the second virtual machine.

5. A bandwidth guarantee method for edge cloud, applied to a system including at least one sending end, one switch and one receiving end, characterized in that: include: The switch receives data packets of the first virtual machine and the second virtual machine that have completed sorting numerical markings and are sent by the sending end; The switch sorts the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking according to the sorting numerical markings, and sends them to the receiving end in sequence according to the sorting; When the sorting value mark of the data packet of the first virtual machine or the second virtual machine is less than or equal to the maximum sorting value mark of the data packet of the first virtual machine or the second virtual machine that has been sent and stored by the switch, the switch marks the corresponding data packet; Among them, the data packets of the first virtual machine and the second virtual machine that have completed sorting and numerical marking are the data packets of the first virtual machine and the second virtual machine obtained by the sending end after marking the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine according to the sending time required for each data packet to be normally sent in the bandwidth allocated to the first virtual machine and the second virtual machine or the length of the sending time, that is, the shorter the required sending time, the earlier the sequence number.

6. The method according to claim 5, characterized in that Also includes: When the congestion time of the switch encapsulated in the data packet of the first virtual machine or the second virtual machine is inconsistent with the latest congestion time stored in the switch, the switch marks the corresponding data packet.

7. A bandwidth guarantee method for edge cloud, applied to a system including at least one sending end, one switch and one receiving end, characterized in that: include: The receiving end receives data packets of the first virtual machine and the second virtual machine sent by the switch; The receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch; The receiving end sends the feedback data packet to the sending end through the switch; The data packets of the first virtual machine and the second virtual machine sent by the switch are obtained by sorting the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine, according to the sending time required for each data packet to be normally sent in the bandwidth allocated to the first virtual machine and the second virtual machine or the length of the sending time, and then sorting them by the switch according to the sorting data value mark, that is, the shorter the required sending time is, the earlier the sequence number is.

8. The method according to claim 7, characterized in that The receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including: The receiving end obtains the correct sorting value mark of the data packet from the data packet marked by the switch, and writes the correct sorting value mark of the data packet into the feedback data packet.

9. The method according to claim 7, characterized in that: The receiving end obtains the data packet marked by the switch and generates a feedback data packet according to the data packet marked by the switch, including: The receiving end obtains the latest congestion time of the switch in the data packet marked by the switch, and writes the latest congestion time of the switch into the feedback data packet.

10. A bandwidth guarantee device for edge cloud, applied to a system including at least one transmitting end, one switch and one receiving end, characterized in that: include: a marking module, configured to sort and numerically mark the data packets sent by the first virtual machine and the second virtual machine respectively according to the bandwidth weights of the first virtual machine and the second virtual machine; A sending module, configured to send the data packets of the first virtual machine and the second virtual machine that have completed sorting numerical marking to the switch; an adjustment module, configured to adjust the sorting value mark of the data packet sent by the first virtual machine or the second virtual machine according to the data packet fed back by the receiving end when receiving the data packet fed back by the receiving end; Among them, the marking module is further configured to mark the data packets sent by the first virtual machine and the second virtual machine according to the bandwidth weights of the first virtual machine and the second virtual machine according to the sending time required for each data packet to be normally sent using the bandwidth allocated to the first virtual machine and the second virtual machine or the length of the sending time, that is, the shorter the required sending time, the earlier the sequence number.

11. An electronic device, characterized in that: include: one or more processors; A storage device configured to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images