A method for resolving message and host cache matching under virtio protocol

Abstract

The present invention provides a method for solving the problem of matching messages with host cache under the virtio protocol, including: step 1, the host cache management module sends a request to the host to obtain host cache information, and the network card caches the host cache information in advance; step 2, the network card completes the processing of the message, sends a message descriptor to the traffic management module, and the traffic management module completes the caching of the message descriptor and performs traffic management; step 3, the traffic management module provides authorization, and the message descriptor is matched with the cached host cache information; step 4, when the message and the host cache information meet the requirements, the traffic management module pulls the actual message based on the message descriptor, and combines the host cache information to correctly transmit the message to the cache address corresponding to the host. After using the method provided by the present invention, the resource consumption of the CPU for matching messages and host cache space in related services can be greatly reduced, thereby saving computing power for processing actual applications.

Description

Technical Field

[0001] The present invention relates to the technical field of network communication, and in particular to a method for solving the matching between a message and a host cache under a virtio protocol. Background Art

[0002] For virtio packets, before the NIC completes processing and sends them to the host, it needs to obtain the host's storage information for the packet in advance. This storage information includes the size of the host cache space, the cache address after the packet is sent, and other information. Then, the packet length of the packet to be sent is compared with the size of the cache space. When in virtio merge buffer mode, a complete datagram can be split into multiple parts and stored in multiple cache spaces respectively; when in virtio normal mode, a complete datagram must be stored in one cache space. After the matching of the packet and the cache space is completed, the NIC sends the packet with the cache address information to the host.

[0003] When receiving a virtio message, the network card, based on the set processing flow, needs to complete the normal message processing flow. In addition, the software needs to pull the host cache information from the host side, complete the matching of the message and the cache, etc., which will consume a lot of processing power of the network card. In addition, in merge buffer mode, after the message or host cache information is read, if it is found that the current cache space is not enough to store the next complete datagram, it is necessary to continue to pull more host cache information from the host side. In the whole process, the order of the message and the coherence of the host cache information need to be maintained. All of the above operations need to consume the kernel resources of the network card, which will affect the processing performance of the network card. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for matching messages with host cache under the virtio protocol in view of the deficiencies of the prior art, including: establishing a host cache management module and a traffic management module;

[0005] The host cache management module is used to send a request to the host to obtain host cache information;

[0006] The traffic management module is used to perform traffic management;

[0007] Then follow these steps:

[0008] Step 1: During the initialization phase of the network card, the host cache management module sends a request to the host to obtain host cache information. Based on the request, the host sends the host cache information to the network card, and the network card caches the information in advance.

[0009] Step 2: After the NIC receives the virtio message, after the kernel completes the message processing, it sends the message descriptor to the traffic management module. The traffic management module completes the caching of the message descriptor and starts traffic management. Traffic management usually includes functions such as scheduler and rate shaping shaper. Rate shaping shaper refers to configuring a fixed rate for the scheduler's export so that the packet rate of this export can be fixed within this configuration value. The scheduler can complete the scheduling of the corresponding port by configuring different scheduling rules (such as sp, dwrr, wfq, etc.) and multi-level scheduling, plus the speed limit of shaper.

[0010] Step 3: The traffic management module provides authorization to the message descriptor of the corresponding queue based on the configured traffic allocation (generally, different messages are allocated to queues based on the vf / pf of the message destination, and then the corresponding priority and bandwidth are configured based on different vf / pf, and finally mapped to the traffic allocation module to realize the traffic allocation for the host side pcie vf / pf), and the message descriptor of the corresponding queue is matched with the host cache information cached in step 1;

[0011] Step 4: When the message and the host cache information meet the requirements of the virtio protocol, the traffic management module pulls the actual message based on the message descriptor, and combines the host cache information to correctly transmit the message to the cache address corresponding to the host.

[0012] In step 1, after the network card is initialized, a queue is established (the queue can be divided based on the final destination of the message. Usually, the message will go to different vf / pf of pcie, and then sent to the host through pcie. If the traffic of the message received by the host is to be distributed, it is usually distributed based on the vf / pf of pcie. Therefore, different queue numbers can be assigned to different vf / pf, and the messages going to the corresponding vf / pf are marked with the corresponding queue number. In addition, queues can be divided according to different message types within the same vf / pf, and these queues can be mounted under the same scheduler. Then the exit of this scheduler corresponds to a certain vf / pf, and then the exits of multiple schedulers are connected to the next-level scheduler, and the traffic distribution between different vf / pfs can be completed. This is what we call mapping and traffic distribution. Finally, no matter how the queues are divided, for a single queue, the first-in-first-out rule must be followed), and the storage space exclusive to each queue and the shared storage space that all queues can use are configured based on the queue.

[0013] In step 1, the virtio mode can be configured based on the queue, and virtio can be configured as virtionormal mode and virtio merge buffer mode (virtio is a virtual interface technology used by the host to receive information. Messages are stored by allocating virtual space to the messages. This patent is mainly responsible for matching actual messages with virtual space. It realizes full utilization of virtual space and messages, and fully fills the corresponding virtual space with the message in a manner that meets the requirements of the virtio protocol. When the host obtains the message, it will extract the corresponding virtual space together, and store it in the actual memory address after conversion. The corresponding mode of Virtio normal1 is that one virtual space is only used to store one message, and multiple virtual spaces are not allowed to store parts of the message separately for splicing; the corresponding mode of Virtio merge buffer mode is that a complete message can be spliced ​​and stored by multiple virtual spaces, and each virtual space can store part of the message);

[0014] When the queue is configured in virtio normal mode, the matching of packets and host cache information is based on the following rules: one cache space of the host cache is used to store one packet. Even if there is redundant space after storing one packet in the host cache, the remaining redundant space will not be used for the second packet. If the host cache space cannot accommodate a complete packet, the network card reports an abnormal interrupt and discards the corresponding packet and host cache information.

[0015] When the queue is configured in virtio merge buffer mode, the following rules are used to match the message and host cache information: Based on the virtio normal mode, when the cache space of a host cannot accommodate a complete message, more than two host caches are allowed to store data messages together. When the cache space of a host cannot accommodate a complete message, the network card no longer reports an exception, but continues to obtain the next host cache information until the next data message can be completely stored.

[0016] In step 1, based on the configured queues and the space allocated to each queue, the hardware automatically initiates a request to the host to obtain the host cache information (the hardware here refers to the chip. There is a module inside the chip. Its function is to pull the corresponding cache information from the host after the queue is configured. If the host already has the prepared cache information, it will be directly sent to the corresponding module in the chip. If not, it will be pulled again after a period of time). After the host sends the corresponding host cache information, it is stored in sequence according to the queue; in addition, the sum of the cache space corresponding to all the current host cache information in the queue must be recorded for matching comparison in virtio merge buffer mode.

[0017] In step 1, the host cache information includes the size of the host cache space and the starting address of the cache space in the host (the host cache space here is a virtual space allocated in the virtio protocol for storing actual messages. The host needs to parse this virtual address to obtain the real storage space and place the data in the message to the corresponding position);

[0018] The host cache space is used to store actual data messages;

[0019] The start address is used to store received messages.

[0020] In step 1, the host cache management module for pulling the host cache manages the host cache information based on the queue. After the network card is initialized, the number of host cache information that can be stored (pre-fetched) in each queue is configured. The host will then periodically send the number of prepared host cache information and the storage address of the host cache information in the host to the host cache management module in the network card; when any of the following conditions is met, the host cache management module will read the host cache information from the host based on the storage address of the host cache information sent by the host:

[0021] Condition 1: The number of host cache information stored in the queue has not reached the maximum value that can be pre-fetched (the maximum value of pre-fetching is related to the depth of the linked list, and is also related to the delay of the host cache module in pulling the host cache information. For example, in the case of continuous host cache information dequeueing, the interval time of each host cache information dequeueing is t, and the time from the host cache management module initiating a request to the host to the host cache information returning is T, then the maximum value that can be pre-fetched here cannot be less than T / t. If n queues are configured, the total depth of the linked list is N, and the number of host cache information that needs to be consumed when the longest packet that the network card can process matches the minimum host cache information is m, then this maximum value cannot exceed (Nn*m) / n), and the host cache information prepared for the queue in the host received by the host cache management module has not been fully pulled, and at this time, a request to pull the host cache information continues to be initiated to the host;

[0022] Condition 2: When the host cache information prepared for the queue in the host received by the host cache management module has not been fully pulled out, and even if the amount of stored host cache information has reached the maximum value that can be pre-fetched, but the sum of the cache space corresponding to the currently cached host cache information is still unable to store the next complete datagram, the host cache management module will continue to initiate a request to pull the host cache information from the host until the sum of the cache space corresponding to the cached host cache information is able to store the next complete datagram.

[0023] In step 1, a linked list module is maintained inside the host cache management module to perform queue management; the linked list module includes two sets of linked lists, one of which is responsible for storing and processing message descriptors, and the other is responsible for storing and processing host cache information.

[0024] In step 1, the linked list stores the message and host cache information by maintaining a head pointer, a tail pointer, a next hop, an idle pointer, a linked list sequence number, and a linked list empty state.

[0025] In step 1, when any of the following three conditions is met, the host cache management module automatically initiates a request to the host to obtain host cache information:

[0026] In the first case, after the queue configuration is completed, the hardware automatically pulls the host cache information until the exclusive storage space of all queues has completed the storage of the host cache information;

[0027] In the second case, when the host cache information stored in a queue is matched with the message and dequeued, if the exclusive storage space of the current queue is idle at this time, the host cache management module will continue to pull new host cache information from the host and queue it to the end of the queue until the exclusive storage space has completed the storage of the host cache information;

[0028] In the third case, for the virtio merge buffer mode, a message cache matching module is established, which matches the message with the virtual cache address one by one, packages them together, and notifies the host to parse and process them;

[0029] If the host cache space corresponding to the host cache information stored in the exclusive storage space cannot store the next complete datagram, the message cache matching module will inform the host cache management module, and the host cache management module will again initiate a request to the host to pull the host cache information. The newly acquired host cache information is stored in the shared storage space and queued at the end of the queue.

[0030] In the second and third cases, joining the team includes the following steps:

[0031] Step 1-1, based on the queue, obtain the queue entry list sequence number;

[0032] Step 1-2, based on the linked list sequence number, operate the corresponding linked list: read the empty state of the linked list, read the free pointer, and read the tail pointer;

[0033] Step 1-3, store the data in the storage address corresponding to the free pointer, and maintain the pointer based on the current state of the linked list: if the current linked list is empty, write the free pointer to the head pointer and the tail pointer at the same time, and update the linked list state to a non-empty state; if the current linked list is not empty, use the tail pointer as the write address, write the free pointer to the next hop table, and use the free pointer to replace the original tail pointer.

[0034] In step 2, the message descriptor records all the information of the message, including the message packet length (used to complete the matching of the message and the host cache information), the storage address of the message in the network card (used to obtain the actual message from the network card after the dequeue is completed) and the queue to which the message belongs; using the message descriptor for queue and traffic management can effectively save the cache space size of the traffic management module.

[0035] In step 2, the traffic management module maintains message descriptors based on queues and uses linked lists to manage queues.

[0036] In step 3, when there is unconsumed host cache information in the queue storing the message, the queue participates in the scheduling of the traffic management module; the traffic management module is based on the service configuration scheduling rules (the service configuration scheduling rules refer to the allocation of traffic bandwidth according to the pcie vf / pf mapping based on the actual service. For example, the traffic priority of video and voice call related services is relatively high and needs to be transmitted as soon as possible; the priority of music video playback is slightly lower), supports SP, DWRR, RR scheduling methods (SP: strict priority, strict priority scheduling; dwrr: deficit weight roundrobin, load weight polling scheduling; rr: round robin, polling scheduling), and also supports adding rate shaping shaper for speed limiting; scheduling is based on the current scheduling result, and outputs dequeue authorization to the corresponding queue.

[0037] In step 3, when the corresponding queue receives the dequeue authorization, the linked list performs the dequeue operation, and after obtaining the message descriptor and host cache information stored in the linked list, the message is matched based on the current virtio mode configuration.

[0038] In step 3, the message matching is performed based on the current virtio mode configuration. If the current queue is configured as the virtio merge buffer mode, the following judgment rules are followed to control the dequeuing of the message and the host cache information:

[0039] Step 3-a1, if the message length recorded in the current message descriptor is less than the sum of the cache spaces corresponding to all host cache information in the current queue, then the host cache information is dequeued and the sum of the cache spaces corresponding to the currently dequeued host cache information is recorded;

[0040] Step 3-a2, when the sum of the cache space corresponding to the current host cache information in the queue and the previously dequeued host cache information is greater than or equal to the packet length of the message in the message descriptor, the message descriptor and the host cache information are dequeued at the same time;

[0041] Step 3-a3, if the message length recorded in the current message descriptor is still greater than the sum of the cache spaces corresponding to all host cache information in the current queue, it means that even if all host cache information in the current queue is dequeued, it cannot support the storage of the current message. At this time, a pull request is sent to the host cache management module, requesting the host cache management module to continue to send pull requests to the host, so that the queue obtains more host cache information.

[0042] The message matching is performed based on the current virtio mode configuration. If the current queue is configured as virtionormal mode, the following judgment rules are followed to control the dequeuing of messages and host cache information:

[0043] Step 3-b1, if the message length recorded in the current message descriptor is less than or equal to the cache space corresponding to the current host cache information, the message descriptor and the host cache information are dequeued at the same time;

[0044] Step 3-b2, if the message length recorded in the current message descriptor is greater than the cache space corresponding to the current host cache information, the cache space sent by the host cannot meet the protocol requirements. The message descriptor and the host cache information are dequeued at the same time, and the packet drop operation is performed, and the exception is reported to the host.

[0045] The message descriptor and the host cache information are dequeued simultaneously, comprising the following steps:

[0046] Step 3-c1, the linked list module receives the authorization to output the dequeue to the corresponding queue (the matching message and the host cache space must have the same queue number. After the match is successful, the queue that is being matched is the one that is being dequeued. The queue here is the minimum granularity that needs to be mapped one-to-one with the actual business as mentioned earlier. According to business requirements, if only traffic management between vf / pf is required, then each vf / pf is assigned an independent queue number; if traffic management between specific businesses is required, then each business type needs to be assigned an independent queue number. The message and host cache space information between different queue numbers are stored using a linked list), the linked list storing the message descriptor and the linked list storing the host cache information simultaneously obtain the current queue head pointer based on the queue number;

[0047] Step 3-c2, based on the current queue head pointer, obtain the storage address pointed to by the head pointer and the next hop pointer information of the current queue, and read the message descriptor and host cache information from the storage address;

[0048] Step 3-c3: When the dequeuing condition is met (i.e., the message matches the host cache space information), the next hop pointer information obtained in step 3-c2 is used as the new head pointer to update the head pointer of the current queue; at the same time, the head pointer that has completed dequeuing is placed in the free pointer pool for use when entering the queue next time;

[0049] Step 3-c4, when the dequeue condition is not met, exit the current dequeue process without rewriting the head pointer of the current queue or recycling the head pointer.

[0050] The linked list storing the host cache information and the linked list storing the message descriptor, because the host cache information and the message descriptor of the same queue are continuously dequeued, in order to absorb the delay consumed by step 3-c1 to step 3-c2, by using more than two linked lists, it is ensured that after the linked list completes step 3-c1 to step 3-c2, the host cache information and the message descriptor that are dequeued are continuous according to the switching order of the linked list, which specifically includes the following steps:

[0051] Step 3-d1, after the linked list module receives the authorization of the scheduling output (scheduling authorization means that the traffic management module divides a variety of queues according to the business type, and the order of packet delivery between different queues is different. Which queue is dequeued first and which queue is dequeued later is based on the scheduling rules and is related to the needs of actual application; when the scheduling module confirms which queue can be dequeued, it will send the queue number of the queue that is about to be dequeued to the linked list, and the linked list completes the pre-dequeue of this queue based on the queue number), based on the authorized queue, obtain the dequeuing linked list sequence number;

[0052] Step 3-d2, based on the sequence number of the dequeued linked list, operate the corresponding linked list and read the head pointer of the linked list;

[0053] Step 3-d3, using the head pointer as the address, read the stored message descriptor and host cache information, and read the corresponding next hop information;

[0054] Step 3-d4, the read message descriptor and host cache information are matched with the virtio protocol, and based on the matching result, the output of the message descriptor and host cache is completed; when the output is confirmed, the current head pointer is recycled, and the next hop information is used as the read address to obtain the new message descriptor, host cache information and next hop information;

[0055] Step 3-d5, based on the delay of step 3-d2 to step 3-d4, absorbs the delay of reading table entry information and protocol matching through cyclic switching of more than two linked lists under the same queue, so that each beat can be output without flow control and sufficient storage message descriptor and host cache information, thereby ensuring a certain acceleration ratio to absorb the delay during the dequeue operation.

[0056] In step 4, the traffic management module pulls the actual message in sequence according to the corresponding byte order based on the message storage address in the network card recorded in the message descriptor, and outputs it to the subsequent message splicing module; when the host cache information is dequeued, it is directly output to the subsequent message splicing module.

[0057] In step 4, the message splicing module splices the actual message with the host cache information, and finally sends it to the host side.

[0058] In this method, in order to ensure that the virtio merge buffer mode can be supported, each queue needs to maintain the sum of the cache space corresponding to all host cache information in the current queue when storing the host cache. When matching, the packet length recorded in the current packet descriptor and the sum of the cache space corresponding to all host cache information in the current queue are prioritized. Only when the sum of the cache space is greater than the packet length, the packet descriptor and the host cache information can be dequeued.

[0059] The present invention relates to a network transmission process in which, when a network card processes a message of a virtio protocol, it is necessary to complete the matching work between the message and the cache space in the host at the message receiving end. Through a specific processing structure, with as little software intervention as possible, after the software completes the message processing, the host cache information acquisition, message traffic management, and message and host cache matching are completed in the traffic management module, and after the processing is completed, the message is correctly transmitted to the cache space corresponding to the host.

[0060] The present invention has the following beneficial effects: after using the method provided by the present invention, the resource consumption of the CPU for matching messages and host cache space in related services can be greatly reduced, thereby saving computing power for processing practical applications. At the same time, the method can complete the continuous output of messages without matching errors through a multi-linked list, thereby absorbing the bandwidth loss caused by queue switching, matching failure, etc., and finally meet the performance requirements of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0062] Figure 1 It is a flow chart of the method of the present invention.

[0063] Figure 2 This is a schematic diagram of multi-linked list operations.

[0064] Figure 3 This is a schematic diagram of the multi-link list enqueue state machine.

[0065] Figure 4 This is a schematic diagram of the multi-link list dequeue state machine.

[0066] Figure 5 This is a schematic diagram of the matching mechanism between messages and cache space. DETAILED DESCRIPTION

[0067] like Figure 1 As shown, this embodiment provides a method for solving the matching between messages and host cache under the virtio protocol, including: establishing a host cache management module and a traffic management module;

[0068] The host cache management module is used to send a request to the host to obtain host cache information;

[0069] The traffic management module is used to perform traffic management;

[0070] Then follow these steps:

[0071] Step 1: The host cache management module pulls the host cache information from the host in advance based on the queue status. The host cache information includes the cache address, cache size, etc. This information is usually stored in the host. When the host receives the message, the CPU matches it based on the packet length information of the received message, and finally stores the message in the corresponding host physical address. Now the host only needs to send the host cache information in the queue directly to the network card after receiving the request from the network card based on the queue, and the subsequent matching work will be completed by the network card.

[0072] The cache address is used to directly store the message received by the host into the corresponding physical address of the host. Or, for information security considerations, if the cache address is a virtual address segment, the host will store the message into the corresponding physical address of the host after decoding.

[0073] The cache size corresponds to the size of the actual physical storage space allocated by the host for storing messages, and is used to match with the message, so as to ensure that the message can be stored in the correct cache address in accordance with the protocol requirements.

[0074] Step 2, as Figure 2 shown, after the host cache management module successfully pulls the cache information from the host, it outputs to the multi-linked list for storing the host cache information, and the multi-linked list enqueue state machine for storing the host cache information starts to work. The multi-linked list enqueue state machine is as Figure 3 shown;

[0075] Based on the number of linked lists N in the multi-linked list for storing the host cache information, each queue maintains a counter 0 ≤ n < N, and the sequence number n is 0 initially.

[0076] The linked list sequence number is maintained based on the queue respectively, and the sequence numbers used for enqueueing and dequeueing different queues are independent. That is, queue 0 may have enqueued 3 packets, so that its corresponding sequence number n0 switches to 3; while queue 1 may have only enqueued 2 packets, then the sequence number n1 corresponding to queue 1 only switches to 2.

[0077] The multi-linked list enqueue state machine is initially in a sleeping state and waits to be woken up after receiving a request for enqueueing. Based on the current multi-linked list sequence number n, perform subsequent enqueue operations on the corresponding linked list. Obtain the tail pointer, free pointer, and empty / full status of the linked list from the multi-linked list control information.

[0078] Among them, the tail pointer corresponds to the physical address where the content enqueued last time in this linked list is stored;

[0079] The free pointer corresponds to the physical address where the content to be enqueued currently will be stored. The multi-linked list maintains a free pointer pool. When there is new content to be enqueued, a free pointer will be extracted from the free pointer pool. When the content stored in the linked list is dequeued, the free pointer corresponding to the physical address where the content is stored will be recycled;

[0080] The empty / full status is used to identify whether there is content that has not been dequeued in the current linked list.

[0081] Based on the empty / full status of the current queue, store the content to be enqueued and update the multi-linked list control information.

[0082] If the queue status is empty (there is no content in the current linked list that has not been dequeued yet): the content to be enqueued this time is stored at the physical address pointed to by the free pointer; the free pointer is used to replace the tail pointer and the head pointer of the current linked list; the empty / full flag of the current linked list is set to the non-empty state.

[0083] If the queue status is non-empty (there is content in the current linked list that has not been dequeued yet): the content to be enqueued this time is stored at the physical address pointed to by the free pointer; the free pointer is stored in the next-hop directory at the address of the current tail pointer of the linked list; the free pointer is used to replace the tail pointer of the current linked list;

[0084] The next-hop directory here is a storage space with the same depth as the physical space for storing the enqueued content, and the next-hop address is stored inside. For example, if content A and B are enqueued successively, the first enqueued content A is stored at physical address a, and the second enqueued content B is stored at physical address b. Then, at address a in the next-hop directory, the value stored is b. In this way, after content A is dequeued, based on the value b stored at address a in the next-hop directory, it can be known that the next content to be dequeued is stored at physical address b.

[0085] After the content to be enqueued this time is successfully enqueued and the control information of the linked list is also updated, the linked list sequence number is updated and switched to the next linked list. If n = N - 1 at this time, then n = 0; if n < N - 1 at this time, then n = n + 1.

[0086] The multi-linked list for storing host cache information will calculate the total sum of the cache sizes of all the cache information that has not been dequeued inside all the multi-linked lists in the current queue, mainly used to pre-identify whether the host cache information already stored in the multi-linked lists in the current queue can be matched when dealing with long packets. If the total sum of the cache sizes of the host cache information already stored in the multi-linked lists in the current queue cannot wrap a complete long packet, at this time, the host cache management module needs to send a pull request for new host cache information to the host in advance.

[0087] Step 3, after the network card kernel finishes processing the packet, the packet descriptor containing information such as the packet length and the storage address of the packet inside the network card is output to the multi-linked list for storing packet descriptors to complete the enqueue operation of the packet descriptor;

[0088] The packet length information is mainly used to calculate the bandwidth consumed by the current packet during traffic bandwidth allocation and for comparison when matching with host cache information.

[0089] The storage address of the packet inside the network card is used to extract the actual packet from the physical storage of the network card and send it to the host after the packet descriptor is dequeued.

[0090] The reason for using the message descriptor for enqueuing is that directly storing the message descriptor in the linked list requires much less storage space than storing the complete message. The steps for enqueuing the message descriptor in the multi-linked list are exactly the same as the host cache information enqueuing in step 2.

[0091] Step 4: When both the host cache information and the message descriptor multi-link list in the same queue have content that has not been dequeued, the queue participates in traffic bandwidth allocation, that is, traffic bandwidth allocation may grant the queue dequeue authorization after scheduling.

[0092] Step 5: The traffic bandwidth allocation module allocates traffic based on queues, where there is a certain mapping relationship between queues and related services of the host, and can be flexibly changed based on actual needs. Common scheduling rules include strict priority-based order preservation, which is usually applied to services with obvious priority requirements, such as live video, etc.; or equal polling rules with configurable weights.

[0093] Based on the scheduling results of traffic management, the traffic bandwidth allocation module issues dequeue authorizations to the multi-linked list storing host cache information and the multi-linked list storing message descriptors, respectively, and performs pre-dequeue on the corresponding multi-linked list;

[0094] Step 6, such as Figure 4 As shown in the upper part, the pre-dequeue operation only reads the head pointer, next hop information, and storage content, and sends the obtained host cache information and message descriptor to the matching module for comparison;

[0095] The queue also maintains the linked list sequence number m, which is used to identify which linked list is about to be dequeued. The initial value of the sequence number m is 0, 0≤m <N;

[0096] The dequeue state machine is also initially in a sleep state, and wakes up after receiving a dequeue authorization port to start subsequent dequeue work.

[0097] Based on the current linked list sequence number m, read the head pointer and tail pointer of the linked list corresponding to the current sequence number from the linked list control information. The head pointer points to the location where the content that needs to be dequeued is stored; the tail pointer is used to compare with the head pointer to determine whether the content currently dequeued is the last one in the current linked list. In the next clock cycle, continue to read the head pointer and tail pointer corresponding to the linked list with the linked list sequence number m'=m+1, and in each subsequent clock cycle, continue to read the head pointer and tail pointer corresponding to the linked list with m'=m'+1; until the head pointer and tail pointer of the linked list corresponding to m'=m=m-1 are also read.

[0098] Use the head pointer corresponding to linked list m as the address, and read the next hop directory mentioned in step 2 from the linked list control information. Get the address where the next content to be dequeued is stored after the content pointed to by the current head pointer is dequeued. Then use the head pointer corresponding to linked list m as the address to read the specific content that is about to be dequeued pointed to by the head pointer. As in the previous paragraph, in the next cycle, continue to read the next hop directory and dequeued content corresponding to the linked list with linked list sequence number m'=m+1, and in each subsequent clock cycle, continue to read the next hop directory and dequeued content corresponding to the linked list with m'=m'+1; until the next hop directory and dequeued content of the linked list corresponding to m'=m=m-1 are also read.

[0099] When the next hop directory and the dequeuing content corresponding to the linked list with sequence number m are returned, the pre-dequeuing is completed. At this time, the dequeuing content read out is uniformly sent to the post-level cache message matching module for matching processing.

[0100] After the next hop directory and the dequeued content corresponding to the linked list with the subsequent sequence number m' are returned, they are uniformly cached in the post-cache message matching module according to their return order. After the content of the previous cache is matched and the real authorization is obtained, it immediately participates in the next match, thereby ensuring the continuity of matching and real authorization.

[0101] Step 7, such as Figure 5 As shown, the cache message matching module completes the matching of the message descriptor and the host cache information based on the virtio mode configured by the current queue;

[0102] The inputs of the cache message matching module are the packet length data i carried in the message descriptor that is about to be dequeued, the cache space size j carried in the host cache information, and the sum k of the cache sizes in the cache information that has not yet been dequeued in all multi-linked lists in the current queue mentioned in step 2.

[0103] First, determine the size of i and j. If i≤j at this time, it means that the host internal cache space corresponding to the host cache information to be dequeued can fully accommodate the message to be dequeued. In this case, a real authorization is issued to the host cache information multi-linked list and the message descriptor multi-linked list at the same time, notifying the linked list dequeuing state machine to perform subsequent real dequeuing work. At the same time, the next set of message descriptors and host cache information cached in the cache message matching module mentioned in step 6 are used for new matching.

[0104] If i>j at this time, it means that the host internal cache space corresponding to the host cache information that is about to be dequeued cannot accommodate the message that is about to be dequeued. In this case, it is necessary to determine the virtio mode of the current queue configuration.

[0105] If the mode of the current queue is not merge buffer (allowing a complete message to be split into multiple fragments and stored in different host cache spaces), the message that is about to be dequeued fails to match the host cache information and cannot be spliced ​​through multiple host cache information. In this case, the network card will report an interrupt to notify the host that this queue has encountered a problem and needs to check whether the software configuration and message are correct. At the same time, clear all the remaining host cache information and messages corresponding to the message descriptor in the current queue. End the cache and message matching, and notify the multi-linked list to resume the sleep state.

[0106] If the current queue mode is merge buffer, the size of i and k is determined again. If i≤k, it means that the host internal physical space corresponding to the host cache information stored in the multi-linked list can store the message corresponding to the message descriptor that is about to be dequeued. In this case, the cache message matching module first issues a real authorization to the multi-linked list storing the host cache, updates the residual message packet length i'=ij, and extracts the next host cache information obtained in advance in step 6, takes its cache size value j', and matches again. Repeat this process until the matching result i' is less than or equal to j', indicating that the message corresponding to the message descriptor can be cached by the host internal physical space corresponding to the host cache information that has been dequeued and is about to be dequeued. At this time, the cache message matching module simultaneously issues a real authorization to the host cache information multi-linked list and the message descriptor multi-linked list, notifying the linked list dequeuing state machine to perform subsequent real dequeuing work. At the same time, the next set of message descriptors and host cache information cached in the cache message matching module mentioned in step 6 are used for new matching.

[0107] If i>k, it means that the host cache information stored in the multi-linked list is unable to store the message corresponding to the message descriptor that is about to be dequeued. In this case, the cache message matching module will notify the host cache management module to pull more host cache information to meet the matching requirements. The cache and message matching ends, and the multi-linked list is notified to resume the sleep state.

[0108] Repeat the above steps until all message descriptors or host cache information in the multi-link list have been truly authorized to be dequeued or an error is reported to stop matching.

[0109] Step 8: When the multi-linked list receives the real authorization output by the cache message matching module, Figure 4 As shown in the lower part, the actual dequeue operation of the packet descriptor or host buffer information is performed.

[0110] After receiving the first real authorization, the data corresponding to the linked list m obtained in step 6 is output to complete the dequeue. It is determined whether the head pointer and the tail pointer of the linked list m are equal. If they are equal, it means that the current dequeued content is the last content in the linked list m. The head and tail pointers are no longer updated, and the empty and full flags corresponding to the linked list m are set to the empty state.

[0111] If they are not equal, it means that the current dequeued content is not the last content in linked list m. At this time, the head pointer is written into the free pointer pool, and the next hop directory of linked list m obtained in step 6 is used as the new head pointer. The next hop directory is used as the address to read the content stored at the address and the new next hop directory, completing a new round of pre-dequeueing of queue m, and waiting until the linked list polls m again.

[0112] After the linked list m is dequeued after the real authorization is completed, the linked list sequence number m'=m+1 is switched, and the dequeuing operation is the same as the above steps after the real authorization is completed.

[0113] When the cache message matching module finishes matching, the linked list sequence number m' currently polled is recorded, so that the linked list sequence number m=m'. In this way, if the queue starts the pre-dequeue process again next time, it will start from the linked list m' to prevent the dequeue order from being wrong.

[0114] The host cache information is directly sent to the subsequent message splicing module after it is actually dequeued, and the splicing is completed after the corresponding actual message is returned.

[0115] After the message descriptor is actually dequeued, it needs to be sent to the network card kernel first to pull the actual message and obtain the actual message.

[0116] Step 9, the network card kernel pulls the actual message based on the message descriptor output in step 8, and outputs the message to the message splicing module;

[0117] Step 10: If the physical storage space inside the host corresponding to a host cache information is sufficient to store the actual message, the host cache information and the corresponding actual message are packaged and sent to the host together, and the subsequent storage work will be solved by the host itself.

[0118] If the internal physical storage space of a host corresponding to the host cache information cannot complete the storage of the actual message, the actual message will be cut into fragments of the same size as the internal physical storage space of the host corresponding to the host cache information. Each fragment will be packaged with the host cache information and sent to the host together. The subsequent storage work will be solved by the host itself.

[0119] The number N of multi-linked lists can be calculated by the following method: if the working clock cycle of the chip is T; as mentioned in step 6, based on the head pointer, the time to pull out the queue content and the next-hop directory is t0; when the match is set to be successful, as mentioned in step 7, the time from the start of the message cache match to the real authorization is t1; then when N>(t0+t1) / T, the pipeline stability of the dequeue operation can be guaranteed, so that the real dequeue message descriptor or host cache information can appear in each clock cycle, thereby absorbing the queue switching caused by the queue state machine loss.

[0120] The present invention provides a method for solving the matching between messages and host cache under the virtio protocol. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for resolving message and host cache matching under the virtio protocol, characterized in that: include: Establish host cache management module and traffic management module; The host cache management module is used to send a request to the host to obtain host cache information; The traffic management module is used to perform traffic management; Then follow these steps: Step 1: During the initialization phase of the network card, the host cache management module sends a request to the host to obtain host cache information. Based on the request, the host sends the host cache information to the network card, and the network card caches the information in advance. Step 2: After the network card receives the virtio message, after the kernel completes the message processing, it sends the message descriptor to the traffic management module, the traffic management module completes the cache of the message descriptor, and starts traffic management; Step 3, the traffic management module provides authorization to the message descriptor of the corresponding queue based on the configured traffic allocation, and the message descriptor of the corresponding queue is matched with the host cache information cached in step 1; In step 3, when there is unconsumed host cache information in the queue storing the message, the queue participates in the scheduling of the traffic management module; the traffic management module supports SP, DWRR, and RR scheduling modes based on the service configuration scheduling rules, and also supports adding rate shaping shapers for speed limiting; the scheduling outputs dequeue authorization to the corresponding queue based on the current scheduling results; Step 4: When the message and the host cache information meet the requirements of the virtio protocol, the traffic management module pulls the actual message based on the message descriptor, and combines the host cache information to correctly transmit the message to the cache address corresponding to the host.

2. The method according to claim 1, characterized in that: In step 1, after the network card is initialized, a queue is established, and a storage space exclusive to each queue and a shared storage space that can be used by all queues are configured based on the queue.

3. The method according to claim 2, characterized in that In step 1, the virtio mode can be configured based on the queue. Virtio can be configured as virtio normal mode and virtio merge buffer mode. When the queue is configured in virtio normal mode, the matching of packets and host cache information is based on the following rules: one cache space of the host cache is used to store one packet. Even if there is redundant space after storing one packet in the host cache, the remaining redundant space will not be used for the second packet. If the host cache space cannot accommodate a complete packet, the network card reports an abnormal interrupt and discards the corresponding packet and host cache information. When the queue is configured in virtio merge buffer mode, the following rules are used to match the message and host cache information: Based on the virtio normal mode, when the cache space of a host cannot accommodate a complete message, more than two host caches are allowed to store data messages together. When the cache space of a host cannot accommodate a complete message, the network card no longer reports an exception, but continues to obtain the next host cache information until the next data message can be completely stored.

4. The method according to claim 3, characterized in that In step 1, based on the configured queues and the space allocated to each queue, the hardware automatically initiates a request to the host to obtain the host cache information. When the host sends the corresponding host cache information, it is stored in sequence according to the queues. In addition, the sum of the cache spaces corresponding to all the current host cache information in the queue must be recorded for matching comparison in virtio merge buffer mode.

5. The method according to claim 4, characterized in that In step 1, the host cache information includes the size of the host cache space and the starting address of the cache space in the host; The host cache space is used to store actual data messages; The start address is used to store received messages.

6. The method according to claim 5, characterized in that In step 1, the host cache management module for pulling the host cache manages the host cache information based on the queue; after the network card is initialized, the number of host cache information that can be stored in each queue is configured, and the subsequent host will periodically send the number of prepared host cache information and the storage address of the host cache information in the host to the host cache management module in the network card; When any of the following conditions is met, the host cache management module reads the host cache information from the host based on the storage address of the host cache information sent by the host in the host: Condition 1: the amount of host cache information stored in the queue has not reached the maximum value that can be pre-fetched, and the host cache information prepared for the queue in the host received by the host cache management module has not been fully fetched, and at this time, a request to fetch the host cache information continues to be initiated to the host; Condition 2: When the host cache information prepared for the queue in the host received by the host cache management module has not been fully pulled out, and even if the amount of stored host cache information has reached the maximum value that can be pre-fetched, but the sum of the cache space corresponding to the currently cached host cache information is still unable to store the next complete datagram, the host cache management module will continue to initiate a request to pull the host cache information from the host until the sum of the cache space corresponding to the cached host cache information is able to store the next complete datagram.

7. The method according to claim 6, characterized in that In step 1, a linked list module is maintained inside the host cache management module to perform queue management; the linked list module includes two sets of linked lists, one of which is responsible for storing and processing message descriptors, and the other is responsible for storing and processing host cache information.

8. The method according to claim 7, characterized in that In step 1, the linked list stores the message and host cache information by maintaining a head pointer, a tail pointer, a next hop, an idle pointer, a linked list sequence number and a linked list empty state.

9. The method according to claim 8, characterized in that In step 1, when any of the following three conditions is met, the host cache management module automatically initiates a request to the host to obtain host cache information: In the first case, after the queue configuration is completed, the hardware automatically pulls the host cache information until the exclusive storage space of all queues has completed the storage of the host cache information; In the second case, when the host cache information stored in a queue is matched with the message and dequeued, if the exclusive storage space of the current queue is idle at this time, the host cache management module will continue to pull new host cache information from the host and queue it to the end of the queue until the exclusive storage space has completed the storage of the host cache information; In the third case, for the virtio merge buffer mode, a message cache matching module is established, which matches the message with the virtual cache address one by one, packages them together, and notifies the host to parse and process them; If the host cache space corresponding to the host cache information stored in the exclusive storage space cannot store the next complete datagram, the message cache matching module will inform the host cache management module, and the host cache management module will again initiate a request to the host to pull the host cache information. The newly acquired host cache information is stored in the shared storage space and queued at the end of the queue.

10. The method according to claim 9, characterized in that In the second and third cases, joining the team includes the following steps: Step 1-1, based on the queue, obtain the queue entry list sequence number; Step 1-2, based on the linked list sequence number, operate the corresponding linked list: read the empty state of the linked list, read the free pointer, and read the tail pointer; Step 1-3, store the data in the storage address corresponding to the free pointer, and maintain the pointer based on the current state of the linked list: if the current linked list is empty, write the free pointer to the head pointer and the tail pointer at the same time, and update the linked list state to a non-empty state; if the current linked list is not empty, use the tail pointer as the write address, write the free pointer to the next hop table, and use the free pointer to replace the original tail pointer.

11. The method according to claim 10, characterized in that In step 2, the message descriptor records all the information of the message, including the message packet length, the storage address of the message in the network card, and the queue to which the message belongs; Using message descriptors for queue and traffic management can effectively save the cache space size of the traffic management module.

12. The method according to claim 11, characterized in that In step 2, the traffic management module maintains message descriptors based on queues and uses linked lists to manage queues.

13. The method according to claim 12, characterized in that In step 3, when the corresponding queue receives the dequeue authorization, the linked list performs the dequeue operation, and after obtaining the message descriptor and host cache information stored in the linked list, the message is matched based on the current virtio mode configuration.

14. The method according to claim 13, characterized in that In step 3, the message matching is performed based on the current virtio mode configuration. If the current queue is configured as the virtio merge buffer mode, the following judgment rules are followed to control the dequeuing of the message and the host cache information: Step 3-a1, if the message length recorded in the current message descriptor is less than the sum of the cache spaces corresponding to all host cache information in the current queue, then the host cache information is dequeued and the sum of the cache spaces corresponding to the currently dequeued host cache information is recorded; Step 3-a2, when the sum of the cache space corresponding to the current host cache information in the queue and the previously dequeued host cache information is greater than or equal to the packet length of the message in the message descriptor, the message descriptor and the host cache information are dequeued at the same time; Step 3-a3, if the message length recorded in the current message descriptor is still greater than the sum of the cache spaces corresponding to all host cache information in the current queue, it means that even if all host cache information in the current queue is dequeued, it cannot support the storage of the current message. At this time, a pull request is sent to the host cache management module, requesting the host cache management module to continue to send pull requests to the host, so that the queue obtains more host cache information.

15. The method according to claim 13, characterized in that The message matching is performed based on the current virtio mode configuration. If the current queue is configured as virtio normal mode, the following judgment rules are followed to control the dequeuing of messages and host cache information: Step 3-b1, if the message length recorded in the current message descriptor is less than or equal to the cache space corresponding to the current host cache information, the message descriptor and the host cache information are dequeued at the same time; Step 3-b2, if the message length recorded in the current message descriptor is greater than the cache space corresponding to the current host cache information, the cache space sent by the host cannot meet the protocol requirements. The message descriptor and the host cache information are dequeued at the same time, and the packet drop operation is performed, and the exception is reported to the host.

16. The method according to claim 14 or 15, characterized in that The message descriptor and the host cache information are dequeued simultaneously, comprising the following steps: Step 3-c1, the linked list module receives the dequeue authorization to the corresponding queue, and the linked list storing the message descriptor and the linked list storing the host cache information simultaneously obtain the current queue head pointer based on the queue number; Step 3-c2, based on the current queue head pointer, obtain the storage address pointed to by the head pointer and the next hop pointer information of the current queue, and read the message descriptor and host cache information from the storage address; Step 3-c3, when the dequeuing condition is met, use the next hop pointer information obtained in step 3-c2 as the new head pointer to update the head pointer of the current queue; at the same time, put the head pointer that has completed dequeuing into the free pointer pool for use when joining the queue next time; Step 3-c4, when the dequeue condition is not met, exit the current dequeue process without rewriting the head pointer of the current queue or recycling the head pointer.

17. The method according to claim 16, characterized in that The linked list storing the host cache information and the linked list storing the message descriptor, because the host cache information and the message descriptor of the same queue are continuously dequeued, in order to absorb the delay consumed by step 3-c1 to step 3-c2, by using two or more linked lists, it is ensured that after the linked list completes step 3-c1 to step 3-c2, the host cache information and the message descriptor that are dequeued are continuous according to the switching order of the linked list, which specifically includes the following steps: Step 3-d1, after the linked list module receives the authorization for scheduling output, it obtains the sequence number of the dequeued linked list based on the authorization queue; Step 3-d2, based on the sequence number of the dequeued linked list, operate the corresponding linked list and read the head pointer of the linked list; Step 3-d3, using the head pointer as the address, read the stored message descriptor and host cache information, and read the corresponding next hop information; Step 3-d4, performing virtio protocol matching on the read message descriptor and host cache information, and completing output of the message descriptor and host cache based on the matching result; When the output is confirmed, the current head pointer is recovered, and the next hop information is used as the read address to obtain the new message descriptor, host cache information and next hop information; Step 3-d5, based on the delay of step 3-d2 to step 3-d4, absorbs the delay of reading table entry information and protocol matching through cyclic switching of more than two linked lists under the same queue, so that each beat can be output without flow control and sufficient storage message descriptor and host cache information, thereby ensuring a certain acceleration ratio to absorb the delay during the dequeue operation.

18. The method according to claim 17, characterized in that In step 4, the traffic management module pulls the actual message in sequence according to the corresponding byte order based on the message storage address in the network card recorded in the message descriptor, and outputs it to the subsequent message splicing module; when the host cache information is dequeued, it is directly output to the subsequent message splicing module.

19. The method according to claim 18, characterized in that In step 4, the message splicing module splices the actual message with the host cache information, and finally sends it to the host side.

Images