Method, device, equipment and medium for remote memory access based on network messages
By configuring the MAC header information and destination address as mapping tables and directly sending packets, the existing technology has solved the problem of complex logic of remote data writing and high bus traffic bandwidth in the cross-chip middle and middle, and achieved a simpler and lower cost remote memory access method.
Patent Information
- Application Number
- CN202010614301.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2040-06-30
AI Technical Summary
When the prior art realizes remote writing of data across chips, the processing logic is complex, the cost is high, and data copy is required, resulting in an increase in the bus traffic bandwidth.
By configuring the MAC header information and the destination address of the data as a mapping table, the destination address and length of the data information transmitted by the third-party component are obtained, and compared with the mapping table to obtain the corresponding MAC header information, and directly group packets to send to avoid data copying.
It realizes a simpler and lower-cost remote memory access method, reduces processing logic, reduces bus traffic bandwidth, and improves packetization efficiency.
Smart Images
Figure CN113873049B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of communication technology, and in particular to a method, device, equipment and medium for remote memory access based on network messages. Background Art
[0002] To solve the efficiency problem of cross-chip remote data writing, the current mainstream method in the industry is that the network processor sending end uses the standard RDMA (Remote Direct Memory Access) protocol to encapsulate the data into network messages and send them out. The network processor receiving end parses the network message according to the RDMA protocol and outputs the data to the destination address, such as Figure 2 As shown. The specific implementation principle of this method is: RDMA is implemented by three protocols: RDMA, DDP (Direct Data Placement), and MPA (Marker-based, Protocol-data-unit-Aligned), which together constitute the iWARP protocol family to ensure the interoperability of high-speed networks. The RDMA layer is used to convert RDMA read, write, and Send operation messages into RDMA messages, and transmit the RDMA messages to the DDP layer. The DDP layer encapsulates the RDMA message segments into DDP data packets and forwards them to the lower MPA layer. The MPA layer inserts the identifier, length, and CRC (Cyclic Redundancy Check) check into the DDP data packet to form an MPA data segment. The TCP (Transmission Control Protocol) layer is responsible for scheduling TCP data segments to ensure that the packets can reach the target location smoothly. The IP layer adds necessary network routing data information to the data packet.
[0003] However, the above method is complex to implement and has high cost, and is mainly used in the field of computer communications. Although it solves the problem of data copying between the kernel space and the user space of the computer operating system, it still requires the RDMA device to copy data from the computer operating system user space to the internal space of the RDMA device after receiving the RDMA read and write data request from the computer operating system user space. Summary of the invention
[0004] The present disclosure provides a method, apparatus, device and medium for remote memory access based on network messages, the technical purpose of which is to reduce transmission logic, realize data transmission without copying data, and reduce bus traffic bandwidth.
[0005] The above technical objectives of the present disclosure are achieved through the following technical solutions:
[0006] A remote memory access method based on network messages, comprising:
[0007] The MAC header information and the destination address of the data are configured as a mapping table, and the mapping table is stored;
[0008] Get the data destination address and data length of the data information transmitted by the third-party component;
[0009] Compare the data destination address and its data length with the destination address in the mapping table to obtain the MAC header information corresponding to the data destination address;
[0010] After receiving the data destination address and its corresponding MAC header information and data information, the receiving device outputs the data information to the data destination address.
[0011] Furthermore, before comparing the data destination address and its data length with the destination address in the mapping table, it is determined whether the different data destination addresses and their data lengths constitute a continuous address space; if so, the different data destination addresses are merged and the smallest data destination address is selected as the merged data destination address.
[0012] Furthermore, the comparison of the data destination address and its data length with the destination address in the mapping table includes:
[0013] Divide the destination address in the mapping table into N groups of address ranges, and number the N groups of address ranges in sequence, and if the data destination address falls within a certain group of address ranges, obtain the MAC header information corresponding to the address range;
[0014] If the data destination address falls within at least two groups of address ranges at the same time, the address range with a smaller number is selected and matched with the corresponding MAC header information; wherein N is a positive integer greater than 1.
[0015] Furthermore, the data destination address is a starting address, the data length is an address length of the data destination address, and the mapping table includes an address range of the destination address.
[0016] A device for remote memory access based on network messages, comprising a network processor, wherein the network processor comprises a sending device and a receiving device, wherein the sending device comprises a header information management unit, an address sensing unit and a message packaging and sending unit, and the header information management unit further comprises:
[0017] Configuration component, configuring the MAC header information of the data and the destination address into a mapping table;
[0018] An internal storage component for storing the mapping table;
[0019] An address selection component compares the data destination address and its data length with the destination address in the mapping table to obtain MAC header information corresponding to the data destination address;
[0020] The address sensing unit obtains the data destination address and data length of the data information transmitted by the third-party component;
[0021] The message packaging and sending unit packages the data destination address and its corresponding MAC header information and data information and sends them to the receiving device;
[0022] The receiving device receives the data destination address and its corresponding MAC header information and data information, and outputs the data information to the data destination address.
[0023] Furthermore, the header information management unit also includes a judgment component. Before comparing the data destination address and its data length with the destination address in the mapping table, the judgment component determines whether the different data destination addresses and their data lengths constitute a continuous address space. If so, the different data destination addresses are merged and the smallest data destination address is selected as the merged data destination address.
[0024] Furthermore, the address selection component includes:
[0025] A division component is used to divide the destination addresses in the mapping table into N groups of address ranges, and the N groups of address ranges are numbered in sequence, where N is a positive integer greater than 1;
[0026] A first acquisition component selects MAC header information corresponding to a certain address range when the data destination address falls within the address range;
[0027] The second acquisition component selects the address range with a smaller number and matches the corresponding MAC header information when the data destination address falls within at least two groups of address ranges at the same time.
[0028] Furthermore, the device also includes a storage unit, which stores the data information output by the receiving device to the data destination address.
[0029] Furthermore, the internal storage component includes:
[0030] RAM memory, storing the mapping table;
[0031] The CACHE storage caches the data information transmitted by the third-party component.
[0032] A device for implementing remote memory access based on network messages, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.
[0033] A computer medium having a computer program stored thereon, wherein the computer program implements any of the above methods when executed by a processor.
[0034] The beneficial effects of the present disclosure are: (1) simpler implementation and lower cost, and compared with the existing RDMA method, the processing logic is reduced; (2) zero copy, reducing the bus traffic bandwidth, the sending device directly receives the data transmitted by the third-party component, and adds MAC header information and destination address information based on the data, and no data copy is required during the message packetization process; (3) shortening the network message header construction and packetization time, the sending device directly obtains the data destination address and MAC header information based on the information transmitted by the third-party component, and composes a complete network message. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 is a flow chart of the disclosed method;
[0036] Figure 2 It is a schematic diagram of the message hierarchy of RDMA in the prior art;
[0037] Figure 3 A schematic diagram of the message hierarchy of the present disclosure;
[0038] Figure 4 A framework diagram of the device disclosed herein;
[0039] Figure 5 The figure is a flow chart of sending and receiving messages disclosed in the present invention. DETAILED DESCRIPTION
[0040] The technical solution of the present disclosure will be described in detail below in conjunction with the accompanying drawings. In the description of the present disclosure, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features, and are only used to distinguish different components.
[0041] The third-party components include but are not limited to the CPU and the baseband accelerator. The third-party components are connected to the sending device through a bus routing. The bus is instantiated as an AXI bus with attributes such as read and write data, read and write addresses, and data length. The internal storage component is a RAM memory, which is used to store a mapping table between header information and data destination addresses, and to cache single data information transmitted by the third-party components.
[0042] As a specific embodiment, the message header information includes MAC header information, and can also be a combination of MAC / IP header information and MAC / IP / UDP header information, such as Figure 3 shown.
[0043] Figure 1 is a flow chart of the disclosed method, such as Figure 1 As shown, the method for remote memory access based on network messages includes: the MAC header information and the destination address of the data are configured as a mapping table, and the mapping table is stored; then the data destination address and the data length of the data information transmitted by the third-party component are obtained; the data destination address and the data length are compared with the destination address in the mapping table to obtain the MAC header information corresponding to the data destination address; after the receiving device receives the data destination address and the corresponding MAC header information and the data information package, the data information is output to the above-mentioned data destination address. As a specific embodiment, the receiving device outputs the data information to the memory corresponding to the data destination address.
[0044] As a specific embodiment, after obtaining the data destination address and data length of the data information transmitted by the third-party component, the data information is cached.
[0045] Generally, the data destination address is the starting address, and the data length indicates the address length of the data destination address, that is, the range of the data destination address from the starting address to the end, and the mapping table also includes the address range of the destination address. Therefore, comparing the data destination address and its data length with the mapping table is actually comparing which destination address ranges in the mapping table the data destination address falls within.
[0046] Figure 4 This is a framework diagram of the device disclosed in the present invention. The functions of each component can be referred to in the following embodiments of the present invention and will not be described in detail.
[0047] Embodiment 1: Figure 5 As shown, after power-on initialization, the configuration component of the header information management unit configures the mapping table between the MAC header information and the data destination address, which is stored in the internal storage component in the header information management unit; the third-party component transmits the data destination address and data length of the data information to the sending device, and routes the single data information to the internal storage component via the bus. The address sensing unit obtains and extracts the data destination address and data length transmitted by the third-party component, and the address selection component compares the obtained information with the destination address in the mapping table stored in the internal storage component to obtain the corresponding MAC header information.
[0048] The address selection component is mainly used to perform address matching on the data information transmitted by the third-party component. The address component includes a division component, a first acquisition component, and a second acquisition component. Before the address matching starts, the division component divides all the destination addresses in the mapping table into N groups of address ranges, and numbers the N groups of address ranges in sequence, where N is a positive integer greater than 1. The address matching method includes: (1) comparing the data destination address and its data length with the N groups of address ranges of the internal storage component at the same time, and the first acquisition component selects the MAC header information corresponding to the group of address ranges that fall within the group; (2) If multiple groups of address ranges are hit at the same time, the second acquisition component selects the address group number with a smaller sequence number as the hit result; (3) If none of them are hit, the access is discarded, an error interrupt is generated, and the access address is reported to the software.
[0049] The message packaging and sending unit adds MAC header information and data destination address information to the data information cached in the internal storage component transmitted by the third-party component according to the network message format, and sends the network message to the receiving device through the network port. After receiving the network message, the receiving device extracts the data destination address and data information, and outputs the data information to the data destination address. The data information is generally stored in the storage unit corresponding to the data destination address.
[0050] Embodiment 2: The difference from embodiment 1 is that the header information management unit also includes a judgment component. Before comparing the data destination address and its data length with the destination address in the mapping table, the judgment component determines whether different data destination addresses and their data lengths constitute a continuous address space. If they constitute a continuous address space, the different data destination addresses are merged and the smallest data destination address is selected as the merged data destination address, and the data length is the total length.
[0051] Embodiment 3: The difference from Embodiment 1 and Embodiment 2 is that the internal storage component of Embodiment 2 includes RAM memory and CACHE memory (usually composed of SRAM (Static Random Access Memory)), wherein the RAM memory is used to store the mapping table between MAC header information and destination address, and the CACHE memory is used to cache multiple data transmitted by third-party components. In Embodiment 1, the internal storage component is only composed of RAM memory, and the RAM memory caches only a single data transmitted by the third-party component, and the CACHE memory is faster.
[0052] In summary, the technical solution provided by the embodiment of the present invention is simple and flexible to implement, and technicians can choose the method to use according to the actual application scenario. This method can effectively reduce the bus traffic bandwidth and improve the efficiency of packet sending.
[0053] The above description is only a preferred embodiment of the present invention. The above embodiments and features in the embodiments can be combined arbitrarily without conflict, and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for remote memory access based on network messages, characterized in that: include: The sending device configures the MAC header information and the destination address of the data into a mapping table, and stores the mapping table; The sending device obtains the data destination address and data length of the data information transmitted by the third-party component, wherein the data destination address is used to indicate the memory space location; The sending device compares the data destination address and its data length with the destination address in the mapping table to obtain the MAC header information corresponding to the data destination address; The sending device packages the data destination address and its corresponding MAC header information and data information and sends them to the receiving device; After receiving the data destination address and its corresponding MAC header information and data information, the receiving device outputs the data information to the memory corresponding to the data destination address.
2. The method for remote memory access based on network messages according to claim 1, characterized in that: Before comparing the data destination address and its data length with the destination address in the mapping table, determine whether the different data destination addresses and their data lengths constitute a continuous address space. If so, merge the different data destination addresses and select the smallest data destination address as the merged data destination address.
3. The method for remote memory access based on network messages as claimed in claim 2, characterized in that: The comparison between the data destination address and its data length and the destination address in the mapping table includes: Divide the destination address in the mapping table into N groups of address ranges, and number the N groups of address ranges in sequence, and if the data destination address falls within a certain group of address ranges, obtain the MAC header information corresponding to the address range; If the data destination address falls within at least two groups of address ranges at the same time, the address range with a smaller number is selected and matched with the corresponding MAC header information; wherein N is a positive integer greater than 1.
4. The method for remote memory access based on network messages according to any one of claims 1 to 3, characterized in that: The data destination address is a starting address, the data length is an address length of the data destination address, and the mapping table includes an address range of the destination address.
5. A device for remote memory access based on network messages, comprising a network processor, characterized in that: The network processor includes a sending device and a receiving device, wherein the sending device includes a header information management unit, an address sensing unit and a message packaging and sending unit, and the header information management unit further includes: Configuration component, configuring the MAC header information of the data and the destination address into a mapping table; An internal storage component for storing the mapping table; An address selection component compares the data destination address and its data length with the destination address in the mapping table to obtain MAC header information corresponding to the data destination address; The address sensing unit obtains a data destination address and a data length of data information transmitted by a third-party component, wherein the data destination address is used to indicate a memory space location; The message packaging and sending unit packages the data destination address and its corresponding MAC header information and data information and sends them to the receiving device; The receiving device receives the data destination address and its corresponding MAC header information and data information, and outputs the data information to the memory corresponding to the data destination address.
6. The device for remote memory access based on network messages as claimed in claim 5, characterized in that: The header information management unit also includes a judgment component. Before comparing the data destination address and its data length with the destination address in the mapping table, the judgment component determines whether the different data destination addresses and their data lengths constitute a continuous address space. If so, the different data destination addresses are merged and the smallest data destination address is selected as the merged data destination address.
7. The device for remote memory access based on network messages as claimed in claim 6, characterized in that: The address selection component comprises: A division component is used to divide the destination addresses in the mapping table into N groups of address ranges, and the N groups of address ranges are numbered in sequence, where N is a positive integer greater than 1; A first acquisition component selects MAC header information corresponding to a certain address range when the data destination address falls within the address range; The second acquisition component selects the address range with a smaller number and matches the corresponding MAC header information when the data destination address falls within at least two groups of address ranges at the same time.
8. The device for remote memory access based on network messages as claimed in claim 7, characterized in that: The device also includes a storage unit, which stores the data information output by the receiving device to the data destination address.
9. The device for remote memory access based on network messages as claimed in claim 8, characterized in that: The internal storage component includes: RAM memory, storing the mapping table; The CACHE storage caches the data information transmitted by the third-party component.
10. The device for remote memory access based on network messages according to any one of claims 5 to 9, characterized in that: The data destination address is a starting address, the data length is an address length of the data destination address, and the mapping table includes an address range of the destination address.
11. A device for implementing remote memory access based on network messages, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.
12. A computer medium, characterized in that The computer medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.