Data management method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a data management method and device, electronic equipment and a medium, and the method comprises the steps: reading the memory management information of business data from a user mode protocol stack through a fusion protocol stack based on a zero copy interface; segmenting a memory chain corresponding to the memory management information based on a slice interface through a fusion protocol stack to obtain a plurality of slice data; converting the plurality of slice data based on a format conversion interface through a fusion protocol stack to obtain target data in a preset format, and sending the target data to an upper layer service; and accessing the service data based on the target data through the upper-layer service. Therefore, the fusion protocol stack receives the memory chain corresponding to the service data to the upper-layer service based on the zero copy interface provided by the user mode protocol stack, so that the upper-layer service accesses the service data based on the memory chain in a zero copy manner, the influence on the performance is reduced, and the use experience of a user is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data management method, a data management device, an electronic device, and a readable storage medium. Background Technology

[0002] In data processor storage scenarios, under limited resources, in order to maximize the data processing efficiency of a single core, a user-space protocol stack can be used in the SPDK network layer. During data transmission, the user-space sock_read interface can be used to copy data from the user-space protocol stack cache to the converged protocol stack. This means that each data packet needs to be copied once. Furthermore, when upper-layer services access data, they also need to go through the data copying process. This approach has a significant impact on performance and, consequently, on the user experience. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a data management method, a data management device, an electronic device, and a readable storage medium that overcome or at least partially solve the above problems.

[0004] In a first aspect, embodiments of the present invention provide a data management method, the method comprising: By integrating the protocol stack and using a zero-copy interface, memory management information of business data is read from the user-space protocol stack. The memory chain corresponding to the memory management information is segmented using the fusion protocol stack based on the slicing interface to obtain multiple slice data. The multiple slice data are converted using the format conversion interface through the fusion protocol stack to obtain target data in a preset format, and the target data is then sent to the upper-layer service. The upper-layer service accesses the service data based on the target data.

[0005] Optionally, the step of reading memory management information of service data from the user-space protocol stack via a zero-copy interface through the converged protocol stack includes: The system receives service data from the network card through the user-mode protocol stack, saves the service data to a preset data structure, and generates memory management information corresponding to the service data. By integrating the protocol stack, it is possible to detect whether a read / write event has occurred in the user-space protocol stack; When the read / write event occurs in the user-space protocol stack, the memory management information is read from the user-space protocol stack through the zero-copy interface of the fusion protocol stack.

[0006] Optionally, the step of segmenting the memory chain corresponding to the memory management information through the fusion protocol stack based on the slicing interface to obtain multiple slice data includes: Based on the memory management information, the fusion protocol stack generates a corresponding memory chain; The memory chain is divided into multiple slice data by using the fusion protocol stack based on the slice interface.

[0007] Optionally, the memory chain is segmented using the fusion protocol stack based on the slicing interface to obtain multiple slice data, including: The memory chain is divided into multiple slice data by means of the fusion protocol stack based on the slice interface and according to the preset input / output length.

[0008] Optionally, the slice data includes: memory address information and corresponding data length information; the step of converting the multiple slice data through the fusion protocol stack based on the format conversion interface to obtain target data in a preset format includes: The fusion protocol stack uses a format conversion interface to convert multiple memory address information and corresponding data length information to obtain target data in a preset format.

[0009] Optionally, the target data includes: multiple memory address information and corresponding data length information; accessing the service data based on the target data through the upper-layer service includes: The upper-layer service accesses the service data based on the multiple memory address information and the corresponding data length information.

[0010] Optionally, after accessing the service data based on the target data through the upper-layer service, the method further includes: After the upper-layer service finishes accessing the service data, the memory corresponding to the accessed service data is released through a pre-registered memory release interface.

[0011] Secondly, embodiments of the present invention provide a data management device, the device comprising: The read module is used to read memory management information of business data from the user-space protocol stack via the zero-copy interface of the converged protocol stack; The segmentation module is used to segment the memory chain corresponding to the memory management information through the fusion protocol stack based on the segmentation interface to obtain multiple slice data; The conversion module is used to convert the multiple slice data through the fusion protocol stack based on the format conversion interface to obtain target data in a preset format, and send the target data to the upper layer service; The access module is used to access the business data based on the target data through the upper-layer business.

[0012] Optionally, the reading module includes: The receiving submodule is used to receive service data from the network card through the user-mode protocol stack, save the service data to a preset data structure, and generate memory management information corresponding to the service data; The detection submodule is used to detect whether a read / write event has occurred in the user-space protocol stack through the fusion protocol stack; The reading submodule is used to read the memory management information from the user-space protocol stack through the fusion protocol stack based on the zero-copy interface when the read / write event occurs in the user-space protocol stack.

[0013] Optionally, the segmentation module includes: A generation submodule is used to generate a corresponding memory chain based on the memory management information through the fusion protocol stack; The slicing module is used to slice the memory chain based on the slicing interface through the fusion protocol stack to obtain multiple slice data.

[0014] Optionally, the molecular cutting module includes: The segmentation unit is used to segment the memory chain according to a preset input / output length based on the fusion protocol stack and the slicing interface to obtain multiple slice data.

[0015] Optionally, the slice data includes: memory address information and corresponding data length information; the conversion module includes: The conversion submodule is used to convert multiple memory address information and corresponding data length information through the format conversion interface of the fusion protocol stack to obtain target data in a preset format.

[0016] Optionally, the target data includes: multiple memory address information and corresponding data length information; the access module includes: The access submodule is used to access the service data based on the multiple memory address information and the corresponding data length information through the upper-layer service.

[0017] Optionally, the device further includes: The release module is used to release the memory corresponding to the accessed service data through a pre-registered memory release interface after the upper-layer service has finished accessing the service data.

[0018] Thirdly, embodiments of the present invention provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the data management method as described in the first aspect.

[0019] Fourthly, embodiments of the present invention provide a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the data management method as described in the first aspect are implemented.

[0020] The embodiments of the present invention have the following advantages: In this embodiment of the invention, the memory management information of the service data is first read from the user-space protocol stack via a zero-copy interface using the converged protocol stack. Then, the memory chain corresponding to the memory management information is segmented using a slicing interface using the converged protocol stack to obtain multiple slice data. Next, the multiple slice data are converted using a format conversion interface using the converged protocol stack to obtain target data in a preset format, and the target data is sent to the upper-layer service. Finally, the upper-layer service accesses the service data based on the target data. Thus, the converged protocol stack, using the zero-copy interface provided by the user-space protocol stack, receives the memory chain corresponding to the service data and sends it to the upper-layer service, enabling the upper-layer service to access the service data based on a zero-copy memory chain, thereby reducing the impact on performance and improving the user experience. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the steps of a data management method provided in an embodiment of the present invention; Figure 2 This is a flowchart of another data management method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a memory structure provided in an embodiment of the present invention; Figure 4 This is a flowchart of another data management method provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a data management device provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] In related technologies, the user-space `sock_read` interface is used to copy data from the user-space protocol stack cache to the converged protocol stack. This requires a copy for each data packet, and subsequent upper-layer services also need to go through this data copying process when accessing data. This significantly impacts performance and consequently affects the user experience. To address these technical problems, this invention provides a data management method. The core concept is to first read the memory management information of the business data from the user-space protocol stack using a zero-copy interface within the converged protocol stack. Then, using a slicing interface, the converged protocol stack segments the memory chain corresponding to the memory management information to obtain multiple slices of data. Next, using a format conversion interface, the converged protocol stack converts these slices of data to obtain target data in a preset format, and sends the target data to the upper-layer services. Finally, the upper-layer services access the business data based on the target data. Thus, the converged protocol stack, using the zero-copy interface provided by the user-space protocol stack, receives the memory chain corresponding to the business data and sends it to the upper-layer services, enabling the upper-layer services to access the business data based on the memory chain with zero copy, thereby reducing the impact on performance and improving the user experience.

[0026] like Figure 1 The diagram illustrates a flowchart of a data management method provided by an embodiment of the present invention. The method may specifically include the following steps: Step 101: Read the memory management information of the business data from the user-space protocol stack through the zero-copy interface of the converged protocol stack.

[0027] In this invention, the converged protocol stack can be SF-STACK (Super Fusion stack), which is a converged protocol stack implemented using C programming language. The user-space protocol stack can be ESAR (Electronically Scanned Array Radar), which can refer to a self-developed high-performance user-space protocol stack. The zero-copy interface can be esar_rbuf_pipe_append.

[0028] In this invention, esar_rbuf_pipe_append is the core data plane interface in the SF-STACK protocol stack used for "appending data to the receive buffer pipe". Its core function is to efficiently append data from the receiving side (such as TCP segments and RDMA message data) to the specified "receive buffer pipe (rbuf_pipe)" to provide a low-latency, high-throughput data reading channel for upper-layer applications.

[0029] This invention, SF-STACK, integrates three transmission protocols: kernel-mode TCP, user-mode TCP, and RDMA. Kernel-mode TCP is characterized by high reliability, serving as a failover solution and suitable for critical business applications. User-mode TCP is characterized by bypassing the kernel, reducing context switching, and providing ultra-high throughput. RDMA is characterized by zero-copy and low latency, suitable for high-speed transmission within data centers.

[0030] In this invention, ESAR stands for Electronically Scanned Array Radar, and this invention is an abbreviation for the self-developed high-performance user-space protocol stack.

[0031] In this invention, SF-STACK can use the esar_rbuf_pipe_append interface to read memory management information of service data from ESAR.

[0032] In this invention, the function of SF-STACK can be a poller, with the entry point being: uint32_tEsarPollSockGroup(zj_ctx_t *ctx, zj_worker_t *worker, uint8_t domain, void *arg, uint32_t budget). The interface for SF-STACK to obtain business data can be: static intEsarReadSockData(zj_ctx_t*ctx, zj_worker_t*worker, TcpWrkPriv_t*esar_wrk_priv, TcpSockCh_t *sock). The interface for SF-STACK to obtain data buffer from ESAR can be: static inline intesar_rbuf_pipe_append(struct esar_rbuf_pipe *pipe). The interface for calling ESAR can be: ssize_tsock_zreadv(void *_sock, struct esar_iovec *iov, int iovcnt). The segbuf segmented buffer is passed to iov via zero-copy for SF-STACK parsing and invocation.

[0033] When reading data, SF-STACK can directly copy the buf address without performing memcopy.

[0034] Step 102: The memory chain corresponding to the memory management information is segmented using the fusion protocol stack based on the slicing interface to obtain multiple slice data.

[0035] In this invention, after the fusion protocol stack obtains the memory management information, it can obtain the memory chain corresponding to the memory management information. Then, the fusion protocol stack divides the memory chain based on the slicing interface to obtain multiple slice data. The slicing interface can be the esar_rbuf_slice interface.

[0036] In this invention, esar_rbuf_slice is the "receive buffer data slice reading" interface that works in conjunction with esar_rbuf_pipe_append. It is responsible for cutting a segment of data from rbuf_pipe (receive buffer pipe) as needed for upper-layer applications to read. It is the core operation interface for "application-side reading" in the "protocol stack receiving data - application consuming data" link, and the "slice" in its name corresponds to the semantics of "data slicing / segmentation".

[0037] Step 103: The multiple slice data are converted using the format conversion interface of the fusion protocol stack to obtain target data in a preset format, and the target data is sent to the upper layer service.

[0038] In this invention, after obtaining multiple slice data, the fusion protocol stack can convert the multiple slice data based on the format conversion interface to obtain target data in a preset format, and then send the target data to the upper layer service.

[0039] In this invention, the format conversion interface can be the esar_rbuf_to_iovec interface, and the target data with the preset format can be data in an iov structure. SF-STACK uses the esar_rbuf_to_iovec interface to convert the multiple sliced ​​data obtained after splitting into an iov structure and pass it to the upper-layer business.

[0040] In this invention, `esar_rbuf_to_iovec` is a conversion / adaptation interface that maps "data in the receive buffer (rbuf)" to a standard `esar_iovec` structure. Its core function is to convert the memory distribution information of the rbuf (or the buffer in the rbuf_pipe) into an `esar_iovec` array (describing the "address + length" of the data) that can be directly used by upper-layer applications / modules. It acts as a "format bridge" between "buffer data" and "IO operations," perfectly aligning with SF-STACK's design philosophy of "unified interface, zero-copy."

[0041] In this invention, iov (full name I / O vector) is a core data structure in the operating system / network protocol stack used to describe "scattered / aggregated I / O". Its core function is to accurately describe one or more non-contiguous memory buffers through the combination of "address + length", so that I / O operations (such as reading and writing files, network transmission, DMA transmission) can directly operate on these buffers without first copying the scattered data to contiguous memory. It is the foundation for achieving high-performance, zero-copy I / O.

[0042] Step 104: Access the service data based on the target data through the upper-layer service.

[0043] In this invention, after the converged protocol stack sends the target data to the upper-layer service, the upper-layer service can access the service data based on the target data.

[0044] In this embodiment of the invention, the converged protocol stack reads memory management information of service data from the user-space protocol stack using a zero-copy interface; the converged protocol stack then segments the memory chain corresponding to the memory management information using a slicing interface to obtain multiple slice data; the converged protocol stack then converts the multiple slice data using a format conversion interface to obtain target data in a preset format, and sends the target data to the upper-layer service; the upper-layer service accesses the service data based on the target data. Thus, the converged protocol stack, using the zero-copy interface provided by the user-space protocol stack, receives the memory chain corresponding to the service data and sends it to the upper-layer service, enabling the upper-layer service to access the service data based on a zero-copy memory chain, thereby reducing the impact on performance and improving the user experience.

[0045] like Figure 2 The diagram illustrates a flowchart of another data management method provided by an embodiment of the present invention. The method may specifically include the following steps: Step 201: Receive service data from the network card through the user-space protocol stack, save the service data to a preset data structure, and generate memory management information corresponding to the service data.

[0046] In this invention, a user-space protocol stack can be used to receive service data from the network card. This service data can be service layer data, and the received service data can be saved to a preset data structure, for example, the service data can be saved to the tcp_seg data structure, thereby generating memory management information corresponding to the service data.

[0047] In this invention, tcp_seg (full name TCP Segment, TCP segment structure) is the core data structure in the TCP protocol stack used to carry and manage a single TCP segment. It runs through the entire TCP receive / send process, not only encapsulating TCP header information (such as sequence number, acknowledgment number, and flags), but also associating it with the segment's data buffer (such as mbuf), and is the fundamental carrier for the TCP protocol to implement "reliable transmission, flow control, and congestion control".

[0048] Step 202: Detect whether a read / write event has occurred in the user-space protocol stack by integrating the protocol stack.

[0049] In this invention, after the user-space protocol stack receives service data, a read / write event can be triggered, and the converged protocol stack can monitor in real time whether a read / write event has occurred in the user-space protocol stack.

[0050] Step 203: When the read / write event occurs in the user-space protocol stack, the memory management information is read from the user-space protocol stack through the zero-copy interface of the fusion protocol stack.

[0051] In this invention, the converged protocol stack can monitor in real time whether read / write events occur in the user-space protocol stack. When the converged protocol stack detects a read / write event in the user-space protocol stack, the converged protocol stack can read the memory management information of the service data from ESAR based on the esar_rbuf_pipe_append interface.

[0052] In this invention, a new custom data structure struct esar_iovec is added to read the payload memory address and release function information of ESAR. The memory address of the payload records the application layer's business data. In this invention, the application layer lava directly accesses the network card's buf space to obtain data. After use, the release function needs to be used to return the network card's buf memory.

[0053] In this invention, the specific assignment process of the data structure can be as follows: iov[cnt].iov_base = seg->buf; / / Pass the buf pointer of seg to the user iov[cnt].iov_len = seg->len; / / Pass the length of segbuf to the user iov[cnt].read.free_done = esar_iovec_free; / / Function for the user to free this seg buf iov[cnt].read.p1 = (void *)sock; / / User releases the input parameter 1 of the seg buf function iov[cnt].read.p2 = (void *)seg; / / User releases the input parameter 2 of the seg buf function. iov[cnt].read.ref = 0; / / Reference count of the seg buf page In this invention, SF-STACK can add an interface for parsing payload content. The struct esar_rbuf_pipe structure is used to manage the information of the payload buf of the specified socket. The information of the payload buf may include address, length and release function. It is mainly used to record the memory location information of the payload. After the application layer lava finishes using the data, it returns the memory for use.

[0054] SF-STACK can read the payload buf into a custom struct esar_rbuf_pipe data structure via the interface esar_rbuf_pipe_append using sock_zreadv. The static inline int esar_rbuf_pipe_append(struct esar_rbuf_pipe *pipe) function is used to read the payload buf memory data from ESAR and store it in a custom data structure.

[0055] In this invention, reading memory management information of service data from the user-space protocol stack via a zero-copy interface through a converged protocol stack can include: receiving service data from the network card via the user-space protocol stack and saving the service data to a preset data structure to generate memory management information corresponding to the service data; detecting whether a read / write event has occurred in the user-space protocol stack via the converged protocol stack; and when a read / write event occurs in the user-space protocol stack, reading memory management information from the user-space protocol stack via a zero-copy interface through the converged protocol stack.

[0056] Step 204: Generate the corresponding memory chain based on the memory management information using the fusion protocol stack.

[0057] In this invention, SF-STACK can generate a corresponding memory chain after obtaining memory management information.

[0058] Step 205: The memory chain is divided using the fusion protocol stack based on the slicing interface to obtain multiple slice data.

[0059] In this invention, SF-STACK can generate a corresponding memory chain after obtaining memory management information, and then SF-STACK can divide the memory chain into multiple slice data based on the esar_rbuf_slice interface.

[0060] In this invention, the memory chain corresponding to the memory management information is segmented by the fusion protocol stack based on the slicing interface to obtain multiple slice data, including: generating the corresponding memory chain based on the memory management information by the fusion protocol stack; and segmenting the memory chain based on the slicing interface by the fusion protocol stack to obtain multiple slice data.

[0061] In one embodiment, the memory chain is divided into multiple slice data by using the fusion protocol stack based on the slice interface. This may include: dividing the memory chain into multiple slice data according to a preset input / output length by using the fusion protocol stack based on the slice interface.

[0062] In this invention, the contents of the payload buf managed by struct esar_rbuf_pipe with preset input / output lengths can be split and unlinked through the esar_rbuf_slice interface.

[0063] In this invention, the segmentation and delinking process is an efficient data extraction mechanism in SF-STACK for "receiving data in chained storage". The core is "pointer-level segmentation + linked list-level delinking", which not only solves the need for "reading on demand", but also adapts to high-concurrency and low-latency scenarios through zero-copy and lock-free design.

[0064] The segmentation and off-chain processing is the underlying core logic of the esar_rbuf_slice interface and a key step in reassembling tcp_rcv_seg into rbuf. It connects the complete chain of "tcp_rcv_seg linked list - segmentation and off-chain - rbuf slice - application reading" and is an important technical support for SF-STACK to achieve "high throughput and low latency" data reception processing.

[0065] In this invention, the static inline int esar_rbuf_slice(struct esar_rbuf_pipe *pipe, struct esar_rbuf *rbuf, uint32_t size) function slices data of a specified length from the sock pipe into the rbuf receive buffer structure for management. For content that has already been read, the offset of iov in esar_rbuf_pipe is updated to record the iov information of the read data. If all iov has been read, the memory release process is performed.

[0066] In this invention, the payload content recorded in `struct esar_rbuf_peek` can be used to read specific data. The function `static inline void *esar_rbuf_peek(struct esar_rbuf *rbuf, uint32_t size, void *holder, int copy)` reads data from the structure of `esar_rbuf`, using a `copy` flag to distinguish whether to copy or pass a pointer. The `esar_rbuf_free` interface is used to release memory for used `esar_rbuf` slices. If the reference count of the memory page referenced by `esar_rbuf` is decremented to 0, the `free` interface in `iov` is called to perform the memory page release process. The function `static inline void esar_rbuf_free(struct esar_rbuf *rbuf)` releases the memory of the started `seg buf` after the `rbuf` is used.

[0067] In this invention, the `free` interface in `iov` is a dedicated interface for releasing resources related to the I / O vector (iov). Its core function is to reclaim the `iov` structure previously allocated through the stack interface, as well as the buffer memory it points to, thus preventing memory leaks.

[0068] Step 206: The multiple slice data are converted using the format conversion interface of the fusion protocol stack to obtain target data in a preset format, and the target data is sent to the upper layer service.

[0069] In this invention, after obtaining multiple slice data, the fusion protocol stack can convert the multiple slice data based on the format conversion interface to obtain target data in a preset format, and then send the target data to the upper layer service.

[0070] In this invention, the format conversion interface can be the esar_rbuf_to_iovec interface, and the target data with the preset format can be data in an iov structure. SF-STACK uses the esar_rbuf_to_iovec interface to convert the multiple sliced ​​data obtained after splitting into an iov structure and pass it to the upper-layer business.

[0071] In one embodiment, the slice data includes: memory address information and corresponding data length information; by converting multiple slice data through a format conversion interface using a fusion protocol stack to obtain target data in a preset format, the process may include: converting multiple memory address information and corresponding data length information through a format conversion interface using a fusion protocol stack to obtain target data in a preset format.

[0072] In this invention, SF-STACK can convert the memory address information and corresponding data length information in the sliced ​​data based on the esar_rbuf_to_iovec interface, and convert them into target data of the iov structure.

[0073] In this invention, the esar_rbuf_to_iovec interface converts the memory managed by esar_rbuf slices into a standard iov data structure. The static inline int esar_rbuf_to_iovec(struct esar_rbuf *rbuf, struct iovec *iov) function converts the payload buf segmented from esar_rbuf into the iov format required by lava.

[0074] Step 207: Access the service data based on the target data through the upper-layer service.

[0075] In this invention, after the converged protocol stack transmits the target data to the upper-layer service, the upper-layer service can access the service data based on the target data.

[0076] In one embodiment, the target data includes: multiple memory address information and corresponding data length information; accessing service data based on the target data through upper-layer services includes: accessing service data based on multiple memory address information and corresponding data length information through upper-layer services.

[0077] In this invention, after the upper-layer service receives the target data in a preset format sent by the converged protocol stack, it can obtain multiple memory address information and corresponding data length information from the target data in the preset format. Then, the upper-layer service can access the service data based on the multiple memory address information and corresponding data length information.

[0078] Step 208: After the upper-layer business finishes accessing the business data, the memory corresponding to the accessed business data is released through the pre-registered memory release interface.

[0079] In this invention, after the upper-layer service accesses the service data based on the target data, that is, after the upper-layer service has finished using the service data, it can release the memory corresponding to the accessed service data through a registered memory release interface. The memory release interface can be the esar_rbuf_free interface.

[0080] In this invention, esar_rbuf_free is the core interface for resource reclamation of the receive buffer (rbuf). Its core function is to release the rbuf and all its associated resources (such as the mbuf memory buffer, the memory described by esar_iovec, and the associated nodes of chained rbufs) to avoid memory leaks. It is a mandatory interface that must be called after "rbuf data consumption is completed" and is used in all scenarios where rbufs are used (such as the release of slices after esar_rbuf_slice splitting and the reclamation of independent rbufs).

[0081] In this invention, after the upper-layer service accesses the service data based on the target data, it may further include: after the upper-layer service finishes accessing the service data, releasing the memory corresponding to the accessed service data through a pre-registered memory release interface.

[0082] To better illustrate and understand the embodiments of the present invention, specific examples are used to explain the embodiments of the present invention. The application layer issues two I / O operations to read 100 bytes and 200 bytes of business data from the backend cluster, respectively. After the backend cluster completes its processing, it returns 100 bytes and 200 bytes of business data through the network. After the network card receives the 100 bytes and 200 bytes of business data from the backend, it saves them to two memory buf spaces in tcp_seg, namely buf[0] and buf[1]. The intermediate layer SF-STACK uses the esar_rbuf_pipe_append function to obtain the tcp_seg linked list data in the network card. The tcp_seg linked list is parsed to obtain the buf[0] information of the first I / O operation and is returned to the application layer lava for processing. After the lava completes its processing, buf[0] is released to the memory pool of the network card. The tcp_seg linked list is parsed to obtain the buf[1] information of the second I / O operation and is returned to the application layer lava for processing. After the lava completes its processing, buf[1] is released to the memory pool of the network card.

[0083] like Figure 3 The diagram illustrates a memory structure provided by an embodiment of the present invention. Figure 2 The diagram shows the memory structure after SF-STACK reads business data.

[0084] like Figure 4The diagram illustrates a flowchart of another data management method provided by an embodiment of the present invention. Service data is received from the network interface card (NIC) via the user-space protocol stack and saved to the tcp_seg data structure. Then, the intermediate layer SF-STACK uses the esar_rbuf_pipe_append function to read the current link's data iov (address and length) information from the tcp_seg linked list of the underlying NIC, parses the data stream on the NIC, performs io parsing according to application layer requirements, and saves the parsed iov in iov (address and length) format.

[0085] In this embodiment of the invention, the converged protocol stack reads memory management information of service data from the user-space protocol stack using a zero-copy interface; the converged protocol stack then segments the memory chain corresponding to the memory management information using a slicing interface to obtain multiple slice data; the converged protocol stack then converts the multiple slice data using a format conversion interface to obtain target data in a preset format, and sends the target data to the upper-layer service; the upper-layer service accesses the service data based on the target data. Thus, the converged protocol stack, using the zero-copy interface provided by the user-space protocol stack, receives the memory chain corresponding to the service data and sends it to the upper-layer service, enabling the upper-layer service to access the service data based on a zero-copy memory chain, thereby reducing the impact on performance and improving the user experience.

[0086] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0087] like Figure 5 The diagram shows a structural block diagram of a data management device provided in an embodiment of the present invention. The device may specifically include the following modules: The reading module 501 is used to read memory management information of business data from the user-space protocol stack via the zero-copy interface of the converged protocol stack; The segmentation module 502 is used to segment the memory chain corresponding to the memory management information through the fusion protocol stack based on the segmentation interface to obtain multiple slice data; The conversion module 503 is used to convert the multiple slice data through the fusion protocol stack based on the format conversion interface to obtain target data in a preset format, and send the target data to the upper layer service; Access module 504 is used to access the service data based on the target data through the upper-layer service.

[0088] In one embodiment, the reading module 501 includes: The receiving submodule is used to receive service data from the network card through the user-mode protocol stack, save the service data to a preset data structure, and generate memory management information corresponding to the service data; The detection submodule is used to detect whether a read / write event has occurred in the user-space protocol stack through the fusion protocol stack; The reading submodule is used to read the memory management information from the user-space protocol stack through the fusion protocol stack based on the zero-copy interface when the read / write event occurs in the user-space protocol stack.

[0089] In one embodiment, the segmentation module 502 includes: A generation submodule is used to generate a corresponding memory chain based on the memory management information through the fusion protocol stack; The slicing module is used to slice the memory chain based on the slicing interface through the fusion protocol stack to obtain multiple slice data.

[0090] In one embodiment, the molecule-cutting module includes: The segmentation unit is used to segment the memory chain according to a preset input / output length based on the fusion protocol stack and the slicing interface to obtain multiple slice data.

[0091] In one embodiment, the slice data includes: memory address information and corresponding data length information; the conversion module 503 includes: The conversion submodule is used to convert multiple memory address information and corresponding data length information through the format conversion interface of the fusion protocol stack to obtain target data in a preset format.

[0092] In one embodiment, the target data includes: multiple memory address information and corresponding data length information; the access module 504 includes: The access submodule is used to access the service data based on the multiple memory address information and the corresponding data length information through the upper-layer service.

[0093] In one embodiment, the device further includes: The release module is used to release the memory corresponding to the accessed service data through a pre-registered memory release interface after the upper-layer service has finished accessing the service data.

[0094] In this embodiment of the invention, a reading module is used to read memory management information of service data from the user-space protocol stack via a zero-copy interface of the converged protocol stack; a segmentation module is used to segment the memory chain corresponding to the memory management information via a slicing interface of the converged protocol stack to obtain multiple slice data; a conversion module is used to convert the multiple slice data via a format conversion interface of the converged protocol stack to obtain target data in a preset format, and send the target data to the upper-layer service; and an access module is used to access the service data based on the target data through the upper-layer service. Thus, the converged protocol stack, based on the zero-copy interface provided by the user-space protocol stack, receives the memory chain corresponding to the service data and sends it to the upper-layer service, enabling the upper-layer service to access the service data based on zero-copy memory chain access, thereby reducing the impact on performance and improving the user experience.

[0095] As the apparatus embodiment is basically similar to the method embodiment, it is described in a relatively simple manner. For relevant details, please refer to the description of the method embodiment.

[0096] This invention also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described data management method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0097] It should be noted that the electronic devices in the embodiments of the present invention include mobile electronic devices and non-mobile electronic devices.

[0098] This invention also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described data management method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0099] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0100] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0106] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0107] The present invention has provided a detailed description of a data management method, a data management device, an electronic device, and a readable storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data management method, characterized in that, The method includes: By integrating the protocol stack and using a zero-copy interface, memory management information of business data is read from the user-space protocol stack. The memory chain corresponding to the memory management information is segmented using the fusion protocol stack based on the slicing interface to obtain multiple slice data. The multiple slice data are converted using the format conversion interface through the fusion protocol stack to obtain target data in a preset format, and the target data is then sent to the upper-layer service. The upper-layer service accesses the service data based on the target data.

2. The data management method according to claim 1, characterized in that, The process of reading service data from the user-space protocol stack via a zero-copy interface through the converged protocol stack includes: The system receives service data from the network card through the user-mode protocol stack, saves the service data to a preset data structure, and generates memory management information corresponding to the service data. By integrating the protocol stack, it is possible to detect whether a read / write event has occurred in the user-space protocol stack; When the read / write event occurs in the user-space protocol stack, the memory management information is read from the user-space protocol stack through the zero-copy interface of the fusion protocol stack.

3. The data management method according to claim 1, characterized in that, The process involves segmenting the memory chain corresponding to the memory management information using the fusion protocol stack based on the slicing interface to obtain multiple slice data, including: Based on the memory management information, the fusion protocol stack generates a corresponding memory chain; The memory chain is divided into multiple slice data by using the fusion protocol stack based on the slice interface.

4. The data management method according to claim 3, characterized in that, The process involves segmenting the memory chain using the fusion protocol stack based on the slicing interface to obtain multiple slice data, including: The memory chain is divided into multiple slice data by means of the fusion protocol stack based on the slice interface and according to the preset input / output length.

5. The data management method according to claim 1, characterized in that, The slice data includes: memory address information and corresponding data length information; the conversion of the multiple slice data through the fusion protocol stack based on the format conversion interface to obtain target data in a preset format includes: The fusion protocol stack uses a format conversion interface to convert multiple memory address information and corresponding data length information to obtain target data in a preset format.

6. The data management method according to claim 1, characterized in that, The target data includes: multiple memory address information and corresponding data length information; accessing the service data based on the target data through the upper-layer service includes: The upper-layer service accesses the service data based on the multiple memory address information and the corresponding data length information.

7. The data management method according to claim 1, characterized in that, After accessing the service data based on the target data through the upper-layer service, the method further includes: After the upper-layer service finishes accessing the service data, the memory corresponding to the accessed service data is released through a pre-registered memory release interface.

8. A data management device, characterized in that, The device includes: The read module is used to read memory management information of business data from the user-space protocol stack via the zero-copy interface of the converged protocol stack; The segmentation module is used to segment the memory chain corresponding to the memory management information through the fusion protocol stack based on the segmentation interface to obtain multiple slice data; The conversion module is used to convert the multiple slice data through the format conversion interface of the fusion protocol stack to obtain target data with a preset structure, and send the target data to the upper layer service; The access module is used to access the business data based on the target data through the upper-layer business.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the data management method as described in claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the data management method as described in claims 1-7.

Images