Data transmission method and device

Abstract

The embodiment of the present application provides a data transmission method and device, including: in response to a first read request, obtaining the size of the space occupied by the target file, and determining the target first memory area required to store the target file; generating a second read request according to the address information of the target first memory area and the file storage parameters corresponding to the target first memory area, and sending the second read request to the server, and receiving the partial content of the target file sent by the server through the target first memory area. The present application saves the system overhead and time consumption of memory registration at each response, and the memory management of the user device and the server based on the memory pool can realize the use of memory according to the amount, which greatly improves the utilization efficiency of the memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data replication, which has a positive effect on reducing system overhead.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data transmission method, device, distributed data synchronization system, distributed computing system, electronic device, and machine-readable medium. Background Art

[0002] Data transmission between ends has always been an indispensable link in realizing complex application interaction functions.

[0003] In the related art, the file transfer process between the sender and the receiver requires the active participation of the kernel processor at each end and the user system running in user mode, and a large amount of cache space is needed to transfer data during the transmission process to realize the conversion of the memory address of the data.

[0004] However, the inventors have found through research that the current solution places high pressure on the core processor and cache and has a large system overhead. Summary of the invention

[0005] The embodiment of the present application provides a data transmission method to solve the problems of high pressure on the core processor and cache and large system overhead in the related art.

[0006] Correspondingly, the embodiments of the present application also provide a data transmission device, a distributed data synchronization system, a distributed computing system, an electronic device, and a machine-readable medium to ensure the implementation and application of the above method.

[0007] In order to solve the above problem, an embodiment of the present application discloses a data transmission method, which is applied to a user device, wherein a plurality of first memory slices are set in a local memory of the user device, and the first memory slices have the same capacity, including:

[0008] In response to a first read request, the size of the space occupied by the target file is obtained, and according to the size of the space occupied by the target file and the capacity of the first memory slice, the number of target first memory slices required to store the target file is determined; the target first memory slice is used to store the local content of the target file;

[0009] Generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice, the identifier of the target file, and the file storage parameter corresponding to the target first memory slice, and send the second read request to the server, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice;

[0010] The local content of the target file sent by the server is received through the target first memory area.

[0011] The embodiment of the present application discloses a data transmission method, which is applied to a user device, wherein a plurality of first memory slices are set in a local memory of the user device, and the method includes:

[0012] In response to a first read request, the size of the space occupied by the target file is obtained, and according to the size of the space occupied by the target file and the capacity of the first memory slice, a target first memory slice required to store the target file is determined; the target first memory slice is used to store the local content of the target file;

[0013] Generate a second read request corresponding to the target first memory slice and send it to the server according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice;

[0014] The local content of the target file sent by the server is received through the target first memory area.

[0015] The embodiment of the present application discloses a data transmission method, which is applied to a server, wherein the server has a plurality of second memory slices locally, and the method comprises:

[0016] Acquire a second read request sent by the user device, the second read request comprising: address information of a target first memory slice and a file storage parameter corresponding to the target first memory slice; the target first memory slice is used to store local content of a target file; the file storage parameter is used to characterize features of the local content to be stored in the target first memory slice;

[0017] Determine the storage location of the target file locally on the server, and extract the local content of the target file from the storage location and copy it to a target second memory slice according to the file storage parameter, where the target second memory slice is any second memory slice;

[0018] According to the address information of the target first memory area, the local content stored in the target second memory area is sent to the target first memory area of ​​the user device for storage.

[0019] Optionally, extracting the local content of the target file from the storage location and copying it to the target second memory area according to the file storage parameter includes:

[0020] Determining a local position of the local content in the storage location according to the file storage parameter;

[0021] According to the local position, the local content is extracted and copied to the target memory area.

[0022] Optionally, the file storage parameter includes: at least one of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, the file offset parameter is used to characterize the starting position of the local content, and the length parameter is used to characterize the content length of the local content; determining the local position of the local content in the storage position according to the file storage parameter includes:

[0023] The local position of the local content in the storage location is determined according to the file offset parameter and / or the length parameter.

[0024] Optionally, extracting the local content of the target file from the storage location and copying it to the target second memory area according to the file storage parameter, the target second memory area being at least one second memory area, includes: sending a third read request to the virtual file system of the server through the second process, the third read request including the file storage parameter, so that the virtual file system responds to the third read request, calls the hard disk input / output interface to extract the local content of the target file from the storage location, and copies the local content to the target second memory area.

[0025] Optionally, the method further comprises:

[0026] In response to a memory registration request for a second process running on the user device, a plurality of second memory slices associated with the second process are set in the local memory of the server, and the second memory slices are used for use by the second process.

[0027] Optionally, sending a third read request to the virtual file system of the server through the second process includes:

[0028] The third read request is sent to the virtual file system of the server by calling the direct read-write interface through the second process.

[0029] The embodiment of the present application discloses a data transmission device, which is applied to a user equipment, wherein a plurality of first memory slices are set in a local memory of the user equipment, and the device includes:

[0030] A first setting module is used to obtain the occupied space size of the target file in response to the first read request, and determine the target first memory slice required to store the target file according to the occupied space size of the target file and the capacity of the first memory slice; the target first memory slice is used to store the local content of the target file;

[0031] A generating module, configured to generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and a file storage parameter corresponding to the target first memory slice, and send the second read request to the server, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice;

[0032] The storage module is used to receive the partial content of the target file sent by the server through the target first memory area.

[0033] The embodiment of the present application discloses a data transmission device, which is applied to a server, wherein the server has a plurality of second memory slices locally, and the device comprises:

[0034] A receiving module, configured to obtain a second read request sent by a user device, wherein the second read request includes: address information of a target first memory slice and a file storage parameter corresponding to the target first memory slice; the target first memory slice is used to store local content of the target file; and the file storage parameter is used to characterize features of the local content to be stored in the target first memory slice;

[0035] an extraction module, configured to determine a storage location of the target file locally on the server according to an identifier of the target file, and to extract a local content of the target file from the storage location and copy it to a target second memory slice according to the file storage parameter, wherein the target second memory slice is any second memory slice;

[0036] The sending module is used to send the local content stored in the target second memory area to the target first memory area of ​​the user device for storage according to the address information of the target first memory area.

[0037] The embodiment of the present application discloses a distributed data synchronization system, including: at least one upload node and at least one data synchronization node; a plurality of first memory slices are set in the local memory of the data synchronization node; a plurality of second memory slices are locally provided in the upload node;

[0038] The upload node is used to obtain the target file to be synchronized and backed up, and send the occupied space size of the target file to the data synchronization node; obtain the backup request sent by the data synchronization node; according to the backup request, extract the local content of the target file and copy it to the target second memory slice area, and the target second memory slice area is any second memory slice area; according to the address information of the target second memory slice area, send the local content stored in the target second memory slice area to the target first memory slice area of ​​the data synchronization node for storage;

[0039] The data synchronization node is used to determine the target first memory slice required to store the target file based on the size of the space occupied by the target file and the capacity of the first memory slice; generate a backup request corresponding to the target first memory slice based on the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice, and send it to the upload node, wherein the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory slice.

[0040] The embodiment of the present application discloses a distributed computing system, including: at least one master computing node and at least one slave computing node; the master computing node has a plurality of first memory slices in its local memory, and the slave computing node has a plurality of second memory slices in its local memory;

[0041] The master computing node is used to obtain a computing task, split the computing task into multiple subtasks and send the subtasks to the slave computing node; determine the target first memory slice required to store the computing result based on the size of the space occupied by the computing result of the subtask and the capacity of the first memory slice; generate a synchronization request corresponding to the target first memory slice based on the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice, and send it to the slave computing node, wherein the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory slice.

[0042] The slave computing node is used to obtain the subtask for calculation, and send the occupied space size of the calculation result to the master computing node; obtain the synchronization request sent by the master computing node; according to the synchronization request, extract the local content of the calculation result and copy it to the target second memory slice area, and the target second memory slice area is any second memory slice area; according to the address information of the target second memory slice area, send the local content stored in the target second memory slice area to the target first memory slice area of ​​the master computing node for storage.

[0043] An embodiment of the present application further discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes one or more methods described in the embodiments of the present application.

[0044] The embodiments of the present application also disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0045] Compared with the related art, the embodiments of the present application have the following advantages:

[0046] In an embodiment of the present application, before starting the transmission process, multiple memory slices are registered at one time in the respective memories of the server and the user device for use, so that after responding to the read request of the user device, the server does not need to perform a memory registration operation, but selects the registered memory slice for use based on the memory management of the memory pool, and divides the file to be transmitted into slices aligned with the memory slice and stores them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration at each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a system architecture diagram of a data transmission process in an embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of a file downloading scenario in an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of a distributed data synchronization system according to an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a distributed computing system according to an embodiment of the present application;

[0051] Figure 5 is a flowchart of a data transmission method according to an embodiment of the present application;

[0052] Figure 6 is a flowchart of another data transmission method according to an embodiment of the present application;

[0053] Figure 7 It is a flowchart of the specific steps of a data transmission method according to an embodiment of the present application;

[0054] Figure 8 is a flowchart of specific steps of another data transmission method according to an embodiment of the present application;

[0055] Fig. 9 is a block diagram of a distributed data synchronization system according to an embodiment of the present application;

[0056] Fig.10 is a block diagram of a distributed computing system according to an embodiment of the present application;

[0057] Fig.11is a block diagram of a data transmission device according to an embodiment of the present application;

[0058] Fig.12 is a block diagram of another data transmission device according to an embodiment of the present application;

[0059] Fig.13 It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0061] In order to enable those skilled in the art to better understand the present application, the concepts involved in the present application are explained below:

[0062] Remote Direct Memory Access (RDMA) is a technology that can transfer data directly from the memory of one computer to the memory of another computer over a network without any impact on the operating system. By directly transferring data between application memories, the data copying process between application memory and operating system cache is avoided. This type of transmission does not require the processor and context switching overhead, freeing up memory bandwidth and processors, and can effectively improve application system performance.

[0063] RDMA write: A computer writes data in its local memory directly to the local memory of another computer.

[0064] Remote direct data access device: A device that uses RDMA technology and can be in the form of a network card.

[0065] Memory slice: A local slice of the local memory of a server or user device. A memory slice is used to store a portion of a larger file or a smaller file in its entirety. A memory slice may also have a corresponding address, through which the server or user device can locate the memory slice.

[0066] Page Cache: A memory area managed by the kernel of a server or user device that caches the logical contents of files and optimizes the way data is read, thereby speeding up access to images and data on disk.

[0067] File storage parameters: When slicing and transmitting a larger file, in order to ensure the continuity between each file slice (partial content of the file), the file slice can be defined by file storage parameters. Specifically, the file storage parameters include: file offset parameter and length parameter. The file offset parameter is used to represent the starting position of the file slice, and the length parameter is used to represent the content length of the file slice.

[0068] Memory registration: The processes in the server and user devices can register the memory slices they need to use in the operating system, and the registered memory slices can be bound to the process.

[0069] Direct I / O: A reading method that bypasses the file write buffer and read cache. When a process reads or writes system files, calling the direct I / O interface can bypass the system's file write buffer and read cache to directly access the memory.

[0070] Virtual File System (VFS): It can read and write different file systems located on different physical media using standard read and write methods, that is, it provides a unified operating interface and application programming interface for various file systems, so that system calls such as reading and writing do not need to care about the underlying storage media and file system type.

[0071] Fourth extended filesystem (EXT4): The file system of the kernel operating system is the underlying file system provided by the kernel and directly connected to the hard disk input / output interface.

[0072] The data transmission method in the embodiment of the present application can be specifically applied to the data transmission scenario based on RDMA between the user device and the server. In a related technical process, when the server receives a file acquisition request from the user device, it can register a memory slice matching the file size locally on the server, and read the file data from the local memory of the server into the Page Cache of the kernel layer. After that, the application process running in the user state in the server extracts the file data from the Page Cache and stores the file data in the registered memory slice. Finally, the server transmits the file data in the memory slice directly to the memory slice of the user device through the network through the RDMA network card to complete the file transmission.

[0073] According to the above implementation process, it is possible to directly transfer files from the memory slice of a server to the memory slice of another user device through the network, thereby reducing the pressure and system overhead of the processors in the server and the user device, and saving the data copying process between the system caches. However, there are still two problems with the above implementation process: 1. Every time the server responds to a file acquisition request, it needs to perform memory registration, and memory registration naturally has the problem of high system overhead and time consumption, which reduces the response speed, and each registration requires the occupation of a new memory slice, resulting in memory waste. 2. In the above implementation process, the server needs to read the file data in the local memory into the Page Cache of the kernel layer, and then store the file data in the Page Cache in the registered memory slice. Its essence is to transfer the file data from one slice of memory to another. This process actually still generates a large memory usage cost.

[0074] In order to solve the above-mentioned problem 1, the embodiment of the present application can register multiple memory slices with the same capacity in the respective memories of the server and the user device at one time before starting the transmission process, thereby forming memory pools in the server and the user device in advance, so that after the server responds to the read request of the user device, it is not necessary to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through RDMA technology, thereby saving the system overhead and time of memory registration for each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory.

[0075] It should be noted that, preferably, the capacity of the memory slices registered by the server and the user device can be the same; in another implementation, the capacity of the memory slices registered by the server and the user device can be different. When the capacity of the memory slices registered by the server and the user device are different, the server and the user device can record the capacity of the memory slices registered by each of them. When using the memory slices, the server and the user device can select the memory slice to be used for transmitting the file from the registered memory slices based on the capacity of the recorded memory slice and the size of the file to be transmitted.

[0076] In order to solve the above-mentioned problem 2, the embodiment of the present application can save the data transfer of Page Cache. The server can bypass Page Cache and directly store the file data in the local memory in the registered memory area, thereby reducing the number of times the data is copied in the memory during the file transfer process, further reducing the memory usage cost and improving the transmission efficiency.

[0077] Reference Figure 1 , which shows a system architecture diagram of a data transmission process provided by an embodiment of the present application, including: a server and a user device. The embodiment of the present application specifically describes the data transmission between the server and the user device through the user layer, the kernel layer, and the hardware layer. The user layer is built based on the user state, and the kernel layer is built based on the kernel state. The user layer is the activity space of the upper-layer application. The execution of the application must rely on the resources provided by the kernel layer. Therefore, the kernel layer controls the hardware resources of the computer and provides an environment for the operation of the upper-layer application. The application running in the kernel layer can access the computer resources in the kernel layer through system calls. The physical computer resources of the server and the user device are the hardware layer, such as hard disks, network cards, etc.

[0078] In the present application, refer to Figure 1 Before starting the file transfer, the user device may perform a memory registration once, register m first memory slices for the first process to form a memory pool, the memory pool registered by the first process includes the first memory slice 1, the first memory slice 2, the first memory slice 3...the first memory slice m, preferably, the first memory slice 1, the first memory slice 2, the first memory slice 3...the first memory slice m have the same capacity, and these first memory slices can be used by the first process. Similarly, the server may also perform a memory registration once in advance, register n second memory slices for the second process to form a memory pool, the memory pool registered by the second process includes the second memory slice 1, the second memory slice 2, the second memory slice 3...the second memory slice n, preferably, the second memory slice 1, the second memory slice 2, the second memory slice 3...the second memory slice n have the same capacity as the first memory slice, and these second memory slices can be used by the second process, wherein the embodiment of the present application does not limit the size of m and n, and m and n may be the same or different. In addition, before starting the file transfer, the server also sends information about the file stored locally on the server (such as file identification and occupied space) to the user device for the user to know.

[0079] When the user device needs to obtain a target file stored in the server for use by the first process, the user device can respond to the first process's first read request for the target file, obtain the occupied space size and identifier of the target file, and determine the number of target first memory slices required to store the target file, that is, the sum of the capacities of the target first memory slices must be greater than or equal to the occupied space size of the target file. After that, the user device can generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice, the identifier of the target file, and the file storage parameters corresponding to the target first memory slice, that is, a second read request is generated for each target first memory slice, so that a second read request is used to request to obtain a file slice (local content) of the target file. Specifically, the file storage parameters in the second read request can define the starting position and content length of the file slice. In the case where the number of target first memory slices is multiple, the user device will obtain a request list of second read requests arranged in sequence, and the user device can send the request list of second read requests to the server. It should be noted that in one implementation, the user device can send the second read requests in the request list to the server in sequence; in another implementation, the user device can send the second read requests in the request list to the server at the same time.

[0080] After obtaining the request list of the second read request, the server can start processing from the first second read request, that is, refer to Figure 1, for each second read request, specifically through the identifier of the target file in the second read request, determine the storage location of the target file in the local hard disk of the server, and call the direct I / O interface of the kernel layer, bypass the Page Cache and EXT4 of the kernel layer to directly access the hard disk input / output interface, slice the target file in the hard disk according to the file storage parameter in the second read request to obtain the corresponding file slice, and copy the file slice to a target second memory area (such as the second memory area 3), and finally the remote direct data access device (RDMA network card) of the server can use the RDMA technology RDMA write, directly slice the file in the target second memory area, and transmit it to the remote direct data access device of the user device through the Internet. The remote direct data access device of the user device can then write the file slice into the target first memory area according to the address information of the target first memory area (such as the first memory area 3) contained in the second read request, until the server transfers all the file slices of the target file to the corresponding target first memory area of ​​the user device, and completes the acquisition of the target file by the user device. At this time, the target file is stored in the target first memory area of ​​the first process, and the target file can be called and used by the first process, such as the first process can transfer the target file from the target first memory area to the local hard disk of the user device to achieve persistent storage. After persistent storage, the memory pool of the user device can release the space of the target first memory area, so as to achieve the release and reuse of memory, which greatly improves the utilization efficiency of memory. In addition, after the server sends the target file, the memory pool of the server can release the space of the target second memory area, which also achieves the release and reuse of memory.

[0081] For the entire process described above, an embodiment of the present application registers multiple memory slices of the same capacity in the respective memories of the server and the user device for use at one time before starting the transmission process, without the need to perform memory registration operations for each response, thereby saving the system overhead and time consumption of memory registration for each response, and the memory management of the server and the user device based on the memory pool can improve the utilization efficiency of the memory.

[0082] It should be noted that, based on the data transmission method provided in the embodiment of the present application, several specific scenarios that can be implemented are as follows:

[0083] In one implementation, referring to Figure 2, which shows a schematic diagram of a file downloading scenario provided by an embodiment of the present application, including: a user device and a server, in which a plurality of first memory slices can be registered in advance for a first process to form a memory pool in the user device; and a plurality of second memory slices can be registered in advance for a second process to form a memory pool in the server. Specifically, the user device can obtain the occupied space size and identification of the file to be downloaded from the server, and generate a request list of a second read request and send it to the server in the manner described in the above embodiment. The server can process the request list of the second read request in the manner described in the above embodiment, and sequentially transfer each file slice of the file to be downloaded by the user device from the memory of the user device to the registered second memory slice, and send the file slice directly from the second memory slice to the first memory slice of the user device through the RDMA method. In this way, data transmission can be carried out continuously to complete the transmission of all the data of the file, so that the user device can obtain the file downloaded from the server, and the downloaded file can be called and used by the first process of the user device.

[0084] In another implementation, referring to Figure 3 , which shows a scenario diagram of a distributed data synchronization system provided by an embodiment of the present application, the distributed data synchronization system includes: an upload node and a data synchronization node, the upload node can receive and store files uploaded by a user device, and the data synchronization node is intended to obtain the file from the upload node to realize data backup synchronization of multiple nodes. Among them, multiple second memory slices can be registered in advance in the upload node to form a memory pool; multiple first memory slices can be registered in advance in the data synchronization node to form a memory pool. Specifically, the upload node is used to obtain the target file to be synchronized and backed up, and send the occupied space size of the target file to the data synchronization node; then the data synchronization node can generate a backup request list in accordance with the method of generating a request list of the second read request described in the above embodiment, and send it to the upload node, so that the upload node can process the backup request list in accordance with the method of processing the request list of the second read request described in the above embodiment, and sequentially transfer each file slice of the target file to be backed up from the memory of the upload node to the registered second memory slice, and send the file slice directly from the second memory slice to the first memory slice of the data synchronization node through the RDMA method. The transmission of all the data of the file can be realized in this way, thereby realizing the purpose of synchronizing and backing up a file uploaded by a user device at the data synchronization node of the distributed data synchronization system. It should be noted that in the embodiment of the present application, the upload node can transmit the target file to be backed up to each data synchronization node separately, or the upload node can transmit the target file to be backed up to a data synchronization node, and then the data synchronization node transmits the target file to the next data synchronization node, and so on until all data synchronization nodes obtain the target file.

[0085] In another implementation, referring to Figure 4 , which shows a scenario diagram of a distributed computing system provided by an embodiment of the present application, the distributed data synchronization system includes: a slave computing node and a master computing node, the master computing node is used to obtain computing tasks, split the computing tasks into multiple subtasks and send the subtasks to the slave computing node for collaborative computing. Among them, multiple second memory slices can be registered in advance in the slave computing node to form a memory pool; multiple first memory slices can be registered in advance in the master computing node to form a memory pool. Specifically, the slave computing node is used to obtain subtasks for computing, and the identifier and occupied space size of the computing results are sent to the master computing node; then the master computing node can generate a synchronization request list in accordance with the method of generating a request list for the second read request described in the above embodiment, and send it to the slave computing node, so that the slave computing node can process the synchronization request list in accordance with the method of processing the request list for the second read request described in the above embodiment, and sequentially transfer each file slice of the computing result to be synchronized from the memory of the slave computing node to the registered six memory slices, and send the file slices directly from the second memory slice to the first memory slice of the master computing node through the RDMA method, thereby realizing the splitting and collaborative computing of a larger computing task in the distributed computing system, and the purpose of summarizing the computing results in the master computing node.

[0086] It should be noted that in the embodiments of the present application, the process of obtaining the occupied space size and identification of the target file, the address information of the memory area, the file storage parameters, and other information, signals or data used is carried out in compliance with the relevant data protection laws and policies of the country where it is located, and with the authorization given by the owner of the corresponding device.

[0087] In an embodiment of the present application, before starting the transmission process, multiple memory slices with the same capacity are registered in the respective memories of the server and the user device for use at one time, so that after responding to the read request of the user device, the server does not need to perform a memory registration operation, but selects the registered memory slice for use based on the memory management of the memory pool, and divides the file to be transmitted into slices aligned with the memory slice and stores them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration at each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0088] Reference Figure 5, which shows a step flow chart of a data transmission method provided by an embodiment of the present application, which is applied to a user device, wherein a plurality of first memory slices are set in a local memory of the user device, and the method comprises:

[0089] Step 101, in response to a first read request, obtain the occupied space size of a target file, and determine the number of target first memory slices required to store the target file according to the occupied space size of the target file and the capacity of the first memory slice.

[0090] The target first memory area is used to store the local content of the target file.

[0091] In the embodiment of the present application, the first read request may be a request initiated by a first process in a user layer of a user device, such as a file download request, a file read request, etc. By responding to the first read request, the user device may obtain the size of the space occupied by the target file, and the size of the space occupied by the target file may be information provided by a server storing the target file, or may be information agreed upon in advance between the server and the user device and stored locally on the user device.

[0092] Furthermore, since a plurality of first memory slices are provided in the local memory of the user device to form a memory pool, preferably, the capacities of the first memory slices are the same, the user device can implement memory management of the memory pool, that is, when responding to the first read request, the number of target first memory slices required to store the target file is determined according to the size of the space occupied by the target file and the capacity of the first memory slice, thereby achieving the purpose of using memory in quantity and reducing the probability of excessive or insufficient memory required for subsequent file storage.

[0093] For example, if a target file is 4 MB in size and a first memory segment is 1 MB in size, then the number of target first memory segments required is 4.

[0094] Step 102: Generate a second read request corresponding to the target first memory area according to the address information of the target first memory area and the file storage parameters corresponding to the target first memory area and send the second read request to the server.

[0095] The file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory area.

[0096] In this step, the user device can generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice, the identifier of the target file, and the file storage parameters corresponding to the target first memory slice. The identifier of the target file is used for the server to find the target file, that is, the embodiment of the present application generates a second read request for each target first memory slice, so that a second read request is used to request to obtain a file slice (local content) of the target file. The number of second read requests can be the same as the number of target first memory slices. For example, when the number of target first memory slices is 4, 4 second read requests can be generated in sequence to form a request list. In addition, the address information of the memory slice can uniquely define the location of the memory slice in the server and the user device, such as the address information of a memory slice can be 0x01ab00a0.

[0097] Of course, in the embodiment of the present application, other features of the target file can also be used by the server to search for the target file. For example, the hash value of the target file can be calculated, and then the hash value of the target file is provided to the server, and the server searches for the file based on the hash value.

[0098] Step 103: receiving the partial content of the target file sent by the server through the target first memory area.

[0099] In the embodiment of the present application, since the server adopts the RDMA method, the file slices in the local target second memory area of ​​the server are directly transmitted to the local target first memory area of ​​the user device through the Internet. That is, through the RDMA technology, the participation of the kernel processor of each terminal in the data transmission process and the data copy pressure of the cache of each terminal are reduced, which greatly reduces the system overhead and improves the system performance.

[0100] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the server and the user device for use at one time before starting the transmission process, so that after the server responds to the read request of the user device, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration for each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0101] optionally optionally optionally

[0102] Reference Figure 6 , which shows a flow chart of the steps of a data transmission method provided by an embodiment of the present application, including a server of a remote direct data access device, the server having a plurality of second memory slices locally, the method comprising:

[0103] Step 201: Acquire a second read request sent by a user device.

[0104] Among them, the second read request may include: address information of the target first memory area, and file storage parameters corresponding to the target first memory area; the target first memory area is used to store local content of the target file; and the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory area.

[0105] The current second read request may also include an identifier of the target file, or a hash value of the target file or other information that can uniquely identify the target file, and then the storage location of the target file in the server may be searched based on the above information.

[0106] In an embodiment of the present application, the user device may send a request list of second read requests consisting of second read requests arranged in sequence to the server, and the server may split the request list of second read requests and obtain each second read request in sequence for processing.

[0107] Step 202, determine the storage location of the target file locally on the server, and extract the local content of the target file from the storage location and copy it to the target second memory area according to the file storage parameters, the target second memory area being any second memory area.

[0108] In this step, preferably, the capacity of the second memory slice is the same as the capacity of the first memory slice locally segmented by the user device. For each second read request, the server determines the storage location of the target file on the server's local hard disk by specifically identifying the target file in the second read request, and calls the direct I / O interface of the kernel layer to slice the target file in the hard disk according to the file storage parameters in the second read request to obtain corresponding file slices, and copies the file slices to a target second memory slice.

[0109] In the embodiment of the present application, when the capacity of the second memory slice is the same as the capacity of the second memory slice, the calculation amount of the server selecting the second memory slice is simple. Of course, during the selection process, if at least part of the second memory slice is different, at this time, according to the storage parameter and the capacity of the second memory slice, the minimum number of target second memory slices that can meet the storage parameter can be selected, and then the corresponding local content can be read based on the above target second memory slice.

[0110] Optionally, in step 203, the local content stored in the target second memory area is sent to the target first memory area of ​​the user device for storage according to the address information of the target first memory area through the remote direct data access device.

[0111] Finally, the server's remote direct data access device (RDMA network card) can use the RDMAwrite operation of the RDMA technology to directly transmit the file slices in the target second memory area to the remote direct data access device of the user device through the Internet. The remote direct data access device of the user device can then write the file slices into the target first memory area according to the address information of the target first memory area contained in the second read request, until the server transfers all file slices of the target file to the corresponding target first memory area of ​​the user device, thereby completing the user device's acquisition of the target file.

[0112] Of course, if an interruption occurs during the transmission process, a breakpoint resume mechanism can be used. When the transmission process is resumed, data transmission can continue from the interrupted position. This application does not impose any restrictions on this.

[0113] Optionally, multiple second memory slices associated with the second process are optionally set. In summary, an embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the server and the user device for use before starting the transmission process, so that after the server responds to the read request of the user device, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through RDMA technology, thereby saving the system overhead and time of memory registration for each response, and the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0114] Reference Figure 7 , which shows a specific step flow chart of a data transmission method provided by an embodiment of the present application, which is applied to a user device, wherein a plurality of first memory slices are set in a local memory of the user device, and the method includes:

[0115] Step 301: In response to a first read request, obtain the size of the space occupied by the target file, and determine the number of slices of the required first memory slice according to the size of the space occupied by the target file and the capacity of the first memory slice.

[0116] In this step, preferably, the capacity of each first memory slice is the same. When the occupied space size of the target file and the capacity of the first memory slice are known, the ratio of the occupied space size of the target file and the capacity of the first memory slice can be used as the number of slices of the required first memory slice, wherein, if the ratio has digits after the decimal point, the ratio can be rounded back to obtain the number of slices, such as assuming that the ratio is 3.1, the number of slices obtained by rounding back is 4. Since the capacity of each first memory slice is the same, the number of first memory slices can be directly calculated in the subsequent processing process without considering the capacity of each memory slice separately, which is simple to calculate and efficient.

[0117] Of course, if at least part of the capacity of each first memory slice is different, when determining the target first memory slice, the target first memory slice that can meet the minimum number of slices for storing the target file can be selected based on the size of the space occupied by the target file and the capacity of each first memory slice, and then the subsequent file transfer process is performed based on these target first memory slices. Compared with the case where the capacity of each first memory slice is the same, the calculation process of this process is relatively complicated.

[0118] In another implementation, the capacity of the first memory slice may also be different. When the capacity of the first memory slice is different, the user device can record the capacity of the registered first memory slice. When the first memory slice is used specifically, the user device can select the target first memory slice to be used for transmitting the file from the registered first memory slices based on the recorded capacity of the first memory slice and the size of the file to be transferred.

[0119] Step 302: According to the number of slices, select target first memory slices of the number of slices from the first memory slices.

[0120] Step 303: Generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice.

[0121] In an embodiment of the present application, after receiving a first read request, the size of the space occupied by the target file can be obtained from the read request, and the size of the pre-registered first memory slice is known. Therefore, the number of slices of the first memory slice that can satisfy the storage of the target file can be calculated based on the size of the space occupied by the target file, and then the target first memory with this number of slices can be selected from the registered first memory slices.

[0122] Optionally, step 302 includes: obtaining the usage status of the first memory slice; determining the target first memory slice whose usage status is idle according to the occupied space size of the target file and the capacity of the first memory slice, and determining the number of target first memory slices required to store the target file.

[0123] The user device can manage the memory pool formed by the first memory slice through the first process, so that the first process can obtain the usage status of each first memory slice in real time, such as occupied, idle, etc. Based on the analysis of the usage status, when the user device selects the target first memory slice for current data transmission, the selected target first memory slice can be a memory slice with an idle usage status, so that the purpose of using memory in quantity and the current memory usage does not conflict with other memory usage operations can be achieved, thereby improving the stability of memory usage. Then, the user device can generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice, and the second read request can be constructed in a list form later.

[0124] Optionally, step 304 includes:

[0125] Sub-step 3041, obtaining file storage parameters corresponding to the target first memory slice according to the size of the space occupied by the target file and the capacity of the target first memory slice;

[0126] Sub-step 3042: Generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice.

[0127] In the embodiment of the present application, since the first memory slice is pre-registered, each first memory slice has a storage upper limit and may not be able to store the entire target file. In order to enable the server side to send data more conveniently later, the data sent by the server side from each of its second memory slices to the user device side is not repeated, and the user device side can read and process it in sequence from each target first memory slice conveniently. Therefore, in the embodiment of the present application, the target file can be divided into multiple segments, each segment is stored by a target first memory slice, then which part of the target file is stored in each target first memory slice, and the storage parameters used to store it in the target first memory slice can be determined according to the size of the space occupied by the target file and the capacity of the target first memory slice.

[0128] Afterwards, a second read request corresponding to the target first memory slice is generated according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice.

[0129] Optionally, the file storage parameters include: one or more of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, the file offset parameter is used to characterize the starting position of the local content, and the length parameter is used to characterize the content length of the local content.

[0130] In an embodiment of the present application, a second read request is used to correspond to a request to obtain a file slice (local content) of a target file. In order to define the file slice to be obtained, the second read request can specifically characterize the starting position of the file slice through a file offset parameter (offset), and a length parameter (length) is used to characterize the content length of the file slice. In addition, since the file slices of the target file are continuous, the second read requests in the request list are also arranged in sequence, and the file offset parameter (offset) and length parameter (length) of each second read request also have the characteristic of sequential arrangement.

[0131] For example, for a file of size 5a, the size of the first memory slice is a, the server may divide the file into 5 file slices of size a (a is converted to a length unit of 1048576) for transmission. In order to obtain the file, the user device may generate a second read request list that includes 5 second read requests in sequence, and the file offset parameters (offset) and length parameters (length) of each of the 5 second read requests are respectively:

[0132] Second read request 1: file offset parameter (offset) = 0; length parameter (length) = 1048576;

[0133] Second read request 2: file offset parameter (offset) = 1048576; length parameter (length) = 1048576;

[0134] Second read request 3: file offset parameter (offset) = 2097152; length parameter (length) = 1048576;

[0135] Second read request 4: file offset parameter (offset) = 3145728; length parameter (length) = 1048576;

[0136] Second read request 5: file offset parameter (offset)=4194304; length parameter (length)=1048576.

[0137] In this way, the server can read the file of the corresponding length according to the above storage parameters, store it into its target second memory slice, and then send it to the corresponding first memory slice.

[0138] Step 304: After constructing the second read request into a request list according to the generation order, send the second read request in the request list to the server.

[0139] Step 305: receiving the partial content of the target file sent by the server through the target first memory area.

[0140] In an embodiment of the present application, since the file slices of the target file are sequential, the second read requests corresponding to each file slice are also in a certain order. In order to record this sequential relationship, a request list of second read requests can be constructed according to the order in which the second read requests are generated. The second read requests in the request list are arranged in sequence, and the user device can send the request list of second read requests to the server.

[0141] Optionally, step 304 may be specifically implemented by sending the second read requests in the request list to the server in sequence; or, sending at least part of the second read requests in the request list to the server at the same time.

[0142] In the embodiment of the present application, according to different actual needs, the user device can choose to send the second read request in the request list to the server in sequence; or, send all or part of the second read request in the request list to the server at the same time, thereby achieving flexibility in request sending. After the server receives the second read request in sequence, it can read the local content in the target file in sequence, return it to the client and store it in the corresponding target first memory slice; after the server receives the second read request in batches, it can read the local content corresponding to each second read request at one time, and then return it to the client in sequence or in batches, and store each local data distribution in the target first memory slice corresponding to each second read request.

[0143] Optionally, the method may further include:

[0144] Step 306: In response to a memory registration request for a first process running on the user device, a plurality of first memory slices associated with the first process are set in a local memory of the user device, the first memory slices being used by the first process.

[0145] In the embodiments of this application, refer to Figure 1The first process is a process that runs in the user layer of the user device and is used to obtain files. The normal operation of the first process is inseparable from the use of storage space. The embodiment of the present application can register multiple first memory areas with the same capacity in the memory of the user device for use by the first process before starting the transmission process, thereby forming a memory pool managed by the first process in the user device in advance, so that the user device does not need to perform memory registration operations in the process of obtaining the target file, but selects the registered memory area for use based on the memory management of the memory pool. In addition, the memory management based on the memory pool can realize the use of memory according to the amount, as well as the release and reuse of memory, which greatly improves the utilization efficiency of memory.

[0146] After step 305, the following steps may also be included:

[0147] Step 307: In response to the processing instruction of the first process, the target file stored in the target first memory area is processed.

[0148] In an embodiment of the present application, after the user device completes the storage of the target file, the target file in the target first memory area can be called and used by the first process, for example, the first process can run the target file, play the target file, etc. The first process can also transfer the target file from the target first memory area to the local hard disk of the user device to achieve persistent storage. After persistent storage, the memory pool of the user device can release the space of the target first memory area, thereby achieving memory release and reuse, greatly improving the memory utilization efficiency.

[0149] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the server and the user device for use at one time before starting the transmission process, so that after the server responds to the read request of the user device, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration for each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0150] Reference Figure 8 , which shows a specific step flow chart of a data transmission method provided by an embodiment of the present application, which is applied to a server, and the local memory of the server has multiple second memory areas, and the method includes:

[0151] Step 401: Obtain a second read request sent by a user device.

[0152] Among them, the second read request includes: address information of the target first memory area and file storage parameters corresponding to the target first memory area; the target first memory area is used to store local content of the target file; the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory area.

[0153] This step may refer to the above step 201 and will not be described in detail here.

[0154] Step 402: Determine the storage location of the target file locally on the server, and send a third read request to the virtual file system of the server through the second process.

[0155] The third read request includes file storage parameters, so that the virtual file system responds to the third read request, calls the hard disk input / output interface to extract the local content of the target file from the storage location, and copies the local content to the target second memory area.

[0156] In the embodiments of this application, refer to Figure 1 In combination with the explanation of the file offset parameter (offset) and the length parameter (length) in the above embodiment, the server can extract the file offset parameter (offset) and the length parameter (length) in the second read request, and generate a third read request according to the file offset parameter (offset) and the length parameter (length) and send it to the virtual file system (VFS). VFS is a file system in the kernel layer of the server that connects to the user layer and can process requests sent by the user layer.

[0157] According to the third read request, VFS can directly call the hard disk interface to extract the local content of the target file from the storage location, and copy the local content to the target second memory area. This process can save the data transfer of Page Cache and EXT4, thereby reducing the number of data copies in the memory during the file transfer process, further reducing the memory usage cost and improving the transmission efficiency.

[0158] Optionally, step 402 may be specifically implemented by the second process calling a direct read / write interface to send a third read request to the virtual file system of the server.

[0159] In an embodiment of the present application, the server can send a third read request to the server's virtual file system by calling the direct I / O interface through the second process. The direct I / O interface can bypass the read access memory of the file write buffer and the read cache, thereby reducing the number of data copies in the memory during the file transfer process.

[0160] Step 403: according to the address information of the target first memory area, the local content stored in the target second memory area is sent to the target first memory area of ​​the user device for storage.

[0161] This step may refer to the above step 203 and will not be described in detail here.

[0162] Optionally, the method may further include:

[0163] Step 404: In response to a memory registration request for a second process running on the client, a plurality of second memory slices associated with the second process are set in the local memory of the server, and the second memory slices are used for use by the second process.

[0164] In the embodiments of this application, refer to Figure 1 The second process is a process that runs in the user layer of the server and is used to send files. The normal operation of the second process is inseparable from the use of storage space. The embodiment of the present application can register multiple second memory areas with the same capacity in the memory of the server at one time for use by the second process before starting the transmission process, thereby forming a memory pool managed by the second process in the server in advance, so that the server does not need to perform memory registration operations in the process of sending the target file, but selects the registered memory area for use based on the memory management of the memory pool. In addition, the memory management based on the memory pool can realize the use of memory according to the amount, as well as the release and reuse of memory, which greatly improves the utilization efficiency of memory.

[0165] Optionally, the remote direct data access device is a remote direct data access network card.

[0166] In the embodiment of the present application, the RDMA network card omits the participation of the kernel processor, and all data transmissions directly reach the RDMA network card from the process, and then forwarded by the RDMA network card to the RDMA network card of another terminal. The RDMA network card can reduce the processor load, improve network throughput, and reduce network latency.

[0167] Optionally, in another embodiment of the present application, in the aforementioned step 202, extracting the local content of the target file from the storage location and copying it to the target second memory area according to the file storage parameter includes:

[0168] Sub-step 2021, determining the local position of the local content in the storage position according to the file storage parameter;

[0169] Sub-step 2022, extracting the local content according to the local position and copying it to the target memory area.

[0170] In an embodiment of the present application, since the user device side sends a second read request to the server side, after the server side finds out where the target file is stored on the disk, it can further locate the local location of the local content corresponding to the file storage parameters in the storage location based on the file storage parameters.

[0171] Optionally, in another embodiment of the present application, the file storage parameter includes: at least one of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, the file offset parameter is used to characterize the starting position of the local content, and the length parameter is used to characterize the content length of the local content; the sub-step 2021 includes:

[0172] Sub-step 20211, determining the local position of the local content in the storage location based on the file offset parameter and / or the length parameter.

[0173] As described above, in the embodiment of the present application, since the file storage parameter includes the aforementioned file offset parameter (offset) and / or length parameter (length), the length parameter corresponds to the storage capacity allocated to the target first memory.

[0174] Then, when receiving the offset, if the memory allocation capacity is the same, the length is fixed, and the server side can directly know the length. Then, after the user device side sends the offset to the server side through the second read request, the server side determines the starting read position to be read from the disk and the length to be read from the starting read position based on the offset and the known length, according to the storage address of the target storage location on the disk.

[0175] When the offset and length are received, the user device sends the offset to the server through a second read request. The server determines the starting read position to be read from the disk and the length to be read from the starting read position based on the offset and the known length and the storage address of the target storage location on the disk.

[0176] When length is received, the user device sends the offset to the server through a second read request. Based on the length, the server defaults the offset of a target second memory slice to 0, and then sequentially adds the starting position of the next target first memory slice to the length of the previous target first memory slice to obtain the offset of the next target first memory slice. In this way, the offset and length of each target first memory slice are obtained, and the starting read position to be read from the disk and the length to be read from the starting position are determined according to the storage address of the target storage position on the disk.

[0177] Then, the local content of the target file can be read from the starting reading position of the disk and the length read from the starting position, and stored in the target second memory area. Then, the server sends the local content in the target second memory area to the target first memory area corresponding to the corresponding request.

[0178] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the server and the user device for use at one time before starting the transmission process, so that after the server responds to the read request of the user device, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration for each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0179] Reference Fig. 9 , which shows a block diagram of a distributed data synchronization system provided by an embodiment of the present application, including:

[0180] At least one upload node and at least one data synchronization node; the local memory of the data synchronization node has a plurality of first memory slices; the local memory of the upload node has a plurality of second memory slices;

[0181] The upload node is used to obtain the target file to be synchronized and backed up, and send the size of the occupied space of the target file to the data synchronization node; obtain the backup request sent by the data synchronization node; according to the backup request, extract the local content of the target file and copy it to the target second memory slice area, the target second memory slice area is any second memory slice area; according to the address information of the target second memory slice area, send the local content stored in the target second memory slice area to the target first memory slice area of ​​the data synchronization node for storage;

[0182] The data synchronization node is used to determine the number of target first memory slices required to store the target file based on the size of the space occupied by the target file and the capacity of the first memory slice; based on the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice, generate a backup request corresponding to the target first memory slice and send it to the upload node, and the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory slice.

[0183] Specifically, the description of the distributed data synchronization system can refer to the above Figure 3 The embodiments of the present invention are not described in detail here.

[0184] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the upload node and the data synchronization node for use at one time before starting the transmission process, so that after the upload node responds to the backup request of the data synchronization node, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the upload node can directly send the slice data of the memory slice of the upload node to the registered memory slice in the data synchronization node through the RDMA technology, thereby saving the system overhead and time of memory registration at each response. In addition, the memory management of the upload node and the data synchronization node based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0185] Reference Fig.10 , which shows a block diagram of a distributed computing system provided by an embodiment of the present application, including:

[0186] At least one master computing node and at least one slave computing node; the master computing node has a plurality of first memory slices in its local memory; the slave computing node has a plurality of second memory slices in its local memory;

[0187] The main computing node is used to obtain a computing task, split the computing task into multiple subtasks and send the subtasks to the slave computing node; determine the number of target first memory slices required to store the computing results based on the size of the space occupied by the computing results of the subtasks and the capacity of the first memory slice; generate a synchronization request corresponding to the target first memory slice based on the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice and send it to the slave computing node, the file storage parameters are used to characterize the characteristics of the local content to be stored in the target first memory slice.

[0188] The slave computing node is used to obtain subtasks for calculation, and send the identifier and occupied space size of the calculation result to the main computing node; obtain the synchronization request sent by the main computing node; according to the synchronization request, extract the local content of the calculation result and copy it to the target second memory slice area, the target second memory slice area is any second memory slice area; according to the address information of the target second memory slice area, send the local content stored in the target second memory slice area to the target first memory slice area of ​​the main computing node for storage.

[0189] Specifically, the description of the distributed data synchronization system can refer to the above Figure 4 The embodiments of the present invention are not described in detail here.

[0190] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the slave computing node and the main computing node for use at one time before starting the transmission process, so that after the slave computing node responds to the synchronization request of the main computing node, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the slave computing node can directly send the slice data of the memory slice of the slave computing node to the registered memory slice in the main computing node through the RDMA technology, thereby saving the system overhead and time of memory registration at each response. In addition, the memory management of the slave computing node and the main computing node based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0191] Reference Fig.11 , which shows a block diagram of a data transmission device provided in an embodiment of the present application, which is applied to a user device, and a plurality of first memory areas are set in a local memory of the user device, including:

[0192] The first setting module 501 is used to obtain the occupied space size of the target file in response to the first read request, and determine the number of target first memory slices required to store the target file according to the occupied space size of the target file and the capacity of the first memory slice; the target first memory slice is used to store the local content of the target file;

[0193] A generating module 502 is used to generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, and send the second read request to the server, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice;

[0194] The storage module 503 is used to receive the local content of the target file sent by the server through the target first memory area.

[0195] Optionally, the first setting module 501 includes:

[0196] The first determination submodule is used to determine the required number of slices of the first memory slice according to the occupied space size of the target file and the capacity of the first memory slice.

[0197] The second determining submodule is used to select target first memory slices of the number of slices from the first memory slices according to the number of slices.

[0198] Optionally, the generating module 502 includes:

[0199] A storage parameter determination submodule, used for acquiring a file storage parameter corresponding to the target first memory slice according to the occupied space size of the target file and the capacity of the target first memory slice;

[0200] The request generation submodule is used to generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameters corresponding to the target first memory slice.

[0201] Optionally, the generating module 502 includes:

[0202] The list sending submodule is used to construct the second read request into a request list according to the generation order, and then send the second read request in the request list to the server.

[0203] Optionally, the list sending submodule includes:

[0204] A first sending unit, used for sending the second read request in the request list to the server in sequence;

[0205] Or, a second sending unit is used to send at least part of the second read requests in the request list to the server at the same time.

[0206] Optionally, the device further comprises:

[0207] A first registration module, configured to respond to a memory registration request for a first process running on a user device, and to set a plurality of first memory slices associated with the first process in a local memory of the user device, wherein the first memory slices are used by the first process;

[0208] Optionally, the device further comprises:

[0209] The processing module is used to process the target file stored in the target first memory area in response to the processing instruction of the first process.

[0210] Optionally, the first setting module 501 includes:

[0211] A status submodule, used to obtain the usage status of the first memory slice;

[0212] The submodule is selected to determine the target first memory slice whose usage state is idle according to the occupied space size of the target file and the capacity of the first memory slice, and to determine the number of the target first memory slices required to store the target file.

[0213] Optionally, the file storage parameters include: one or more of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, the file offset parameter is used to characterize the starting position of the local content, and the length parameter is used to characterize the content length of the local content.

[0214] In summary, the embodiment of the present application is to register multiple memory slices with the same capacity in the respective memories of the server and the user device for use at one time before starting the transmission process, so that after the server responds to the read request of the user device, there is no need to perform a memory registration operation, but based on the memory management of the memory pool, select the registered memory slice for use, and divide the file to be transmitted into slices aligned with the memory slice and store them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration for each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0215] Reference Fig.12, which shows a block diagram of another data transmission device provided by an embodiment of the present application, which is applied to a server, and the server has multiple second memory areas locally, including:

[0216] The receiving module 601 is used to obtain a second read request sent by the user device, where the second read request includes: address information of a target first memory slice and a file storage parameter corresponding to the target first memory slice; the target first memory slice is used to store local content of a target file; and the file storage parameter is used to characterize features of the local content to be stored in the target first memory slice;

[0217] Extraction module 602, used to determine the storage location of the target file in the local server, and extract the local content of the target file from the storage location and copy it to the target second memory slice according to the file storage parameter, the target second memory slice is any second memory slice;

[0218] The sending module 603 is used to send the local content stored in the target second memory area to the target first memory area of ​​the user device for storage according to the address information of the target first memory area.

[0219] Optionally, the extraction module 602 includes:

[0220] A local position extraction submodule, used to determine the local position of the local content in the storage location according to the file storage parameter;

[0221] The copy submodule is used to extract the local content according to the local position and copy it to the target memory area.

[0222] Optionally, the file storage parameters include: at least one of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, wherein the file offset parameter is used to characterize the starting position of the local content; the copy submodule is further used to determine the local position of the local content in the storage location based on the file offset parameter and / or the length parameter.

[0223] Optionally, the device further comprises:

[0224] The second registration module is used to respond to a memory registration request for a second process running on a user device, and to set a plurality of second memory slices associated with the second process in the local memory of the server, the second memory slices being used by the second process.

[0225] Optionally, the file storage parameters include: a file offset parameter and a length parameter corresponding to the local content of the target file, the file offset parameter is used to represent the starting position of the local content, and the length parameter is used to represent the content length of the local content;

[0226] The extraction module 602 includes:

[0227] The direct read submodule is used to send a third read request to the virtual file system of the server through the second process. The third read request includes file storage parameters, so that the virtual file system responds to the third read request, calls the hard disk input / output interface to extract the local content of the target file from the storage location, and copies the local content to the target second memory area.

[0228] Optionally, submodules are read directly, including:

[0229] The direct reading unit is used to call the direct reading and writing interface through the second process to send a third reading request to the virtual file system of the server.

[0230] Optionally, the remote direct data access device is a remote direct data access network card.

[0231] In an embodiment of the present application, before starting the transmission process, multiple memory slices with the same capacity are registered in the respective memories of the server and the user device for use at one time, so that after responding to the read request of the user device, the server does not need to perform a memory registration operation, but selects the registered memory slice for use based on the memory management of the memory pool, and divides the file to be transmitted into slices aligned with the memory slice and stores them accordingly. After that, the server can directly send the slice data of the server's memory slice to the registered memory slice in the user device through the RDMA technology, thereby saving the system overhead and time of memory registration at each response. In addition, the memory management of the server and the user device based on the memory pool can realize the use of memory in quantity and the release and reuse of memory, which greatly improves the utilization efficiency of memory. For the entire RDMA transmission process, the present application can greatly reduce the participation of the kernel processor and the pressure of cached data copying, which has a positive effect on reducing system overhead.

[0232] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0233] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0234] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices and servers (clusters). Fig.13 An exemplary device 1000 that can be used to implement various embodiments described in the embodiments of the present application is schematically shown.

[0235] For one embodiment, Fig.13 An exemplary apparatus 1000 is shown having one or more processors 1002, a control module (chip set) 1004 coupled to at least one of the processor(s) 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.

[0236] The processor 1002 may include one or more single-core or multi-core processors, and the processor 1002 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1000 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present application.

[0237] In some embodiments, the apparatus 1000 may include one or more computer-readable media (e.g., memory 1006 or NVM / storage device 1008) having instructions 1014 and one or more processors 1002 combined with the one or more computer-readable media and configured to execute the instructions 1014 to implement a module to perform the actions described in the present disclosure.

[0238] For one embodiment, the control module 1004 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1002 and / or any suitable device or component in communication with the control module 1004 .

[0239] The control module 1004 may include a memory controller module to provide an interface to the memory 1006. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0240] The memory 1006 may be used, for example, to load and store data and / or instructions 1014 for the device 1000. For one embodiment, the memory 1006 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1006 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0241] For one embodiment, control module 1004 may include one or more input / output controllers to provide an interface to NVM / storage device 1008 and input / output device(s) 1010 .

[0242] For example, NVM / storage 1008 may be used to store data and / or instructions 1014. NVM / storage 1008 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0243] NVM / storage device 1008 may include storage resources that are physically part of the device on which apparatus 1000 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1008 may be accessed via input / output device(s) 1010 over a network.

[0244] (One or more) input / output devices 1010 may provide an interface for the apparatus 1000 to communicate with any other appropriate device, and the input / output device 1010 may include a communication component, an audio component, a sensor component, etc. The network interface 1012 may provide an interface for the apparatus 1000 to communicate through one or more networks, and the apparatus 1000 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0245] For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers of the control module 1004 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004 to form a system-on-chip (SoC).

[0246] In various embodiments, the apparatus 1000 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the apparatus 1000 may have more or fewer components and / or a different architecture. For example, in some embodiments, the apparatus 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0247] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0248] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0249] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0250] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0251] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0252] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0253] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0254] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0255] The above is a detailed introduction to a data transmission method, device, distributed data synchronization system, distributed computing system, electronic device, and machine-readable medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data transmission method, characterized in that: Applied to a user device, a plurality of first memory slices are set in a local memory of the user device, and the method includes: In response to a first read request, the size of the space occupied by the target file is obtained, and according to the size of the space occupied by the target file and the capacity of the first memory slice, a target first memory slice required to store the target file is determined; the target first memory slice is used to store the local content of the target file; Generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, and after constructing the second read request into a request list in the generation order, send the second read requests in the request list to the server in sequence; or send at least part of the second read requests in the request list to the server at the same time, the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice; the number of the second read requests is the same as the number of the target first memory slice; The local content of the target file sent by the server is received through the target first memory area.

2. The method according to claim 1, characterized in that The step of determining a target first memory slice required for storing the target file according to the size of the space occupied by the target file and the capacity of the first memory slice comprises: Determine the number of slices of the first memory slice required according to the space occupied by the target file and the capacity of the first memory slice; According to the number of slices, target first memory slices of the number of slices are selected from the first memory slices.

3. The method according to claim 1, characterized in that The step of generating a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice comprises: According to the occupied space size of the target file and the capacity of the target first memory slice, obtaining a file storage parameter corresponding to the target first memory slice; A second read request corresponding to the target first memory slice is generated according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice.

4. The method according to claim 1, characterized in that: The method further comprises: In response to a memory registration request for a first process running on the user device, divide a plurality of first memory slices associated with the first process in a local memory of the user device, the first memory slices being used by the first process; After receiving the partial content of the target file sent by the server through the target first memory area, the method further includes: In response to the processing instruction of the first process, the target file stored in the target first memory area is processed.

5. The method according to any one of claims 1 to 3, characterized in that: The file storage parameters include: one or more of a file offset parameter corresponding to the local content of the target file and a length parameter corresponding to the local content of the target file, wherein the file offset parameter is used to characterize the starting position of the local content, and the length parameter is used to characterize the content length of the local content.

6. A data transmission method, characterized in that: Applied to a server, the server has a plurality of second memory slices locally, and the method comprises: After the user device constructs the second read request into a request list in the order of generation, the second read request in the request list sent in sequence by the user device, or at least part of the second read request in the request list sent simultaneously by the user device, the second read request including: address information of the target first memory slice and a file storage parameter corresponding to the target first memory slice; the target first memory slice is used to store the local content of the target file; the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice; the number of the second read requests is the same as the number of the target first memory slice; Determine the storage location of the target file locally on the server, and extract the local content of the target file from the storage location and copy it to a target second memory slice according to the file storage parameter, wherein the target second memory slice is at least one second memory slice; According to the address information of the target first memory area, the local content stored in the target second memory area is sent to the target first memory area of ​​the user device for storage.

7. A data transmission device, characterized in that: Applied to a user device, a plurality of first memory slices are set in a local memory of the user device, and the device comprises: A first setting module is used to obtain the occupied space size of the target file in response to the first read request, and determine the target first memory slice required to store the target file according to the occupied space size of the target file and the capacity of the first memory slice; the target first memory slice is used to store the local content of the target file; A generation module, configured to generate a second read request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, and after constructing the second read request into a request list in a generation order, send the second read requests in the request list to the server in sequence; or send at least part of the second read requests in the request list to the server at the same time, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice; the number of the second read requests is the same as the number of the target first memory slice; A storage module is used to receive the local content of the target file sent by the server through the target first memory area.

8. A data transmission device, characterized in that: Applied to a server, the server has a plurality of second memory slices locally, and the device comprises: A receiving module, used for acquiring the second read requests in the request list sent in sequence by the user device after the user device constructs the second read requests into a request list in a generated order, or at least part of the second read requests in the request list sent simultaneously by the user device, wherein the second read request includes: address information of a target first memory slice and a file storage parameter corresponding to the target first memory slice; the target first memory slice is used to store local content of a target file; the file storage parameter is used to characterize features of the local content to be stored in the target first memory slice; the number of the second read requests is the same as the number of the target first memory slice; an extraction module, configured to determine a storage location of the target file locally on the server according to an identifier of the target file, and to extract local content of the target file from the storage location and copy it to a target second memory slice area according to the file storage parameter, wherein the target second memory slice area is any second memory slice area; The sending module is used to send the local content stored in the target second memory area to the target first memory area of ​​the user equipment for storage according to the address information of the target first memory area.

9. A distributed data synchronization system, characterized in that: include: At least one upload node and at least one data synchronization node; The local memory of the data synchronization node has a plurality of first memory slices; The upload node has a plurality of second memory slices locally; The upload node is used to obtain the target file to be synchronized and backed up, and send the occupied space size of the target file to the data synchronization node; Obtaining a backup request sent by the data synchronization node; According to the backup request, extract the local content of the target file and copy it to the target second memory slice, where the target second memory slice is any second memory slice; according to the address information of the target first memory slice, send the local content stored in the target second memory slice to the target first memory slice of the data synchronization node for storage; The data synchronization node is used to determine a target first memory slice required to store the target file according to the occupied space size of the target file and the capacity of the first memory slice; Generate a backup request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, and after constructing the backup requests into a request list in the order of generation, send the backup requests in the request list to the upload node in sequence; Or, at least part of the backup requests in the request list are sent to the uploading node at the same time, and the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice; The number of the backup requests is the same as the number of the target first memory slices.

10. A distributed computing system, characterized in that: include: at least one master computing node and at least one slave computing node; The local memory of the master computing node has a plurality of first memory slices, and the local memory of the slave computing node has a plurality of second memory slices; The master computing node is used to obtain a computing task, split the computing task into multiple subtasks and send the subtasks to the slave computing node; Determine a target first memory slice required to store the calculation result according to the size of the space occupied by the calculation result of the subtask and the capacity of the first memory slice; Generate a synchronization request corresponding to the target first memory slice according to the address information of the target first memory slice and the file storage parameter corresponding to the target first memory slice, and after constructing the synchronization request into a request list in the generation order, send the synchronization requests in the request list to the slave computing node in sequence; or send at least part of the synchronization requests in the request list to the slave computing node at the same time, wherein the file storage parameter is used to characterize the characteristics of the local content to be stored in the target first memory slice; The number of the synchronization requests is the same as the number of the target first memory slices; The slave computing node is used to obtain the subtask for computing, and send the size of the space occupied by the computing result to the master computing node; Obtaining a synchronization request sent by the main computing node; According to the synchronization request, the local content of the calculation result is extracted and copied to the target second memory slice, and the target second memory slice is any second memory slice; according to the address information of the target first memory slice, the local content stored in the target second memory slice is sent to the target first memory slice of the main computing node for storage.

11. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when the executable codes are executed, enable the processor to perform the method according to any one of claims 1 to 6.

12. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to perform the method according to any one of claims 1 to 6.

Images