Data transmission method and related apparatus

Abstract

A data transmission method, which is applied to data transmission between different computing apparatuses. In the method, a first computing apparatus needing to consume data acquires a virtual address and a page table of data on a second computing apparatus serving as a data producer, and determines a physical address of the data on the second computing apparatus on the basis of the virtual address and the page table of the data, so as to directly read the required data from the physical address on the second computing apparatus, thereby avoiding a data forwarding process, and a serialization process and a deserialization process of the data, so that the data transmission efficiency can be effectively improved, and the performance overhead caused by data transmission can be reduced.

Description

A data transmission method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 29, 2023, with application number 202311863899.5 and application name “A Data Transmission Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data transmission method and related devices. Background Art

[0003] In the cloud computing field, serverless platforms have emerged to reduce the burden on developers when deploying applications. These platforms provide a minimal architecture that simplifies application deployment, eliminating the need for developers to deploy, configure, or manage server services. Instead, they provide all the server services required for application operation.

[0004] When using the Serverless platform, developers simply abstract their business into functions, package the functions and their dependencies into container images, and upload the container images to the Serverless platform. When a function needs to be called (for example, when a network request is made to call the function), the Serverless platform automatically starts a container to run the function. Different functions on the Serverless platform may run on different servers, and different functions often need to exchange data. Therefore, the Serverless platform involves transmitting the data that functions interact with between different servers.

[0005] Currently, in related technologies, a function acting as a data producer serializes the generated data and sends the serialized data to a coordinator. The coordinator then forwards the serialized data to a function acting as a data consumer, which ultimately deserializes the data to restore the original data. Data producers, coordinators, and data consumers are often located on different servers, and data transmission requires forwarding by the coordinator. Furthermore, the data transmission process involves serialization and deserialization, resulting in low data transmission efficiency between functions. Summary of the Invention

[0006] The present application provides a data transmission method that can reduce the performance overhead caused by data transmission.

[0007] In a first aspect, the present application provides a data transmission method for transmitting data between different computing devices. The method includes: first, the first computing device obtains a first virtual address, a network address, and a page table, the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the network address is the address of the second computing device, and the page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device. In addition, the first computing device is the consumer of the first data, and the second computing device is the producer of the first data. That is, the first data generated by the second computing device needs to be transmitted to the first computing device.

[0008] Then, based on the first virtual address and the page table, the first computing device determines the first physical address of the first data on the second computing device, that is, obtains the physical address where the first data is actually located on the memory of the second computing device.

[0009] Secondly, based on the network address of the second computing device, the first computing device can read the first data from the first physical address of the second computing device through the network, thereby avoiding the serialization and deserialization process of the first data, and there is no need to use other coordination devices to help forward the first data.

[0010] In this solution, the first computing device that needs to consume data obtains the virtual address and page table of the data on the second computing device that is the data producer, and determines the physical address of the data on the second computing device based on the virtual address and page table of the data, and then directly reads the required data from the physical address of the second computing device, avoiding the data forwarding process, serialization process and deserialization process, which can effectively improve data transmission efficiency and reduce the performance overhead brought by data transmission.

[0011] In one possible implementation, the data transmission method is applied to a serverless platform. A first computing device and a second computing device are each used to run different functions on the serverless platform. The input to the function run by the first computing device includes data generated by the function run by the second computing device. In other words, the function run by the first computing device depends on the function run by the second computing device. Therefore, the first data generated by the function run by the second computing device needs to be transmitted to the function run on the first computing device.

[0012] In one possible implementation, the first computing device determines the first physical address of the first data on the second computing device, specifically including: in response to a page fault exception caused by a function running on the first computing device requesting access to a first virtual address, the first computing device determines the first physical address of the first data on the second computing device based on the first virtual address and a page table. That is, the first computing device determines the first physical address corresponding to the first data by being triggered when a function running on the first computing device accesses data and a page fault exception occurs.

[0013] In this solution, by expanding the conventional page fault exception processing flow, the computing device can choose to read the required data from other computing devices through the network, thereby realizing efficient data transmission between computing devices and improving the feasibility of the solution; and the computing device does not need the function itself to complete the reading of the required data through the network, which can ensure that the existing function can be adapted to this solution without modification, thereby improving the applicability of the solution.

[0014] In one possible implementation, the first computing device first receives the first virtual address, network address and virtual address segment sent by the coordination device, and the coordination device is used to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; then, based on the network address, the first computing device obtains the page table corresponding to the virtual address segment from the second computing device, and the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address.

[0015] In this solution, the coordination device forwards virtual addresses and virtual address segments with lower data volume to the first computing device, and the first computing device directly obtains page tables with larger data volume from the second computing device based on the virtual address segments. This can reduce the amount of data forwarded by the coordination device and improve the efficiency of the first computing device in obtaining various information.

[0016] In one possible implementation, the first computing device obtains a page table corresponding to a virtual address segment from the second computing device, including: the first computing device sends a page table read request to the second computing device, the page table read request includes the virtual address segment, and the page table read request is used to request to read the page table corresponding to the virtual address segment; the first computing device receives the page table sent by the second computing device.

[0017] In one possible implementation, the data transmission method also includes: the first computing device receives the first virtual address space sent by the coordination device; the first computing device runs the function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0018] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the second virtual address space is a virtual address space used by a function running on the second computing device.

[0019] In this solution, the coordination device that coordinates the various computing devices is responsible for allocating the virtual address space used by the functions in the first computing device, so that the virtual address space used by the functions in the first computing device does not overlap with the virtual address space used by the functions in other computing devices, thereby ensuring that there will be no address conflict when the computing devices read data from each other based on virtual addresses, and ensuring that the solution can be executed normally.

[0020] A second aspect of the present application provides a data transmission method, including: a second computing device sends a first virtual address and a virtual address segment to a coordination device, the first virtual address being a virtual address corresponding to first data generated by the second computing device on the second computing device, the virtual address segment being used to indicate a virtual address range corresponding to the data generated by the second computing device, and the first virtual address being located in the virtual address segment, the coordination device being used to forward the first virtual address and the virtual address segment to the first computing device; in response to a request from the first computing device to read the virtual address segment, the second computing device sends a page table to the first computing device, the page table being used to record a mapping relationship between a virtual address in the virtual address segment and a physical address, and the page table records a mapping relationship between the first virtual address and the first physical address; in response to a request from the first computing device to access the first physical address, the second computing device returns the first data at the first physical address to the first computing device.

[0021] In this solution, the first computing device that needs to consume data obtains the virtual address and page table of the data on the second computing device that is the data producer, and determines the physical address of the data on the second computing device based on the virtual address and page table of the data, and then directly reads the required data from the physical address of the second computing device, avoiding the data forwarding process, serialization process and deserialization process, which can effectively improve data transmission efficiency and reduce the performance overhead brought by data transmission.

[0022] In one possible implementation, the above-mentioned data transmission method is applied to a Serverless platform, and the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes data generated by the function run by the second computing device.

[0023] In a possible implementation, the above-mentioned data transmission method further includes: the second computing device sets the permission of the target physical address to copy on write, and the target physical address is a physical address recorded in the page table and whose original permission is write.

[0024] That is, after the first computing device obtains the page table, all permissions corresponding to the physical addresses recorded in the page table that were originally written are changed to copy-on-write permissions. This way, even if the first computing device subsequently triggers a call and modifies the data in the second computing device, it will not affect the data in the second computing device. In other words, the data in the second computing device will not be modified by the first computing device, thus ensuring that the data generated by the function can be stored normally in the second computing device.

[0025] In one possible implementation, the above-mentioned data transmission method also includes: the second computing device receives the second virtual address space sent by the coordination device; the second computing device runs the function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0026] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the first virtual address space is a virtual address space used by a function running on the first computing device.

[0027] A third aspect of the present application provides a first computing device, including: a transceiver module, used to obtain a first virtual address, a network address and a page table, the first virtual address is a virtual address corresponding to the first data generated by the second computing device on the second computing device, the network address is the address of the second computing device, and the page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device; a processing module, used to determine the first physical address of the first data on the second computing device based on the first virtual address and the page table; the transceiver module, used to read the first data from the first physical address of the second computing device based on the network address.

[0028] In one possible implementation, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes data generated by the function run by the second computing device.

[0029] In one possible implementation, the processing module is further used to: in response to a page fault exception caused by a function running in the first computing device requesting access to the first virtual address, the first computing device determines, based on the first virtual address and the page table, a first physical address of the first data on the second computing device.

[0030] In one possible implementation, the transceiver module is also used to: receive the first virtual address, network address and virtual address segment sent by the coordination device, the coordination device is used to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; based on the network address, obtain the page table corresponding to the virtual address segment from the second computing device, and the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address.

[0031] In one possible implementation, the transceiver module is further used to: send a page table read request to the second computing device, the page table read request includes a virtual address segment, and the page table read request is used to request to read the page table corresponding to the virtual address segment; and receive the page table sent by the second computing device.

[0032] In one possible implementation, the transceiver module is further used to receive the first virtual address space sent by the coordination device; the processing module is further used to run the function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0033] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the second virtual address space is a virtual address space used by a function running on the second computing device.

[0034] The fourth aspect of the present application provides a second computing device, including: a transceiver module, used to send a first virtual address and a virtual address segment to a coordination device, the first virtual address being the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment being used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address being located in the virtual address segment, the coordination device being used to forward the first virtual address and the virtual address segment to the first computing device; in response to the first computing device requesting to read the virtual address segment, the transceiver module is further used to send a page table to the first computing device, the page table being used to record the mapping relationship between the virtual address in the virtual address segment and the physical address, and the page table records the mapping relationship between the first virtual address and the first physical address; in response to the first computing device requesting to access the first physical address, the transceiver module is further used to return the first data on the first physical address to the first computing device.

[0035] In one possible implementation, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes data generated by the function run by the second computing device.

[0036] In a possible implementation, the second computing device further includes: a processing module configured to set the permission of the target physical address to copy-on-write, where the target physical address is a physical address recorded in the page table and whose original permission is write.

[0037] In one possible implementation, the above-mentioned second computing device also includes a processing module; a transceiver module, which is also used to receive the second virtual address space sent by the coordination device; a processing module, which is used to run the function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0038] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the first virtual address space is a virtual address space used by a function running on the first computing device.

[0039] In a fifth aspect, the present application provides a computing device comprising: a memory and a processor; the memory stores code, the processor is configured to execute the code, and when the code is executed, the computing device executes a method as implemented in any one of the first or second aspects.

[0040] In a sixth aspect, the present application provides a data transmission system, comprising: a first computing device as in any one of the implementations of the third aspect, a second computing device as in any one of the implementations of the fourth aspect, and a coordination device, the coordination device being used to implement data forwarding between the first computing device and the second computing device.

[0041] In a seventh aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes a method as in any one of the implementation methods of the first aspect.

[0042] In an eighth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute a method as in any one of the implementations of the first aspect.

[0043] In a ninth aspect, the present application provides a chip comprising one or more processors, wherein some or all of the processors are configured to read and execute a computer program stored in a memory to perform the method in any one of the implementations of the first aspect.

[0044] Optionally, the chip includes a memory, and the memory is connected to the processor via a circuit or wire. Optionally, the chip also includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and / or information to be processed, and the processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing results through the communication interface. The communication interface can be an input / output interface. The method provided in this application can be implemented by a single chip or by multiple chips working together.

[0045] Among them, the technical effects brought about by any design method in the second to sixth aspects can refer to the technical effects brought about by different implementation methods in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of a serverless workflow provided in an embodiment of the present application;

[0047] FIG2 is a schematic diagram of data transfer between functions in a Serverless workflow provided by an embodiment of the present application;

[0048] FIG3 is a schematic diagram of data transfer between functions in the related art;

[0049] FIG4 is a schematic structural diagram of an electronic device 101 provided in an embodiment of the present application;

[0050] FIG5 is a flow chart of a data transmission method provided in an embodiment of the present application;

[0051] FIG6 is a flow chart of a data transmission method provided in an embodiment of the present application;

[0052] FIG7 is a schematic diagram of the architecture of a Serverless platform provided in an embodiment of the present application;

[0053] FIG8 is a schematic diagram of a process for planning a virtual address space for a function in a Serverless workflow, provided by an embodiment of the present application;

[0054] FIG9 is a schematic diagram of a flow chart of a computing device calling an rmap interface according to an embodiment of the present application;

[0055] FIG10 is a schematic diagram of a process for handling a page fault exception according to an embodiment of the present application;

[0056] FIG11 is a schematic diagram of some scenarios in which the data transmission method provided in an embodiment of the present application is applied;

[0057] FIG12 is a schematic structural diagram of a first computing device provided in an embodiment of the present application;

[0058] FIG13 is a schematic structural diagram of a second computing device provided in an embodiment of the present application;

[0059] FIG14 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0060] FIG15 is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of this application, rather than all embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0062] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchangeable where appropriate so that the embodiments can be implemented in a sequence other than that illustrated or described in this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0063] The division of units in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. Moreover, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0064] To facilitate understanding, some technical terms involved in this embodiment are introduced below.

[0065] (1) Serverless

[0066] Serverless is a cloud computing model and, in essence, a new type of internet architecture. Serverless doesn't mean the absence of servers, but rather that developers don't need to worry about server management and maintenance. In traditional application architectures, developers need to focus on server configuration, scaling, maintenance, and monitoring. In a serverless architecture, developers focus solely on the application's business logic, without having to worry about underlying server management. Cloud service providers automatically scale and manage the underlying server resources, allocating them based on application load.

[0067] (2) Function

[0068] A function is the basic operating unit in serverless computing. It is essentially a piece of program code uploaded by the user. When the serverless platform receives a request to call a function, it launches a container and runs the function.

[0069] (3) Container

[0070] Containers are a way to run processes that can simplify application deployment and provide stronger isolation between processes. In this embodiment, the container is actually the runtime environment for functions, which can achieve mutual isolation between functions.

[0071] (4) Scheduling Node

[0072] When a Serverless platform runs multiple functions simultaneously, a scheduling node will centrally schedule the functions based on their dependencies. This scheduling node is also called a coordinator.

[0073] (5) Host

[0074] A host is a server that runs a function. In a serverless platform, the scheduling node selects an idle host to execute the function.

[0075] (6)Serialization

[0076] Serialization is the process of converting the state information of an object into a form that can be stored or transmitted. Simply put, serialization is the process of converting the memory data of a language into a continuous binary data.

[0077] (7) Deserialization

[0078] Deserialization refers to the process of converting the continuous binary data generated during the serialization process into data in memory.

[0079] (8) Page Table

[0080] A page table is a data structure that records the mapping relationship between virtual addresses and physical addresses. Generally, in a computer, a memory management unit (MMU) can convert between virtual addresses and physical addresses by querying the page table.

[0081] (9) Virtual address (VA)

[0082] A virtual address is the address of a storage unit in a computer architecture as seen by an application. Virtual addresses are often different from the physical addresses of storage units and require an address translator to convert virtual addresses into physical addresses (PA).

[0083] Simply put, a virtual address is usually an address generated by the CPU in a computer when an application is running; a physical address is a real address in physical memory.

[0084] (10) Copy on write (COW)

[0085] Copy-on-write is a resource management technique used in computer programming to efficiently implement copying or duplicating modifiable resources. If a resource is copied but not modified, there's no need to create a new resource; the resource can be shared between the copy and the original. However, if a resource is modified, a copy must still be created. By sharing resources using copy-on-write, resource consumption by the unmodified copy can be significantly reduced, while adding a small overhead to resource modification operations.

[0086] (11) Remote Procedure Call (RPC)

[0087] RPC is a protocol for requesting services from remote computers over a network without requiring knowledge of the underlying network technology. Specifically, for Server A and Server B, an application deployed on Server A wants to call a function provided by the application on Server B. Since the two are not in the same memory space, a direct call cannot be made. Therefore, the call semantics and data must be expressed over the network. RPC provides a way to obtain services from remote servers over the network.

[0088] (12) Page fault exception

[0089] A page fault exception occurs when the central processing unit (CPU) accesses a virtual address but the MMU cannot find the corresponding physical address mapping or the access rights to the physical page are inconsistent.

[0090] Specifically, the CPU can access all peripherals connected to the address bus through the address bus, including physical memory, IO devices, etc., but the access address sent by the CPU is not the physical address of these peripherals on the address bus, but a virtual address. The MMU converts the virtual address into a physical address and then sends it out from the address bus. This virtual address and physical address conversion relationship on the MMU needs to be created, and the access rights of this physical page also need to be set.

[0091] (13) Primitive

[0092] Primitives are instructions that call core subroutines within an operating system. Primitives generally refer to program segments consisting of several instructions that implement a specific function and cannot be interrupted during execution. Certain operations called by processes within an operating system, such as queue operations, semaphore manipulation, and peripheral device checks and activation, cannot be interrupted once they begin. Otherwise, operational errors will occur, causing system chaos. Therefore, these operations must be implemented using primitives. Primitives are an integral part of the operating system core (which consists of a set of program modules, not processes), reside in memory, and typically execute in a managed state. Once a primitive begins execution, it must continue to execute without interruption.

[0093] In the Serverless platform, a serverless workflow is usually used to achieve collaboration between functions. Serverless workflow provides a way to build more complex applications by combining different functions. Please refer to Figure 1, which is a schematic diagram of a Serverless workflow provided by an embodiment of the present application. As shown in Figure 1, in the Serverless workflow, after function A is executed, function B will continue to execute with the output data of function A as input, and function C will continue to execute with the output data of function B as input, and finally the output data of function C will be returned to the user as a result. In Figure 1, there is a data dependency relationship between function B and function A, that is, function B depends on the data output by function A; there is also a data dependency relationship between function C and function B, that is, function C depends on the data output by function B.

[0094] Generally speaking, data dependencies between functions are defined by a directed acyclic graph (DAG), where the nodes in the DAG are functions and the edges between the nodes represent the data dependencies between the functions.

[0095] Please refer to Figure 2, which is a schematic diagram of data transfer between functions in a serverless workflow provided by an embodiment of the present application. As shown in Figure 2, in a serverless workflow, different functions may typically run on different servers, so data needs to be transferred between different functions over the network.

[0096] Please refer to Figure 3, which is a schematic diagram of data transfer between functions in the related art. As shown in Figure 3, assume that function A needs to pass a piece of data to function B, and function A and function B run on different servers. First, the coordinator calls the server of function A to start a container to run function A, and generates data that needs to be transferred during the running of function A. In order to transfer the data generated by function A to function B, the server of function A will first serialize the data (i.e., step 1 shown in Figure 3), and send the serialized data to the coordinator via the network. After the coordinator calls the server of function B to start function B, it sends the serialized data to the server of function B, and the server of function B deserializes the serialized data and passes the obtained data to function B.

[0097] As can be seen from Figure 3, when functions on different servers transmit data, not only does the coordinator need to forward the transmitted data, but the data also needs to be serialized and deserialized during the transmission process, resulting in low data transmission efficiency between functions and high performance overhead of data transmission.

[0098] In view of this, an embodiment of the present application provides a data transmission method, in which a first computing device that needs to consume data obtains the virtual address and page table of the data on a second computing device that is a data producer, and determines the physical address of the data on the second computing device based on the virtual address and page table of the data, and then directly reads the required data from the physical address of the second computing device, avoiding the data forwarding process, serialization process and deserialization process, which can effectively improve the data transmission efficiency and reduce the performance overhead brought by data transmission.

[0099] The data transmission method provided in the embodiments of the present application can be applied to an electronic device, which can serve as a computing device in a computing cluster. For example, the electronic device can be a physical device such as a server, a smartphone, a personal computer (PC), or a laptop. Furthermore, the electronic device can also be a virtualized device such as a virtual machine running on a server.

[0100] In general, the electronic device to which the data transmission method provided by this embodiment is applied may be a physical device such as a server serving as a computing device in a computing cluster, or may be a virtualized device such as a virtual machine.

[0101] Please refer to Figure 4, which is a schematic diagram of the structure of an electronic device 101 provided in an embodiment of the present application. As shown in Figure 4, electronic device 101 includes a processor 103, which is coupled to a system bus 105. Processor 103 can be one or more processors, each of which can include one or more processor cores. A display adapter (video adapter) 107 can drive a display 109, which is coupled to system bus 105. System bus 105 is coupled to an input / output (I / O) bus via a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with various I / O devices, such as an input device 117 (e.g., a touch screen), an external memory 121 (e.g., a hard disk, floppy disk, optical disk, or USB flash drive), a multimedia interface, etc., a transceiver 123 (capable of sending and / or receiving radio communication signals), a camera 155 (capable of capturing still and dynamic digital video images), and an external USB port 125. Optionally, the interface connected to the I / O interface 115 may be a USB interface.

[0102] The processor 103 may be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination thereof. Alternatively, the processor may be a dedicated device such as an ASIC.

[0103] Electronic device 101 can communicate with software deployment server 149 via network interface 129. Exemplarily, network interface 129 is a hardware network interface, such as a network card. Network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Alternatively, network 127 can be a wireless network, such as a WiFi network or a cellular network.

[0104] The hard drive interface 131 is coupled to the system bus 105. The hard drive interface is connected to the hard drive 133. The internal memory 135 is coupled to the system bus 105. The data running in the internal memory 135 may include the operating system (OS) 137 of the electronic device 101, the application 143, and the scheduler.

[0105] The operating system consists of a shell 139 and a kernel 141. Shell 139 is an interface between the user and the operating system's kernel. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: it waits for user input, interprets user input to the operating system, and processes various operating system output.

[0106] The kernel 141 consists of the parts of the operating system that manage memory, files, peripherals, and system resources. The kernel 141 directly interacts with the hardware. The operating system kernel typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, and I / O management.

[0107] The above describes the application scenarios and execution devices of the methods provided in the embodiments of the present application. The following describes the specific execution process of the methods provided in the embodiments of the present application. Please refer to Figure 5, which is a flowchart of a data transmission method provided in the embodiments of the present application. As shown in Figure 5, the data transmission method includes the following steps 501-503.

[0108] Step 501: A first computing device obtains a first virtual address, a network address, and a page table.

[0109] In this embodiment, the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between virtual addresses and physical addresses in the second computing device. Furthermore, the first computing device is the consumer of the first data, and the second computing device is the producer of the first data. In other words, the first data generated by the second computing device needs to be transmitted to the first computing device.

[0110] Generally speaking, after a function or process on a second computing device generates first data, the function or process on the second computing device will have a virtual address for the first data. However, the first data is actually stored in the memory of the second computing device, and the physical address corresponding to the first data in memory is unknown to the function or process that generated the first data. The second computing device maintains the mapping relationship between the virtual address and the physical address of the data using a page table. That is, the page table records the mapping relationship between the virtual address and the physical address of the first data.

[0111] Optionally, the data transmission method provided in this embodiment is applied to a serverless platform, where a first computing device and a second computing device are each used to run different functions on the serverless platform, and the input to the function run by the first computing device includes data generated by the function run by the second computing device. That is, the function run by the first computing device depends on the function run by the second computing device, so the first data generated by the function run by the second computing device needs to be transmitted to the function run on the first computing device.

[0112] The first computing device and the second computing device are, for example, different servers, or computing hardware (such as a central processing unit) or virtual machines deployed on different servers. In short, data transmission between the first computing device and the second computing device needs to be achieved through a network.

[0113] It should be noted that in addition to the Serverless platform, the data transmission method provided in this embodiment can also be applied to other distributed systems, that is, scenarios where data needs to be transmitted between different computing devices. This embodiment does not limit the specific scenarios in which the data transmission method is applied.

[0114] Step 502: Based on the first virtual address and the page table, the first computing device determines a first physical address of the first data on the second computing device.

[0115] After obtaining the first virtual address and the page table, the first computing device can determine the first physical address of the first data on the second computing device by querying the physical address corresponding to the first virtual address in the page table, that is, obtain the actual physical address of the first data on the memory of the second computing device.

[0116] Optionally, the way in which the first computing device determines the first physical address corresponding to the first data may be triggered by a function run by the first computing device. For example, when the first computing device runs the function, since the input data of the function includes the above-mentioned first data, the function will request access to the first data at runtime. Furthermore, since the first computing device does not actually store the first data, the first computing device does not record the physical address corresponding to the virtual address of the first data (i.e., the first virtual address). Therefore, when the function run by the first computing device requests access to the first virtual address corresponding to the first data, a page fault exception will be triggered.

[0117] In this way, in response to a page fault exception caused by a function running in the first computing device requesting access to the first virtual address, the first computing device determines the first physical address of the first data on the second computing device based on the first virtual address and the page table.

[0118] In this solution, by expanding the conventional page fault exception processing flow, a computing device can choose to read the required data from other computing devices through the network, thereby achieving efficient data transmission between computing devices and improving the feasibility of the solution.

[0119] Step 503: Based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

[0120] After determining the first physical address where the first data is stored on the second computing device, the first computing device can directly read the first data from the first physical address of the second computing device through the network based on the network address of the second computing device, thereby avoiding the serialization and deserialization process of the first data, and no longer needing to use other coordination devices to help forward the first data.

[0121] In general, this solution is equivalent to establishing a distributed shared memory on the second computing device, so that the data in the memory of the second computing device can be shared by other computing devices (such as the first computing device), so that other computing devices that rely on the data generated by the second computing device can directly read the data from the distributed shared memory of the second computing device, avoiding the serialization and deserialization processes on the data.

[0122] For ease of understanding, the following describes in detail how the first computing device obtains the first virtual address, the network address, and the page table.

[0123] Please refer to Figure 6, which is a flow chart of a data transmission method provided in an embodiment of the present application. As shown in Figure 5, the data transmission method includes the following steps 601-605.

[0124] Step 601: The second computing device sends a first virtual address and a virtual address segment to the coordination device.

[0125] In this embodiment, the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The first data may be generated by a function run by the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment. For example, the virtual address segment is the virtual address range corresponding to the data generated by the function run on the second computing device. The coordination device is used to forward the first virtual address and virtual address segment to the first computing device.

[0126] The coordination device is, for example, the scheduling node introduced above, which is used to schedule each computing device to run the corresponding function.

[0127] Step 602: The coordination device sends a first virtual address, a network address, and a virtual address segment to the first computing device.

[0128] After obtaining the first virtual address and virtual address segment sent by the second computing device, the coordination device may determine that the computing device that relies on the first data generated by the second computing device is the first computing device, and forward the first virtual address and virtual address segment to the first computing device. Furthermore, to ensure that the first computing device can successfully obtain the first data from the second computing device, the coordination device may also send the network address of the second computing device to the first computing device.

[0129] For example, since the coordination device schedules each computing device to run the corresponding function, the coordination device can determine the data dependency between computing devices running different functions based on the data dependency between the functions. For example, if function B depends on the data output by function A, and the first computing device runs function B and the second computing device runs function A, the coordination device can determine that the data generated on the second computing device needs to be transferred to the first computing device, so the coordination device forwards the first virtual address and virtual address segment sent by the second computing device to the first computing device.

[0130] Step 603: Based on the network address, the first computing device obtains a page table corresponding to the virtual address segment from the second computing device.

[0131] The page table obtained by the first computing device is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address. Since the virtual address segment includes the first virtual address, the page table actually includes the mapping relationship between the first virtual address and the first physical address.

[0132] Optionally, the first computing device reads a page table from the second computing device, for example, via RPC. Exemplarily, the first computing device sends a page table read request to the second computing device, where the page table read request includes a virtual address segment, and the page table read request is used to request reading a page table corresponding to the virtual address segment. In response to the page table read request sent by the first computing device, the second computing device sends the page table to the first computing device, so that the first computing device receives the page table sent by the second computing device.

[0133] Optionally, before the first computing device obtains the page table from the second computing device, the second computing device may set the permission of the target physical address to copy on write, where the target physical address is a physical address recorded in the page table and whose original permission is write. That is to say, after the first computing device obtains the page table, all the permissions corresponding to the physical addresses whose original permission is write recorded in the page table are changed to copy on write. In this way, even if the first computing device subsequently triggers a call and modifies the data in the second computing device, it will not affect the data in the second computing device, that is, the data in the second computing device will not be modified by the first computing device, thereby ensuring that the data generated by the function can be stored normally in the second computing device.

[0134] For example, assuming that the first data generated by the second computing device will be called by multiple computing devices, after the second computing device modifies the permission of the target physical address to copy on write, when any computing device calls and modifies the data in the second computing device, the data stored in the memory of the second computing device will not actually be modified, ensuring that other computing devices can call the data normally.

[0135] Step 604: Based on the first virtual address and the page table, the first computing device determines a first physical address of the first data on the second computing device.

[0136] Step 605: Based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

[0137] In this embodiment, steps 604-605 are similar to the above steps 502-503. Please refer to the above steps 502-503 for details, which will not be repeated here.

[0138] Since the first computing device and the second computing device are independent (for example, two independent servers), the first computing device and the second computing device may allocate the same virtual address space when allocating virtual address space to the function.

[0139] Based on this, in some embodiments, in order to avoid the situation where the functions run by the first computing device and the second computing device do not use the same virtual address, thereby causing address conflicts when data are called, in this embodiment, the virtual address space used by the functions on the first computing device and the second computing device can be uniformly allocated by the coordination device.

[0140] Exemplarily, before the first computing device runs a function that uses the above-mentioned first data, the first computing device receives a first virtual address space sent by the coordination device; then, the first computing device runs the function according to the first virtual address space, where the first virtual address space is used to indicate the space where the virtual address used by the function run by the first computing device is located.

[0141] In addition, before the first computing device runs the function that generates the above-mentioned first data, the second computing device receives the second virtual address space sent by the coordination device, and the second computing device runs the function according to the second virtual address space, wherein the second virtual address space is used to indicate the space where the virtual address used by the function run by the second computing device is located.

[0142] The first virtual address space and the second virtual address space do not overlap. In other words, the virtual address space used by the function running on the first computing device does not overlap with the virtual address space used by the function running on the second computing device, thereby preventing address segment mapping conflicts between the functions and ensuring that data calls between the functions can be carried out normally.

[0143] The above describes the execution process of the data transmission method provided in the embodiment of the present application. For ease of understanding, the following will introduce in detail the execution process of the data transmission method on the Serverless platform with specific examples.

[0144] Please refer to Figure 7, which is a schematic diagram of the architecture of a serverless platform provided in an embodiment of the present application. As shown in Figure 7, the serverless platform includes a coordination device and multiple computing devices, where the coordination device and the multiple computing devices can be deployed on different servers. The multiple computing devices are respectively used to run serverless functions. For example, different computing devices can run different functions on the same serverless workflow. Each computing device used to run a function can include a kernel module and a serverless runtime module.

[0145] Among them, the coordination device is used to plan the address space of each function in the Serverless workflow, thereby avoiding memory access conflicts when different functions use distributed shared memory to transmit data.

[0146] The kernel module on the computing device is the operating system kernel, providing functions with basic primitives for accessing distributed shared memory. Furthermore, when a function accesses unread remote memory data, the kernel module reads the corresponding data from the remote computing device over the network to the computing device where the function is located.

[0147] The Serverless runtime module on the computing device is used to receive the plan of the coordination device at runtime and dynamically create distributed shared memory for functions by calling the primitives of the kernel module, so that distributed shared memory can be used to efficiently transmit data between functions.

[0148] In this embodiment, the data transmission process can be divided into two parts: pre-execution and post-execution. The pre-execution process involves planning the virtual address space used by each function at runtime based on the serverless workflow's computational graph (DAG). The post-execution process involves a computing device reading the generated data from the distributed shared memory of another computing device after the function generates it.

[0149] For example, refer to Figure 8, which is a schematic diagram of a process for planning a virtual address space for a function in a serverless workflow, provided in an embodiment of the present application. As shown in Figure 8, the process for planning a virtual address space for a function in a serverless workflow includes the following steps 801-8012.

[0150] Step 801: Obtain the computation graph of the Serverless workflow.

[0151] The computation graph of a serverless workflow is, for example, uploaded by a user of the serverless platform. Furthermore, the computation graph of a serverless workflow can be a directed acyclic graph (DAG) as shown in Figure 1, which indicates the data dependencies between multiple functions in the serverless workflow.

[0152] Step 802: Get the maximum runtime memory limit of each function in the Serverless workflow.

[0153] The maximum memory limit of a function at runtime refers to the maximum memory space that the function can use at runtime.

[0154] Step 803: Get the maximum number of concurrent executions of each function in the Serverless workflow.

[0155] The maximum number of concurrent executions of a function refers to the number of instances of the same type of function that can be started simultaneously.

[0156] Step 804: Set the available address space to the maximum virtual address space supported by the operating system.

[0157] For example, in Linux operating systems, the maximum available virtual address space is 2 48 Bytes, so the available address space of Serverless workflow can be set to 2 48 byte.

[0158] Step 805, initial address (start) = the starting address of the maximum virtual address space.

[0159] Specifically, the variable start is initialized as the initial address, and the value of start is the starting address of the maximum virtual address space, for example, 0x400000.

[0160] Step 806: Initialize the content of the queue. Each element in the queue corresponds to an instance of a function. The number of instances of a function is the maximum number of concurrent executions of the function.

[0161] Based on the Serverless workflow's computation graph and the maximum number of concurrent executions for each function, a queue is initialized. Each element in the queue corresponds to an instance of a function, and the number of instances of a function is the maximum number of concurrent executions of the function.

[0162] Step 807: Determine whether the queue is empty.

[0163] If the queue is not empty, continue to step 808; if the queue is empty, stop planning the virtual address space for the function in the Serverless workflow.

[0164] Step 808: Take the element from the queue.

[0165] Step 809: Query the maximum memory limit of the function corresponding to the element.

[0166] Step 8010,<start,start+limit> Set to the virtual address space of the function corresponding to the element.

[0167] Step 8011, set start to start plus limit.

[0168] Step 8012, remove the element from the queue.

[0169] After step 8012 is completed, continue to execute the above step 807.

[0170] In general, based on the process shown in Figure 8, the coordination device can plan a corresponding virtual address space for each function in the Serverless workflow, and ensure that different functions in the same Serverless workflow correspond to different virtual address spaces.

[0171] During the execution of a serverless workflow, the coordination device schedules a computing device to run a function in the serverless workflow and sends the function's corresponding virtual address space to the computing device running the function. This way, when the computing device launches a container to run the function based on the coordination device's schedule, it calls the operating system kernel to ensure that the virtual address space of the launched container matches the virtual address space planned by the coordination device, thereby ensuring that the virtual address used by the running function complies with the plan. In other words, the operating system kernel in the computing device creates a container to run the function based on the virtual address space sent by the coordination device.

[0172] The following will introduce the process of the coordination device scheduling different computing devices to run functions as data producers and functions as data consumers.

[0173] When the coordination device schedules computing device A (corresponding to the second computing device in the above embodiment) to execute a function as a data producer, after computing device A completes the function and generates data, computing device A can call register_mem (where register_mem is a function provided by the operating system kernel) to create distributed shared memory. This allows subsequent functions executed on other computing devices to access the data generated by computing device A using shared memory. Specifically, the steps executed by computing device A after calling register_mem are as follows.

[0174] Step 1, traverse the page table of the container running the function.

[0175] The function running in the container acts as a data producer, generating data for use by other functions. The container's page table refers to the page table corresponding to the virtual address space used by the function.

[0176] Step 2: Set the permissions of all pages in the page table that have write permissions to copy on write.

[0177] Step 3: Record the container's page table.

[0178] Step 4: Generate an authorization key and return it to the user to prevent unauthorized containers from accessing data on the memory of the current computing device.

[0179] After computing device A has finished calling register_mem, it sends the authorization key, the virtual address of the data to be transmitted, and the virtual address segment corresponding to the function to the coordination device. The coordination device further sends the authorization key, the virtual address of the data to be transmitted, the virtual address segment corresponding to the function, and the network address of computing device A to computing device B (corresponding to the first computing device in the above embodiment) that runs the next function in the Serverless workflow. After computing device B receives the above information, it calls the rmap interface to establish a distributed shared memory with the container in computing device A to facilitate quick access to the data generated by the function in computing device A. The process of computing device B receiving information and calling the rmap interface is shown in Figure 9. Figure 9 is a schematic diagram of a process of a computing device calling the rmap interface provided in an embodiment of the present application. The process shown in Figure 9 includes the following steps 901-906.

[0180] Step 901 : The container in computing device B receives the authorization key, the virtual address of the data, the virtual address segment, and the network address of computing device A.

[0181] In step 902 , the container in computing device B calls the rmap interface to trigger execution of steps 903 - 906 .

[0182] Step 903, enter the kernel.

[0183] In step 904 , the kernel in computing device B communicates with computing device A corresponding to the network address via RPC to read the page table corresponding to the virtual address segment.

[0184] In step 905 , the kernel in computing device B maps the page table to the container of computing device A and records the page table in the kernel.

[0185] That is, the kernel establishes a relationship between the page table and the container of computing device A, so that when the physical address in the page table is queried, the data at the physical address can be read from computing device A.

[0186] Step 906: Return from the kernel to execute the container on computing device B.

[0187] After computing device B calls rmap, the function on computing device B can trigger the virtual address of the accessed data. Since the virtual address of the data accessed by the function is the virtual address on computing device A, no mapping relationship is established between the virtual address and the physical address in computing device B. Therefore, a page fault exception is triggered when accessing the virtual address of the data. After the page fault exception is triggered, the kernel in computing device B can perform page fault exception handling based on the steps shown in Figure 10. Figure 10 is a schematic flow chart of a page fault exception handling provided in an embodiment of the present application. The process in Figure 10 includes the following steps 1001-1006.

[0188] Step 1001: A function on computing device B accesses a virtual address of data, triggering a page fault exception.

[0189] Step 1002: Determine whether the address segment corresponding to the page in the page fault exception is mapped to a remote computing device.

[0190] Step 1003: If the address segment corresponding to the page is mapped to a remote computing device, a physical page is allocated.

[0191] Specifically, when computing device B executes step 905 shown in FIG9 , computing device B maps the virtual address segment to remote computing device A, thereby determining that the address segment corresponding to the page in the page fault exception is mapped to the remote computing device.

[0192] Step 1004: read the contents of the memory page in computing device A to the allocated physical page via the network.

[0193] That is, data at a specific physical address in the memory of computing device A is read to the allocated physical page through the network.

[0194] Step 1005 , record the page table entry of the page fault address as the allocated physical page, thereby eliminating the page fault exception, so that when the function on the computing device B accesses the virtual address of the data, it can trigger access to the data on the allocated physical page.

[0195] Step 1006: If the address segment corresponding to the page is not mapped to the remote computing device, the kernel's original page fault handling mechanism is called.

[0196] In this embodiment, the embodiment shown in Figure 9 and the embodiment shown in Figure 10 are combined to implement data transmission between different computing devices, and it is implemented based on the operating system kernel and is compatible with existing functions, so that existing functions can enjoy the benefits of reducing data serialization and deserialization overhead without modification, thereby improving data transmission efficiency.

[0197] For example, please refer to Figure 11, which is a schematic diagram of some scenarios in which the data transmission method provided in the embodiment of the present application is applied. As shown in Figure 11, the data transmission method provided in the embodiment of the present application can be applied to a variety of scenarios, where different scenarios are used to execute different Serverless workflows. For example, in Figure 11 (a), ML training represents a model training scenario in machine learning; in Figure 11 (b), ML prediction represents a scenario in machine learning where predictions are performed through a model; in Figure 11 (c), word count represents a batch processing application similar to the programming model MapReduce; in Figure 11 (d), FINRA represents a financial application whose main logic is to review various transactions.

[0198] When the data transmission method provided by the embodiment of the present application is applied in the various scenarios shown in Figure 11, the end-to-end delay in different scenarios can be reduced by 14-98%, where the performance improvement mainly comes from the reduction of serialization and deserialization.

[0199] The above embodiments introduce the data transmission method provided in the embodiments of the present application. The following will introduce the device that executes the above data transmission method.

[0200] Please refer to Figure 12, which is a schematic diagram of the structure of a first computing device provided in an embodiment of the present application. As shown in Figure 12, the first computing device provided in this embodiment includes: a transceiver module 1201, which is used to obtain a first virtual address, a network address, and a page table, wherein the first virtual address is a virtual address corresponding to the first data generated by the second computing device on the second computing device, the network address is the address of the second computing device, and the page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device; a processing module 1202, which is used to determine the first physical address of the first data on the second computing device based on the first virtual address and the page table; and the transceiver module 1201 is further used to read the first data from the first physical address of the second computing device based on the network address.

[0201] In one possible implementation, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes data generated by the function run by the second computing device.

[0202] In one possible implementation, the processing module 1202 is further used to: in response to a page fault exception caused by a function running in the first computing device requesting access to the first virtual address, the first computing device determines, based on the first virtual address and the page table, a first physical address of the first data on the second computing device.

[0203] In one possible implementation, the transceiver module 1201 is also used to: receive the first virtual address, network address and virtual address segment sent by the coordination device, the coordination device is used to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; based on the network address, obtain the page table corresponding to the virtual address segment from the second computing device, and the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address.

[0204] In one possible implementation, the transceiver module 1201 is further used to: send a page table read request to the second computing device, the page table read request includes a virtual address segment, and the page table read request is used to request to read the page table corresponding to the virtual address segment; and receive the page table sent by the second computing device.

[0205] In one possible implementation, the transceiver module 1201 is also used to receive the first virtual address space sent by the coordination device; the processing module 1202 is also used to run the function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0206] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the second virtual address space is a virtual address space used by a function running on the second computing device.

[0207] Please refer to Figure 13, which is a structural diagram of a second computing device provided by an embodiment of the present application. As shown in Figure 13, the second computing device provided by this embodiment includes: a transceiver module 1301, which is used to send a first virtual address and a virtual address segment to the coordination device, the first virtual address being the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment being used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address being located in the virtual address segment, and the coordination device being used to forward the first virtual address and the virtual address segment to the first computing device; in response to the first computing device requesting to read the virtual address segment, the transceiver module 1301 is also used to send a page table to the first computing device, the page table being used to record the mapping relationship between the virtual address in the virtual address segment and the physical address, and the page table records the mapping relationship between the first virtual address and the first physical address; in response to the first computing device requesting to access the first physical address, the transceiver module 1301 is also used to return the first data at the first physical address to the first computing device.

[0208] In one possible implementation, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes data generated by the function run by the second computing device.

[0209] In a possible implementation, the second computing device further includes: a processing module 1302, configured to set the permission of the target physical address to copy on write, where the target physical address is a physical address recorded in the page table and whose original permission is write.

[0210] In one possible implementation, the above-mentioned second computing device also includes a processing module 1302; the transceiver module 1301 is also used to receive the second virtual address space sent by the coordination device; the processing module 1302 is used to run the function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0211] In a possible implementation, the first virtual address space and the second virtual address space do not overlap, and the first virtual address space is a virtual address space used by a function running on the first computing device.

[0212] Next, an electronic device provided in an embodiment of the present application is introduced. Please refer to Figure 14. Figure 14 is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device 1400 can be specifically manifested as a server, a mobile phone, a tablet, a laptop computer, a smart wearable device, etc., which is not limited here. Specifically, the electronic device 1400 includes: a receiver 1401, a transmitter 1402, a processor 1403 and a memory 1404 (wherein the number of processors 1403 in the electronic device 1400 can be one or more, and Figure 14 takes one processor as an example), wherein the processor 1403 may include an application processor 14031 and a communication processor 14032. In some embodiments of the present application, the receiver 1401, the transmitter 1402, the processor 1403 and the memory 1404 may be connected via a bus or other means.

[0213] Memory 1404 may include read-only memory and random access memory, and provides instructions and data to processor 1403. A portion of memory 1404 may also include non-volatile random access memory (NVRAM). Memory 1404 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0214] Processor 1403 controls the operation of the electronic device. In specific applications, the various components of the electronic device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0215] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1403. Processor 1403 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1403. The above processor 1403 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1403 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1404, and processor 1403 reads the information in memory 1404 and, in conjunction with its hardware, completes the steps of the above method.

[0216] Receiver 1401 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the electronic device. Transmitter 1402 can be used to output digital or character information through the first interface. Transmitter 1402 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group. Transmitter 1402 can also include a display device such as a display screen.

[0217] The electronic device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the rendering method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0218] Please refer to Figure 15, which is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the method disclosed in Figure 5 above can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or products.

[0219] 15 schematically illustrates a conceptual partial view of an example computer-readable storage medium including a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein.

[0220] In one embodiment, computer-readable storage medium 1500 is provided using signal-bearing medium 1501. Signal-bearing medium 1501 may include one or more program instructions 1502 that, when executed by one or more processors, may provide the functionality or portions of the functionality described above with respect to FIG5 . Furthermore, program instructions 1502 in FIG15 also depict example instructions.

[0221] In some examples, signal bearing medium 1501 may include computer readable medium 1503 such as, but not limited to, a hard drive, compact disk (CD), digital video disk (DVD), digital tape, memory, ROM or RAM, and the like.

[0222] In some embodiments, the signal-bearing medium 1501 may include a computer-recordable medium 1504, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, or the like. In some embodiments, the signal-bearing medium 1501 may include a communication medium 1505, such as, but not limited to, a digital and / or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, or the like). Thus, for example, the signal-bearing medium 1501 may be communicated via a wireless form of the communication medium 1505 (e.g., a wireless communication medium conforming to the IEEE 802 standard or other transmission protocol).

[0223] The one or more program instructions 1502 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 1502 communicated to the computing device via one or more of computer-readable media 1503, computer-recordable media 1504, and / or communication media 1505.

[0224] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0225] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0226] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0228] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0229] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0230] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0231] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data transmission method, characterized in that, including: A first computing device obtains a first virtual address, a network address, and a page table. The first virtual address is the virtual address corresponding to first data generated by a second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between virtual addresses and physical addresses in the second computing device; Based on the first virtual address and the page table, the first computing device determines the first physical address where the first data is located on the second computing device; Based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

2. The method according to claim 1, characterized in that, The method is applied to a serverless platform. The first computing device and the second computing device are respectively used to run different functions on the serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

3. The method according to claim 1 or 2, characterized in that The step that based on the first virtual address and the page table, the first computing device determines the first physical address where the first data is located on the second computing device includes: In response to a page fault exception caused by a function running in the first computing device requesting to access the first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

4. The method according to any one of claims 1-3, characterized in that The step that the first computing device obtains a first virtual address, a network address, and a page table includes: The first computing device receives the first virtual address, the network address, and a virtual address segment sent by a coordination device. The coordination device is used to obtain the virtual address of the data generated by the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; Based on the network address, the first computing device obtains the page table corresponding to the virtual address segment from the second computing device. The page table is used to record the mapping relationship between virtual addresses and physical addresses in the virtual address segment.

5. The method according to claim 4, wherein The step that the first computing device obtains the page table corresponding to the virtual address segment from the second computing device includes: The first computing device sends a page table read request to the second computing device. The page table read request includes the virtual address segment and is used to request to read the page table corresponding to the virtual address segment; The first computing device receives the page table sent by the second computing device.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: The first computing device receives a first virtual address space sent by a coordination device; The first computing device runs a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

7. The method according to claim 6, characterized in that, The first virtual address space does not overlap with a second virtual address space, and the second virtual address space is the virtual address space used by the function running on the second computing device.

8. A data transmission method, characterized in that, including: The second computing device sends a first virtual address and a virtual address segment to the coordination device. The first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located within the virtual address segment. The coordination device is used to forward the first virtual address and the virtual address segment to the first computing device; In response to the first computing device requesting to read the virtual address segment, the second computing device sends a page table to the first computing device. The page table is used to record the mapping relationship between the virtual addresses in the virtual address segment and the physical addresses, and the page table records the mapping relationship between the first virtual address and the first physical address; In response to the first computing device requesting to access the first physical address, the second computing device returns the first data on the first physical address to the first computing device.

9. The method according to claim 8, wherein The method is applied to a Serverless platform. The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

10. The method according to claim 8 or 9, characterized in that The method further includes: The second computing device sets the permission of the target physical address to copy-on-write. The target physical address is the physical address recorded in the page table and whose original permission is write.

11. The method according to any one of claims 8-10, characterized in that The method further includes: The second computing device receives the second virtual address space sent by the coordination device; The second computing device runs a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

12. The method according to claim 11, wherein The first virtual address space does not overlap with the second virtual address space. The first virtual address space is the virtual address space used by the function running on the first computing device.

13. A first computing device, characterized in that, Including: A transceiver module, used to obtain a first virtual address, a network address, and a page table. The first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between the virtual addresses and the physical addresses in the second computing device; A processing module, used to determine the first physical address where the first data is located on the second computing device based on the first virtual address and the page table; The transceiver module is further used to read the first data from the first physical address of the second computing device based on the network address.

14. The device according to claim 13, characterized in that, The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

15. The device according to claim 13 or 14, characterized in that, The processing module is further used to: In response to a page fault exception caused by a function running in the first computing device requesting access to the first virtual address, the first computing device determines, based on the first virtual address and the page table, the first physical address where the first data is located on the second computing device.

16. The device according to any one of claims 13-15, characterized in that, The transceiver module is further configured to: Receive the first virtual address, the network address, and the virtual address segment sent by the coordination device, where the coordination device is configured to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; Based on the network address, obtain the page table corresponding to the virtual address segment from the second computing device, where the page table is used to record the mapping relationship between the virtual addresses and the physical addresses in the virtual address segment.

17. The device according to claim 16, characterized in that, The transceiver module is further configured to: Send a page table read request to the second computing device, where the page table read request includes the virtual address segment and is used to request to read the page table corresponding to the virtual address segment; Receive the page table sent by the second computing device.

18. The device according to any one of claims 13-17, wherein: The transceiver module is further configured to receive the first virtual address space sent by the coordination device; The processing module is further configured to run a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

19. The device according to claim 18, characterized in that, The first virtual address space does not overlap with the second virtual address space, and the second virtual address space is the virtual address space used by the function running on the second computing device.

20. A second computing device, characterized in that, Comprising: A transceiver module, configured to send a first virtual address and a virtual address segment to a coordination device, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment, and the coordination device is configured to forward the first virtual address and the virtual address segment to the first computing device; In response to the first computing device requesting to read the virtual address segment, the transceiver module is further configured to send a page table to the first computing device, where the page table is used to record the mapping relationship between the virtual addresses and the physical addresses in the virtual address segment, and the page table records the mapping relationship between the first virtual address and the first physical address; In response to the first computing device requesting to access the first physical address, the transceiver module is further configured to return the first data on the first physical address to the first computing device.

21. The device according to claim 20, characterized in that, The first computing device and the second computing device are respectively configured to run different functions on a Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

22. The device according to claim 20 or 21, characterized in that, The device further includes: A processing module, configured to set the permission of a target physical address to copy-on-write, where the target physical address is a physical address recorded in the page table and whose original permission is write.

23. The device according to any one of claims 20-22, characterized in that, The device further includes a processing module; The transceiver module is further configured to receive a second virtual address space sent by a coordination device; The processing module is configured to run a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

24. The device according to claim 23, characterized in that, The first virtual address space does not overlap with the second virtual address space, and the first virtual address space is the virtual address space used by a function running on the first computing device.

25. A computing device, characterized in that, Comprising a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the device executes the method according to any one of claims 1 to 12.

26. A data transmission system, characterized in that, Comprising a first computing device according to any one of claims 13-19, a second computing device according to any one of claims 20-24, and a coordination device, where the coordination device is used to implement data forwarding between the first computing device and the second computing device.

27. A computer storage medium, characterized in that, The computer storage medium stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

28. A computer program product, characterized in that, The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

Images