Data transmission method and related device

Abstract

A data transmission method is applied to data transmission among different computing devices. In the method, a first computing device needing to consume data acquires a virtual address of the data on a second computing device serving as a data producer and a page table, and determines a physical address of the data on the second computing device based on the virtual address of the data and the page table, therefore, the required data is directly read from the physical address of the second computing device, the forwarding process, the serialization process and the deserialization process of the data are avoided, the data transmission efficiency can be effectively improved, and the performance overhead caused by data transmission is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a data transmission method and related devices. Background Art

[0002] In the field of cloud computing, to reduce the burden on developers for deploying applications, the Serverless platform has emerged. The Serverless platform provides a micro architecture that can simplify the deployment of applications, enabling developers not to deploy, configure, or manage server services, that is, all server services required for application operation are provided by the Serverless platform.

[0003] When using the Serverless platform, developers only need to abstract the business into functions, package the functions and their dependencies into a container image, and upload the container image to the Serverless platform. When a function needs to be called (for example, when a network request needs to call the function), the Serverless platform will automatically start a container to implement the operation of the function. For functions on the Serverless platform, different functions may run on different servers, and different functions often need to interact with each other. Therefore, data interaction between functions on the Serverless platform involves data transmission between different servers.

[0004] Currently, in the related art, a function acting as a data producer serializes the generated data and sends the serialized data to a coordinator. The coordinator forwards the serialized data to a function acting as a data consumer, and finally the function acting as a data consumer performs deserialization to restore the original data. Among them, the data producer, coordinator, and data consumer are often located on different servers. The data needs to be forwarded by the coordinator during the transmission process, and there are serialization and deserialization operations during the data transmission process, resulting in low data transmission efficiency between functions. Summary of the Invention

[0005] This application provides a data transmission method that can reduce the performance overhead caused by data transmission.

[0006] The first aspect of the present application provides a data transmission method, which is applied to transmit data between different computing devices. The method includes: First, a first computing device obtains a first virtual address, a network address, and a page table. The first virtual address is the virtual address corresponding to the first data generated by a second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device. And the first computing device is the consumer of the first data, and the second computing device is the producer of the first data. That is, the first data generated by the second computing device needs to be transmitted to the first computing device.

[0007] Then, based on the first virtual address and the page table, the first computing device determines the first physical address where the first data is located on the second computing device, that is, obtains the physical address where the first data actually is in the memory of the second computing device.

[0008] Secondly, based on the network address of the second computing device, the first computing device can read the first data from the first physical address of the second computing device through the network, thereby avoiding the process of serializing and deserializing the first data, and there is no need to forward the first data through other coordination devices.

[0009] In this solution, the first computing device that needs to consume data obtains the virtual address and the page table of the data on the second computing device that is the data producer, and determines the physical address of the data on the second computing device based on the virtual address and the page table of the data, and then directly reads the required data from the physical address of the second computing device, avoiding the data forwarding process, serialization process, and deserialization process, and can effectively improve the data transmission efficiency and reduce the performance overhead brought by data transmission.

[0010] In a possible implementation, the data transmission method is applied to a Serverless platform. The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device. That is, the function run by the first computing device depends on the function run by the second computing device. Therefore, the first data generated by the function run by the second computing device needs to be transmitted to the function run by the first computing device.

[0011] In a possible implementation, the first computing device determines the first physical address where the first data is located on the second computing device, specifically including: in response to a page fault exception caused by a function running in the first computing device requesting access to a first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table. That is, the way for the first computing device to determine the first physical address corresponding to the first data can be triggered when a function running in the first computing device accesses the data and a page fault exception occurs.

[0012] In this solution, by expanding the processing flow of the conventional page fault exception, the computing device can select to read the required data from other computing devices through the network, thereby realizing efficient data transmission between computing devices and improving the feasibility of the solution; moreover, the computing device does not need the function itself to read the required data through the network, which can ensure that the existing functions can adapt to this solution without modification, improving the applicability of the solution.

[0013] In a possible implementation, the first computing device first receives the first virtual address, network address, and virtual address segment sent by the coordination device. The coordination device is used to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; then, based on the network address, the first computing device obtains the page table corresponding to the virtual address segment from the second computing device. The page table is used to record the mapping relationship between the virtual address and the physical address in the virtual address segment.

[0014] In this solution, by the coordination device forwarding the virtual address and virtual address segment with a lower data volume to the first computing device, and the first computing device directly obtaining the page table with a larger data volume from the second computing device based on the virtual address segment, the data volume forwarded by the coordination device can be reduced, and the efficiency of the first computing device to obtain various information can be improved.

[0015] In a possible implementation, the first computing device obtains the page table corresponding to the virtual address segment from the second computing device, including: the first computing device sends a page table reading request to the second computing device. The page table reading request includes the virtual address segment, and the page table reading request is used to request to read the page table corresponding to the virtual address segment; the first computing device receives the page table sent by the second computing device.

[0016] In a possible implementation, the data transmission method further includes: the first computing device receives the first virtual address space sent by the coordination device; the first computing device runs a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0017] In a possible implementation, the first virtual address space does not overlap with the second virtual address space, and the second virtual address space is the virtual address space used by functions running on the second computing device.

[0018] In this solution, the coordination device responsible for coordinating each computing device is responsible for allocating the virtual address space used by the functions in the first computing device, which can ensure that the virtual address space used by the functions in the first computing device does not overlap with the virtual address spaces used by the functions in other computing devices, thereby ensuring that there will be no address conflict when the computing devices read data from each other based on virtual addresses, and ensuring that the solution can be executed normally.

[0019] The second aspect of this application provides a data transmission method, including: the second computing device sends a first virtual address and a virtual address segment to the coordination device, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment, and the coordination device is used to forward the first virtual address and the virtual address segment to the first computing device; in response to the first computing device requesting to read the virtual address segment, the second computing device sends a page table to the first computing device, the page table is used to record the mapping relationship between the virtual addresses in the virtual address segment and the physical addresses, and the page table records the mapping relationship between the first virtual address and the first physical address; in response to the first computing device requesting to access the first physical address, the second computing device returns the first data on the first physical address to the first computing device.

[0020] In this solution, the first computing device that needs to consume data obtains the virtual address and page table of the data on the second computing device that is the data producer, and determines the physical address of the data on the second computing device based on the virtual address and page table of the data, and then directly reads the required data from the physical address of the second computing device, avoiding the data forwarding process, serialization process, and deserialization process, and can effectively improve the data transmission efficiency and reduce the performance overhead brought by data transmission.

[0021] In a possible implementation, the above data transmission method is applied to a Serverless platform, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

[0022] In a possible implementation, the above data transmission method further includes: the second computing device sets the permission of the target physical address to copy-on-write, where the target physical address is the physical address recorded in the page table and the original permission is write.

[0023] That is to say, after the first computing device obtains the page table, all the original permissions recorded in the page table corresponding to the physical addresses with the original permission of writing are changed to copy-on-write. In this way, even if the first computing device subsequently triggers a call to modify the data in the second computing device, it will not affect the data in the second computing device, that is, the data in the second computing device will not be modified by the first computing device, thus ensuring that the data generated by the function can be normally stored in the second computing device.

[0024] In a possible implementation manner, the above data transmission method further includes: the second computing device receives the second virtual address space sent by the coordination device; the second computing device runs a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0025] In a possible implementation manner, the first virtual address space does not overlap with the second virtual address space, and the first virtual address space is the virtual address space used by the function running on the first computing device.

[0026] The third aspect of this application provides a first computing device, including: a transceiver module, configured to obtain a first virtual address, a network address, and a page table, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the network address is the address of the second computing device, and the page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device; a processing module, configured to determine the first physical address where the first data is located on the second computing device based on the first virtual address and the page table; a transceiver module, configured to read the first data from the first physical address of the second computing device based on the network address.

[0027] In a possible implementation manner, the first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

[0028] In a possible implementation manner, the processing module is further configured to: in response to a page fault exception caused by a function running in the first computing device requesting to access the first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

[0029] In a possible implementation, the transceiver module is further configured to: receive a first virtual address, a network address, and a virtual address segment sent by a coordination device, where the coordination device is configured to obtain the virtual address of the data generated by a second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; based on the network address, obtain a page table corresponding to the virtual address segment from the second computing device, where the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address.

[0030] In a possible implementation, the transceiver module is further configured to: send a page table reading request to the second computing device, where the page table reading request includes the virtual address segment and is used to request to read the page table corresponding to the virtual address segment; receive the page table sent by the second computing device.

[0031] In a possible implementation, the transceiver module is further configured to receive a first virtual address space sent by the coordination device; the processing module is further configured to run a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0032] In a possible implementation, the first virtual address space does not overlap with the second virtual address space, and the second virtual address space is the virtual address space used by the function running on the second computing device.

[0033] A second computing device provided in the fourth aspect of this application includes: a transceiver module, configured to send a first virtual address and a virtual address segment to a coordination device, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment, and the coordination device is configured to forward the first virtual address and the virtual address segment to the first computing device; in response to the first computing device requesting to read the virtual address segment, the transceiver module is further configured to send a page table to the first computing device, where the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address, and the page table records the mapping relationship between the first virtual address and the first physical address; in response to the first computing device requesting to access the first physical address, the transceiver module is further configured to return the first data on the first physical address to the first computing device.

[0034] In a possible implementation, the first computing device and the second computing device are respectively configured to run different functions on a Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

[0035] In a possible implementation, the above-mentioned second computing device further includes: a processing module, configured to set the permission of the target physical address to copy-on-write, where the target physical address is the physical address recorded in the page table and whose original permission is write.

[0036] In a possible implementation, the above-mentioned second computing device further includes a processing module; a transceiver module, further configured to receive a second virtual address space sent by the coordination device; a processing module, configured to run a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0037] In a possible implementation, the first virtual address space does not overlap with the second virtual address space, and the first virtual address space is the virtual address space used by the function running on the first computing device.

[0038] A fifth aspect of the present application provides a computing device, including: a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the computing device executes the method according to any one of the implementations in the first aspect or the second aspect.

[0039] A sixth aspect of the present application provides a data transmission system, including: a first computing device according to any one of the implementations in the third aspect, a second computing device according to any one of the implementations in the fourth aspect, and a coordination device, where the coordination device is configured to implement data forwarding between the first computing device and the second computing device.

[0040] A seventh aspect of the present application provides a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, it causes the computer to execute the method according to any one of the implementations in the first aspect.

[0041] An eighth aspect of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the method according to any one of the implementations in the first aspect.

[0042] A ninth aspect of the present application provides a chip, including one or more processors. Some or all of the processors are used to read and execute the computer program stored in the memory to execute the method according to any one of the implementations in the first aspect above.

[0043] Optionally, the chip includes a memory, which is connected to the processor through a circuit or wire. Optionally, the chip further includes a communication interface, to which the processor is connected. The communication interface is used to receive data and / or information to be processed. The processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface may be an input / output interface. The method provided in this application may be implemented by one chip or by multiple chips working together.

[0044] Among them, for the technical effects brought by any of the design manners in the second aspect to the sixth aspect, reference may be made to the technical effects brought by different implementation manners in the first aspect above, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of a Serverless workflow provided by an embodiment of this application;

[0046] Figure 2 Schematic diagram of data transfer between functions in a Serverless workflow provided by an embodiment of this application;

[0047] Figure 3 Schematic diagram of data transfer between functions in related art;

[0048] Figure 4 Schematic diagram of the structure of an electronic device 101 provided by an embodiment of this application;

[0049] Figure 5 Schematic diagram of the process of a data transmission method provided by an embodiment of this application;

[0050] Figure 6 Schematic diagram of the process of a data transmission method provided by an embodiment of this application;

[0051] Figure 7 Schematic diagram of the architecture of a Serverless platform provided by an embodiment of this application;

[0052] Figure 8 Schematic diagram of the process of planning a virtual address space for functions in a Serverless workflow provided by an embodiment of this application;

[0053] Figure 9 Schematic diagram of the process of a computing device calling the rmap interface provided by an embodiment of this application;

[0054] Figure 10 Schematic diagram of the process of page fault exception handling provided by an embodiment of this application;

[0055] Figure 11 Some schematic diagrams of scenarios applied to the data transmission method provided by the embodiments of the present application;

[0056] Figure 12 Schematic diagram of the structure of a first computing device provided by the embodiments of the present application;

[0057] Figure 13 Schematic diagram of the structure of a second computing device provided by the embodiments of the present application;

[0058] Figure 14 Schematic diagram of the structure of an electronic device provided by the embodiments of the present application;

[0059] Figure 15 Schematic diagram of the structure of a computer-readable storage medium provided by the embodiments of the present application. Detailed implementation manners

[0060] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Those of ordinary skill in the art will know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0061] Terms such as "first" and "second" in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such descriptions can be interchanged under appropriate circumstances so that the embodiments can be implemented in an order other than that shown or described in the present application. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The named or numbered process steps can be changed in the execution order according to the technical objectives to be achieved, as long as the same or similar technical effects can be achieved.

[0062] The division of units in this application is a logical division. In actual implementation, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed among each other can be through some interfaces. The indirect couplings or communication connections between units can be in electrical or other similar forms, which are not limited in this application. Moreover, the units or subunits described as separate components can be physically separated or not, can be physical units or not, or can be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0063] For ease of understanding, some technical terms involved in this embodiment will be introduced below.

[0064] (1) Serverless

[0065] Serverless is a model of cloud computing and is essentially a new type of Internet architecture. Serverless does not mean there are no servers, but rather that developers do not need to concern themselves with the management and maintenance of servers. In traditional application architectures, developers need to focus on server configuration, scaling, maintenance, and monitoring. In a Serverless architecture, developers only need to focus on the business logic of the application without having to consider the underlying server management. The cloud service provider will automatically scale and manage the underlying server resources and allocate resources automatically according to the load of the application.

[0066] (2) Function

[0067] A function is the basic running unit in serverless computing and is essentially a piece of program code uploaded by the user. When the Serverless platform receives a request to invoke a function, the platform will start a container and run the function.

[0068] (3) Container

[0069] A container is a way to run processes, which can simplify the deployment of application programs and provide stronger isolation between processes. In this embodiment, the container is actually the running environment of the function and can achieve isolation between functions.

[0070] (4) Scheduling Node

[0071] When the Serverless platform runs multiple functions simultaneously, a scheduling node will uniformly schedule the functions according to the dependency relationships between the functions. Usually, the scheduling node is also called the coordinator.

[0072] (5) Host

[0073] A host machine refers to the server that runs functions. In a Serverless platform, the scheduling node will select an idle host machine to execute functions.

[0074] (6) Serialization

[0075] Serialization is the process of converting the state information of an object into a form that can be stored or transmitted. Simply put, serialization is to convert the in-memory data of a language into a continuous binary data.

[0076] (7) Deserialization

[0077] Deserialization refers to the process of converting the continuous binary data generated during the serialization process into data in memory.

[0078] (8) Page table

[0079] A page table is a data structure that records the mapping relationship between virtual addresses and physical addresses. Generally, in a computer, the Memory Management Unit (MMU) can achieve the conversion between virtual addresses and physical addresses by querying the page table.

[0080] (9) Virtual address (VA)

[0081] A virtual address refers to the address of a storage unit seen from the perspective of an application program in a computer architecture. A virtual address often differs from the physical address of the storage unit, and it is necessary to use an address translator to convert the virtual address into a physical address (PA).

[0082] Simply put, a virtual address is usually the address generated by the CPU in a computer when an application program is running; the physical address is a real address in physical memory.

[0083] (10) Copy on write (COW)

[0084] Copy on write is a resource management technique used in computer programming, which can effectively implement the copying or replication operation of modifiable resources. If a resource is copied but not modified, there is no need to create a new resource, and the resource can be shared between the copy and the original; a copy must still be created when the resource is modified. By sharing resources in the way of copy on write, the resource consumption of unmodified copies can be significantly reduced, while adding a small amount of overhead to the resource modification operation.

[0085] (11) Remote Procedure Call (RPC)

[0086] RPC is a protocol for requesting services from a remote computer over a network without the need to understand the underlying network technology. Specifically, for servers A and B, an application is deployed on server A and wants to call a function provided by an application on server B. Since they are not in the same memory space and cannot be directly called, it is necessary to express the semantics of the call and convey the call data over the network. RPC provides a way to obtain services from a remote server over the network.

[0087] (12) Page fault

[0088] A page fault occurs when the Central Processing Unit (CPU) accesses a virtual address and the Memory Management Unit (MMU) is unable to find the corresponding physical address mapping relationship or the access rights to the physical page are inconsistent.

[0089] Specifically, the CPU can access all peripherals connected to the address bus through the address bus, including physical memory, I / O devices, etc. However, the access address sent from the CPU is not the physical address of these peripherals on the address bus, but a virtual address. The MMU converts the virtual address into a physical address and then sends it from the address bus. The conversion relationship between the virtual address and the physical address on the MMU needs to be created, and the access rights of this physical page also need to be set.

[0090] (13) Primitive

[0091] A primitive is an instruction for calling a subroutine in the core layer of an operating system. A primitive generally refers to a program segment composed of several instructions to implement a specific function and cannot be interrupted during execution. In an operating system, some operations called by a process, such as queue operations, operations on semaphores, and operations to check and start peripherals, once started, cannot be interrupted, otherwise operation errors will occur and the system will be chaotic. Therefore, these operations are all implemented using primitives. A primitive is a component of the operating system core (composed of a set of program modules rather than a process) and resides in memory permanently, usually executed in supervisor mode. Once a primitive starts execution, it must be executed continuously without interruption.

[0092] In a Serverless platform, a Serverless workflow is usually adopted to achieve collaboration between functions. Serverless workflow provides a way to build more complex applications by combining different functions. Please refer to Figure 1 , Figure 1A schematic diagram of a Serverless workflow provided by an embodiment of the present application. As Figure 1 shown, in Serverless workflow, after function A finishes execution, function B will continue to execute with the output data of function A as input, and function C will continue to execute with the output data of function B as input. Finally, the output data of function C will be returned to the user as the result. In Figure 1 , there is a data dependency relationship between function B and function A, that is, function B depends on the data output by function A; there is also a data dependency relationship between function C and function B, that is, function C depends on the data output by function B.

[0093] Generally speaking, the data dependency relationship between functions is defined by a directed acyclic graph (DAG). Among them, the nodes in the DAG are functions, and the edges between the nodes in the DAG represent the data dependency relationship between functions.

[0094] Please refer to Figure 2 , Figure 2 A schematic diagram of data transfer between functions in a Serverless workflow provided by an embodiment of the present application. As Figure 2 shown, in Serverless workflow, different functions may usually run on different servers. Therefore, data needs to be transmitted between different functions through the network.

[0095] Please refer to Figure 3 , Figure 3 A schematic diagram of data transfer between functions in related technologies. As Figure 3 shown, assume that function A needs to transfer a piece of data to function B, and function A and function B run on different servers. First, the coordinator calls the server of function A to start a container to run function A, and generates the data to be transferred during the running of function A. In order to transfer the data generated by function A to function B, the server of function A will first serialize the data (i.e., Figure 3 step 1 shown in

[0096] and send the serialized data to the coordinator through the network. After the coordinator calls the server of function B to start function B, it sends the serialized data to the server of function B. The server of function B deserializes the serialized data, and passes the obtained data to function B. Figure 3 It can be seen that when functions on different servers transfer data, not only does the coordinator need to forward the transferred data, but also the data needs to be serialized and deserialized during the transfer process, resulting in low data transfer efficiency between functions and large performance overhead for data transfer.

[0097] In view of this, an embodiment of the present application provides a data transmission method. A first computing device that needs to consume data obtains the virtual address and page table of the data on a second computing device that is a data producer, and determines the physical address of the data on the second computing device based on the virtual address and page table of the data. Furthermore, the required data is directly read from the physical address of the second computing device, avoiding the data forwarding process, serialization process, and deserialization process, which can effectively improve the data transmission efficiency and reduce the performance overhead brought by data transmission.

[0098] The data transmission method provided by the embodiment of the present application can be applied to an electronic device, and this electronic device can be a computing device in a computing cluster. Exemplarily, this electronic device can be a physical device such as a server, a smartphone (mobilephone), a personal computer (PC), a laptop computer, etc. In addition, this electronic device can also be a virtualized device such as a virtual machine running on a server.

[0099] Generally speaking, the electronic device applying the data transmission method provided by this embodiment can be a physical device such as a server that is a computing device in a computing cluster, or a virtualized device such as a virtual machine.

[0100] Reference can be made to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device 101 provided by an embodiment of the present application. As Figure 4 shown, the electronic device 101 includes a processor 103, and the processor 103 is coupled to a system bus 105. The processor 103 can be one or more processors, and each processor can include one or more processor cores. A display adapter (videoadapter) 107, and the display adapter can drive a display 109, and the display 109 is coupled to the system bus 105. The system bus 105 is coupled to an input / output (I / O) bus through a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with a variety of I / O devices, such as an input device 117 (such as: a touch screen, etc.), an external memory 121, (for example, a hard disk, a floppy disk, an optical disc, or a USB flash drive), a multimedia interface, etc.). A transceiver 123 (which can send and / or receive radio communication signals), a camera 155 (which can capture static and dynamic digital video images), and an external USB port 125. Optionally, the interface connected to the I / O interface 115 can be a USB interface.

[0101] Among them, the processor 103 can be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination of the above. Optionally, the processor can be a dedicated device such as an ASIC.

[0102] The electronic device 101 can communicate with the software deployment server 149 through the network interface 129. Exemplarily, the network interface 129 is a hardware network interface, such as a network card. The network 127 can be an external network, such as the Internet, or an internal network, such as an Ethernet or a virtual private network (VPN). Optionally, the network 127 can also be a wireless network, such as a WiFi network, a cellular network, etc.

[0103] The hard disk drive interface 131 is coupled to the system bus 105. The hardware drive interface is connected to the hard disk drive 133. The internal memory 135 is coupled to the system bus 105. The data running in the internal memory 135 can include the operating system (OS) 137, application programs 143, and a schedule of the electronic device 101.

[0104] The operating system includes a Shell 139 and a kernel 141. The Shell 139 is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waits for the user's input, interprets the user's input to the operating system, and processes various output results of the operating system.

[0105] The kernel 141 consists of those parts in the operating system that are used to manage memory, files, peripherals, and system resources. The kernel 141 directly interacts with the hardware. The operating system kernel usually runs processes and provides inter-process communication, provides CPU time slice management, interrupts, memory management, and IO management, etc.

[0106] The above has introduced the scenarios and execution devices to which the method provided in the embodiments of the present application is applied. The following will introduce the specific execution process of the method provided in the embodiments of the present application. Please refer to Figure 5 , Figure 5 which is a schematic flowchart of a data transmission method provided in an embodiment of the present application. As Figure 5 shown, the data transmission method includes the following steps 501-503.

[0107] Step 501, the first computing device obtains a first virtual address, a network address, and a page table.

[0108] In this embodiment, the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device. Moreover, the first computing device is the consumer of the first data, and the second computing device is the producer of the first data. That is, the first data generated by the second computing device needs to be transmitted to the first computing device.

[0109] Generally speaking, after a function or process on the second computing device generates the first data, the function or process on the second computing device will have the virtual address of the first data. In fact, the first data is stored in the memory of the second computing device, and the physical address corresponding to the first data in the memory is unknown to the function or process that generates the first data. The second computing device will maintain the mapping relationship between the virtual address and the physical address of the data through the page table, that is, the mapping relationship between the virtual address and the physical address of the first data will be recorded in the page table.

[0110] Optionally, the data transmission method provided in this embodiment is applied to the Serverless platform. The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device. That is, the function run by the first computing device depends on the function run by the second computing device. Therefore, the first data generated by the function run by the second computing device needs to be transmitted to the function run on the first computing device.

[0111] Among them, the first computing device and the second computing device are, for example, different servers, or computing hardware (such as a central processing unit) or virtual machines deployed on different servers. In short, data transmission between the first computing device and the second computing device needs to be realized through a network.

[0112] It should be noted that in addition to the Serverless platform, the data transmission method provided in this embodiment can also be applied to other distributed systems, that is, scenarios where data needs to be transmitted between different computing devices. This embodiment does not limit the specific scenario to which the data transmission method is applied.

[0113] Step 502, based on the first virtual address and the page table, the first computing device determines the first physical address where the first data is located on the second computing device.

[0114] After obtaining the first virtual address and the page table, the first computing device can determine the first physical address where the first data is located on the second computing device by querying the physical address corresponding to the first virtual address in the page table, that is, obtain the physical address where the first data actually is in the memory of the second computing device.

[0115] Optionally, the way for the first computing device to determine the first physical address corresponding to the first data can be triggered by a function run by the first computing device. Exemplarily, when the first computing device runs a function, since the input data of the function includes the above-mentioned first data, the function will request to access the first data during operation. And since the first computing device actually does not store the first data and there is no record in the first computing device of the physical address corresponding to the virtual address of the first data (i.e., the first virtual address), when the function run by the first computing device requests to access the first virtual address corresponding to the first data, a page fault exception will be triggered.

[0116] In this way, in response to the page fault exception caused by the function run by the first computing device requesting to access the first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

[0117] In this solution, by expanding the processing flow of the conventional page fault exception, the computing device can choose to read the required data from other computing devices through the network, thereby realizing efficient data transmission between computing devices and improving the feasibility of the solution.

[0118] Step 503, based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

[0119] After determining the first physical address where the first data is stored on the second computing device, the first computing device can directly read the first data from the first physical address of the second computing device through the network based on the network address of the second computing device, thus avoiding the process of serializing and deserializing the first data and not requiring other coordination devices to help forward the first data.

[0120] Generally speaking, this solution is equivalent to establishing a distributed shared memory on the second computing device, enabling the data in the memory of the second computing device to be shared by other computing devices (such as the first computing device), so that other computing devices dependent on the data generated by the second computing device can directly read the data from the distributed shared memory of the second computing device, avoiding the process of serializing and deserializing the data.

[0121] For ease of understanding, the following will detail how the first computing device obtains the first virtual address, network address, and page table.

[0122] Please refer to Figure 6 , Figure 6 , which is a schematic flowchart of a data transmission method provided by an embodiment of the present application. As Figure 5 shown, the data transmission method includes the following steps 601-605.

[0123] Step 601, the second computing device sends a first virtual address and a virtual address segment to the coordination device.

[0124] In this embodiment, the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. Among them, the first data may be generated by a function run by the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment. For example, the virtual address segment is the virtual address range corresponding to the data generated by the function running on the second computing device. The coordination device is used to forward the first virtual address and the virtual address segment to the first computing device.

[0125] Among them, the coordination device is, for example, the scheduling node introduced above, and is used to schedule each computing device to run the corresponding function.

[0126] Step 602, the coordination device sends the first virtual address, the network address, and the virtual address segment to the first computing device.

[0127] After obtaining the first virtual address and the virtual address segment sent by the second computing device, the coordination device can determine that the computing device dependent on the first data generated by the second computing device is the first computing device, and forward the first virtual address and the virtual address segment to the first computing device. In addition, in order to ensure that the first computing device can successfully obtain the first data from the second computing device, the coordination device may also send the network address of the second computing device to the first computing device.

[0128] Exemplarily, since the coordination device schedules each computing device to run the corresponding function, the coordination device can determine the data dependency relationship between the computing devices running different functions based on the data dependency relationship between the functions. For example, when function B depends on the data output by function A, and the first computing device runs function B and the second computing device runs function A, the coordination device can determine that the data generated on the second computing device needs to be transmitted to the first computing device, so the coordination device forwards the first virtual address and the virtual address segment sent by the second computing device to the first computing device.

[0129] Step 603, based on the network address, the first computing device obtains a page table corresponding to the virtual address segment from the second computing device.

[0130] Among them, the page table obtained by the first computing device is used to record the mapping relationship between the virtual addresses in the virtual address segment and the physical addresses. Since the virtual address segment includes the first virtual address, the page table actually includes the mapping relationship between the first virtual address and the first physical address.

[0131] Optionally, the first computing device reads the page table from the second computing device in a way such as through RPC. Exemplarily, the first computing device sends a page table reading request to the second computing device. The page table reading request includes the virtual address segment, and the page table reading request is used to request to read the page table corresponding to the virtual address segment. In response to the page table reading request sent by the first computing device, the second computing device will send the page table to the first computing device, so that the first computing device receives the page table sent by the second computing device.

[0132] Optionally, before the first computing device obtains the page table from the second computing device, the second computing device can set the permission of the target physical address to copy-on-write. The target physical address is the physical address recorded in the page table and whose original permission is write. That is to say, after the first computing device obtains the page table, the permissions corresponding to the physical addresses whose original permissions are write recorded in the page table are all changed to copy-on-write. In this way, even if the first computing device triggers a call and modifies the data in the second computing device subsequently, it will not affect the data in the second computing device, that is, the data in the second computing device will not be modified by the first computing device, thus ensuring that the data generated by the function can be normally stored in the second computing device.

[0133] For example, assuming that the first data generated by the second computing device will be called by multiple computing devices, after the second computing device modifies the permission of the target physical address to copy-on-write, when any computing device calls and modifies the data in the second computing device, it will not actually modify the data stored in the memory of the second computing device, ensuring that other computing devices can normally call the data.

[0134] Step 604, based on the first virtual address and the page table, the first computing device determines the first physical address where the first data is located on the second computing device.

[0135] Step 605, based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

[0136] In this embodiment, steps 604 - 605 are similar to the above steps 502 - 503. For details, please refer to the above steps 502 - 503 and will not be elaborated here.

[0137] Since the first computing device and the second computing device are independent of each other (for example, two independent servers), when the first computing device and the second computing device allocate virtual address spaces for functions, it is possible that the same virtual address space will be allocated.

[0138] Based on this, in some embodiments, in order to prevent functions running on the first computing device and the second computing device from using the same virtual address, which may cause an address conflict during data call, in this embodiment, the coordination device may be used to uniformly allocate the virtual address spaces used by the functions on the first computing device and the second computing device.

[0139] Exemplarily, before the first computing device runs a function that uses the above-mentioned first data, the first computing device receives the first virtual address space sent by the coordination device; then, the first computing device runs the function according to the first virtual address space, where the first virtual address space is used to indicate the space where the virtual address used by the function running on the first computing device is located.

[0140] In addition, before the first computing device runs a function that generates the above-mentioned first data, the second computing device receives the second virtual address space sent by the coordination device, and the second computing device runs the function according to the second virtual address space, where the second virtual address space is used to indicate the space where the virtual address used by the function running on the second computing device is located.

[0141] Among them, the above-mentioned first virtual address space does not overlap with the second virtual address space. That is to say, the virtual address space used by the function running in the first computing device does not overlap with the virtual address space used by the function running in the second computing device, thus ensuring that there will be no problem of address segment mapping conflict between functions and ensuring that data calls can be normally implemented between functions.

[0142] The above describes the execution process of the data transmission method provided in the embodiments of the present application. For ease of understanding, the following will introduce in detail the execution process of the data transmission method in the Serverless platform with specific examples.

[0143] Please refer to Figure 7 , Figure 7 which is a schematic diagram of the architecture of a Serverless platform provided in the embodiments of the present application. As Figure 7As shown in the figure, the Serverless platform includes a coordination device and multiple computing devices, where the coordination device and the multiple computing devices can be deployed on different servers. Among them, the multiple computing devices are respectively used to run Serverless functions. For example, different multiple computing devices respectively run different functions on the same Serverless workflow. Each computing device used to run a function may include a kernel module and a Serverless runtime module.

[0144] Among them, the coordination device is used to plan the address space of each function in the Serverless workflow, so as to avoid memory access conflicts when different functions use distributed shared memory to transfer data.

[0145] The kernel module on the computing device is the operating system kernel, which provides basic primitives for functions to access distributed shared memory. And when a function accesses remote memory data that has not been read, the kernel module will read the corresponding data from the remote computing device to the computing device where the function is located through the network.

[0146] The Serverless runtime module on the computing device is used to receive the plan of the coordination device at runtime, and dynamically create distributed shared memory for the function by calling the primitives of the kernel module, so that functions can efficiently transfer data using distributed shared memory.

[0147] In this embodiment, the data transmission process can be divided into two parts, namely before running and after running. Among them, the function implemented before running is to plan the virtual address space used during the running of each function according to the computational graph (i.e., DAG) of the Serverless workflow. The function implemented after running is that after the function generates data, the computing device reads the data generated by the function from the distributed shared memory of another computing device.

[0148] Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic flow diagram of planning the virtual address space for functions in the Serverless workflow provided by the embodiment of the present application. As Figure 8 shown, the process of planning the virtual address space for functions in the Serverless workflow includes the following steps 801-8012.

[0149] Step 801, obtain the computational graph of the Serverless workflow.

[0150] Among them, the computation graph of Serverless workflow is uploaded by users using the Serverless platform, for example. And the computation graph of Serverless workflow can be a Figure 1 directed acyclic graph as shown, which is used to indicate the data dependency relationships among multiple functions in Serverless workflow.

[0151] Step 802: Obtain the maximum memory limit at runtime for each function in the Serverless workflow.

[0152] Among them, the maximum memory limit at runtime of a function refers to the maximum memory space that the function can use during runtime.

[0153] Step 803: Obtain the maximum concurrent execution number for each function in the Serverless workflow.

[0154] Among them, the maximum concurrent execution number of a function refers to the number of instances of the same type of function that are started simultaneously.

[0155] Step 804: Set the available address space to the maximum virtual address space supported by the operating system configuration.

[0156] For example, in the Linux operating system, the maximum available virtual address space is 2 48 bytes. Therefore, the available address space of the Serverless workflow can be set to 2 48 bytes.

[0157] Step 805: Initial address (start) = the starting address of the maximum virtual address space.

[0158] Specifically, initialize the variable start as the initial address, and the value of start is the starting address of the maximum virtual address space, for example, 0x400000.

[0159] Step 806: Initialize the content of the queue. Each element in the queue corresponds to an instance of a function, and the number of instances of a function is the maximum concurrent execution number of the function.

[0160] Based on the computation graph of the Serverless workflow and the maximum concurrent execution number of each function, initialize a queue. Each element in the queue corresponds to an instance of a function, and the number of instances of a function is the maximum concurrent execution number of the function.

[0161] Step 807: Determine whether the queue is empty.

[0162] If the queue is not empty, continue to execute step 808; if the queue is empty, stop planning the virtual address space for the functions in Serverless workflow.

[0163] Step 808, take out the element from the queue.

[0164] Step 809, query the maximum memory limit (limit) of the function corresponding to the element.

[0165] Step 8010, set <start, start + limit> as the virtual address space of the function corresponding to the element.

[0166] Step 8011, set start as start plus limit.

[0167] Step 8012, remove the element from the queue.

[0168] After executing step 8012, continue to execute the above step 807.

[0169] Generally speaking, based on Figure 8 the process shown, the coordination device can plan the corresponding virtual address space for each function in Serverless workflow, and ensure that different functions in the same Serverless workflow correspond to different virtual address spaces.

[0170] Then, during the execution of Serverless workflow, the coordination device schedules a computing device to run a function in Serverless workflow, and sends the virtual address space corresponding to the function to the computing device running the function. In this way, when the computing device starts a container to run the function based on the scheduling of the coordination device, it calls the operating system kernel to ensure that the virtual address space of the started container conforms to the virtual address space planned by the coordination device, and further ensures that the virtual address used by the running function conforms to the plan. That is, the operating system kernel in the computing device will create a container for running the function according to the virtual address space sent by the coordination device.

[0171] The following will separately introduce the processes when the coordination device schedules different computing devices to run the function as a data producer and the function as a data consumer.

[0172] When coordinating the scheduling of computing device A (corresponding to the second computing device in the above embodiment) to run a function as a data producer, after computing device A finishes running the function and generates data, computing device A can call register_mem (where register_mem is a function provided by the operating system kernel) to create a distributed shared memory, so that subsequent functions running on other computing devices can access the data generated by computing device A in the way of shared memory. Specifically, the steps executed after computing device A calls register_mem are as follows.

[0173] Step 1, traverse the page table of the container running the function.

[0174] Among them, the function run by the container is a function as a data producer, which generates data for other functions to use. The page table of the container refers to the page table corresponding to the virtual address space used by the function.

[0175] Step 2, set the permissions of all pages with write permissions in the page table to copy-on-write.

[0176] Step 3, record the page table of the container.

[0177] Step 4, generate an authorization key and return it to the user to prevent unauthorized containers from accessing the data in the memory of the current computing device.

[0178] After computing device A finishes calling register_mem, it sends the authorization key, the virtual address of the data to be transmitted, and the virtual address segment corresponding to the function to the coordination device. The coordination device further sends the authorization key, the virtual address of the data to be transmitted, the virtual address segment corresponding to the function, and the network address of computing device A to computing device B (corresponding to the first computing device in the above embodiment) that runs the next function in the Serverless workflow. After receiving the above information, computing device B calls the rmap interface to establish a distributed shared memory with the container in computing device A, so as to quickly access the data generated by the function in computing device A. Among them, the process of computing device B receiving information and calling the rmap interface is as Figure 9 shown. Figure 9 It is a schematic flowchart of a process for a computing device to call the rmap interface provided by an embodiment of the present application. Figure 9 The process shown includes the following steps 901-906.

[0179] Step 901, the container in computing device B receives the authorization key, the virtual address of the data, the virtual address segment, and the network address of computing device A.

[0180] Step 902, the container in computing device B invokes the rmap interface to trigger the execution of steps 903-906.

[0181] Step 903, enter the kernel.

[0182] Step 904, the kernel in computing device B communicates with computing device A corresponding to the network address through RPC to read the page table corresponding to the virtual address segment.

[0183] Step 905, the kernel in computing device B maps the page table to the container of computing device A and records the page table in the kernel.

[0184] That is, the kernel establishes a relationship between the page table and the container of computing device A, so that when the physical address in the page table is queried, the data at the physical address can be read from computing device A.

[0185] Step 906, return from the kernel to execute the container on computing device B.

[0186] After computing device B invokes rmap, the function on computing device B can trigger the virtual address for accessing data. Since the virtual address of the data accessed by the function is the virtual address on computing device A and there is no mapping relationship with the physical address established in computing device B, a page fault exception will be triggered when accessing the virtual address of the data. After the page fault exception is triggered, the kernel in computing device B can be based on Figure 10 The steps shown are used to handle the page fault exception. Among them, Figure 10 is a schematic flow diagram of a page fault exception handling provided by an embodiment of the present application. Figure 10 The process in includes the following steps 1001-1006.

[0187] Step 1001, the function on computing device B accesses the virtual address of the data, triggering a page fault exception.

[0188] Step 1002, determine whether the address segment corresponding to the page in the page fault exception is mapped to a remote computing device.

[0189] Step 1003, if the address segment corresponding to the page is mapped to a remote computing device, allocate a physical page.

[0190] Specifically, in the case where computing device B executes Figure 9 the steps shown in step 905, computing device B will map the virtual address segment to the remote computing device A, thereby determining that the address segment corresponding to the page in the page fault exception is mapped to a remote computing device.

[0191] Step 1004, read the content of the memory page in computing device A into the allocated physical page through the network.

[0192] That is, data at a specific physical address in the memory of computing device A is read via a network onto the allocated physical page.

[0193] Step 1005: Record the page table entry of the page fault address as the allocated physical page, thereby eliminating the page fault exception, such that when a function on computing device B accesses the virtual address of the data, it can trigger access to the data on the allocated physical page.

[0194] Step 1006: If the address segment corresponding to the page is not mapped to the remote computing device, call the original page fault handling mechanism of the kernel.

[0195] In this embodiment, in combination with Figure 9 the embodiment shown in Figure 10 and the embodiment shown in

[0196] to implement data transmission between different computing devices, and it is implemented based on the operating system kernel, which is compatible with existing functions, enabling existing functions to enjoy reduced overhead of data serialization and deserialization without modification, thereby improving data transmission efficiency. Figure 11 Figure 11 Exemplarily, please refer to Figure 11 which is some schematic diagrams of scenarios to which the data transmission method provided in the embodiments of the present application is applied. As Figure 11 shown, the data transmission method provided in the embodiments of the present application can be applied to a variety of scenarios, where different scenarios are used to execute different Serverless workflows. For example, in Figure 11 (a) of Figure 11 , ML training represents a model training scenario in machine learning; in Figure 11 (b) of

[0197] In Figure 11 when applying the data transmission method provided in the embodiments of the present application in the various scenarios shown, the end-to-end latency of different scenarios can be reduced by 14 - 98%, and the performance improvement mainly comes from reducing serialization and deserialization.

[0198] The above embodiments introduce the data transmission method provided in the embodiments of the present application. The following will introduce the device for executing the above data transmission method.

[0199] Please refer to Figure 12 Figure 12 ​​The structural schematic diagram of a first computing device provided by an embodiment of this application. As Figure 12 shown, the first computing device provided by this embodiment includes: a transceiver module 1201, configured to obtain a first virtual address, a network address, and a page table, where the first virtual address is the virtual address corresponding to the first data generated by a second computing device on the second computing device, the network address is the address of the second computing device, and the page table is used to record the mapping relationship between the virtual address and the physical address in the second computing device; a processing module 1202, configured to determine the first physical address where the first data is located on the second computing device based on the first virtual address and the page table; the transceiver module 1201 is further configured to read the first data from the first physical address of the second computing device based on the network address.

[0200] In a possible implementation manner, the first computing device and the second computing device are respectively configured to run different functions on a Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

[0201] In a possible implementation manner, the processing module 1202 is further configured to: in response to a page fault exception caused by a function request in the first computing device to access the first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

[0202] In a possible implementation manner, the transceiver module 1201 is further configured to: receive the first virtual address, the network address, and a virtual address segment sent by a coordination device, where the coordination device is configured to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; based on the network address, obtain a page table corresponding to the virtual address segment from the second computing device, and the page table is used to record the mapping relationship between the virtual address in the virtual address segment and the physical address.

[0203] In a possible implementation manner, the transceiver module 1201 is further configured to: send a page table reading request to the second computing device, where the page table reading request includes the virtual address segment, and the page table reading request is used to request to read the page table corresponding to the virtual address segment; receive the page table sent by the second computing device.

[0204] In a possible implementation manner, the transceiver module 1201 is further configured to receive the first virtual address space sent by the coordination device; the processing module 1202 is further configured to run a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

[0205] In a possible implementation, the first virtual address space does not overlap with the second virtual address space, and the second virtual address space is the virtual address space used by the functions running on the second computing device.

[0206] Please refer to Figure 13 , Figure 13 which is a schematic structural diagram of a second computing device provided by an embodiment of the present application. As Figure 13 shown, the second computing device provided in this embodiment includes: a transceiver module 1301, configured to send a first virtual address and a virtual address segment to a coordination device, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment, and the coordination device is configured to forward the first virtual address and the virtual address segment to the first computing device; in response to the first computing device requesting to read the virtual address segment, the transceiver module 1301 is further configured to send a page table to the first computing device, the page table is used to record the mapping relationship between the virtual addresses in the virtual address segment and the physical addresses, and the page table records the mapping relationship between the first virtual address and the first physical address; in response to the first computing device requesting to access the first physical address, the transceiver module 1301 is further configured to return the first data on the first physical address to the first computing device.

[0207] In a possible implementation, the first computing device and the second computing device are respectively configured to run different functions on a Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

[0208] In a possible implementation, the above-mentioned second computing device further includes: a processing module 1302, configured to set the permission of the target physical address to copy-on-write, where the target physical address is the physical address recorded in the page table and whose original permission is write.

[0209] In a possible implementation, the above-mentioned second computing device further includes a processing module 1302; the transceiver module 1301 is further configured to receive the second virtual address space sent by the coordination device; the processing module 1302 is configured to run a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

[0210] In a possible implementation, the first virtual address space does not overlap with the second virtual address space, and the first virtual address space is the virtual address space used by the functions running on the first computing device.

[0211] Next, an electronic device provided by an embodiment of the present application will be introduced. Please refer to Figure 14 ,Figure 14 FIG. 1400 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 1400 may specifically be embodied as a server, a mobile phone, a tablet computer, a laptop computer, a smart wearable device, etc., which is not limited herein. Specifically, the electronic device 1400 includes a receiver 1401, a transmitter 1402, a processor 1403, and a memory 1404 (where the number of processors 1403 in the electronic device 1400 may be one or more, Figure 14 and one processor is taken as an example herein). Among them, the processor 1403 may include an application processor 14031 and a communication processor 14032. In some embodiments of the present application, the receiver 1401, the transmitter 1402, the processor 1403, and the memory 1404 may be connected through a bus or other means.

[0212] The memory 1404 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1403. A part of the memory 1404 may also include a non-volatile random access memory (NVRAM). The memory 1404 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof. Among them, the operation instructions may include various operation instructions for implementing various operations.

[0213] The processor 1403 controls the operation of the electronic device. In a specific application, the various components of the electronic device are coupled together through a bus system. Among them, the bus system may include a power bus, a control bus, a status signal bus, etc. in addition to a data bus. However, for the sake of clear illustration, all kinds of buses are referred to as a bus system in the figure.

[0214] The method disclosed in the embodiments of the present application can be applied to or implemented by the processor 1403. The processor 1403 can be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by the integrated logic circuit in hardware or instructions in software form in the processor 1403. The above-mentioned processor 1403 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 1403 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or can be executed and completed by the combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 1404, and the processor 1403 reads the information in the memory 1404 and combines its hardware to complete the steps of the above method.

[0215] The receiver 1401 can be used to receive input digital or character information, and generate signal inputs related to the relevant settings and function controls of the electronic device. The transmitter 1402 can be used to output digital or character information through the first interface; the transmitter 1402 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1402 can also include a display device such as a display screen.

[0216] The electronic device provided by the embodiments of the present application may specifically be a chip, and the chip includes a processing unit and a communication unit. The processing unit may be a processor, for example, and the communication unit may be an input / output interface, a pin, a circuit, or the like. The processing unit may execute the computer-executable instructions stored in the storage unit to cause the chip in the execution device to execute the rendering method described in the above embodiments. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0217] May refer to Figure 15 , Figure 15 is a schematic structural view of a computer-readable storage medium provided by the embodiments of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the above Figure 5 disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles.

[0218] Figure 15 Schematically shows a conceptual partial view of an example computer-readable storage medium arranged according to at least some of the embodiments shown here. The example computer-readable storage medium includes a computer program for executing a computer process on a computing device.

[0219] In one embodiment, the computer-readable storage medium 1500 is provided using a signal-bearing medium 1501. The signal-bearing medium 1501 may include one or more program instructions 1502, which when run by one or more processors may provide the functions or partial functions described above for Figure 5 description. In addition, Figure 15 The program instructions 1502 in also describe example instructions.

[0220] In some examples, the signal-bearing medium 1501 may include a computer-readable medium 1503, such as but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital tape, a memory, a ROM, or a RAM, etc.

[0221] In some embodiments, the signal-bearing medium 1501 may include a computer-readable recording medium 1504, such as, but not limited to, a memory, a read / write (R / W) CD, an R / W DVD, and the like. In some embodiments, the signal-bearing medium 1501 may include a communication medium 1505, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Thus, for example, the signal-bearing medium 1501 may be conveyed by a wireless form of the communication medium 1505 (e.g., a wireless communication medium compliant with the IEEE 802 standard or other transmission protocols).

[0222] One or more program instructions 1502 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, a computing device of the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 1502 communicated to the computing device via one or more of the computer-readable medium 1503, the computer-readable recording medium 1504, and / or the communication medium 1505.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0224] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or a wireless manner (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, a data center, etc. that includes one or more integrated available media. The available media may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)).

[0225] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0226] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in various embodiments of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0229] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0230] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data transmission method, characterized in that, Including: A first computing device obtains a first virtual address, a network address, and a page table. The first virtual address is the virtual address corresponding to first data generated by a second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between virtual addresses and physical addresses in the second computing device; Based on the first virtual address and the page table, the first computing device determines a first physical address where the first data is located on the second computing device; Based on the network address, the first computing device reads the first data from the first physical address of the second computing device.

2. The method according to claim 1, characterized in that, The method is applied to a Serverless platform. The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

3. The method according to claim 1 or 2, characterized in that, The determining, by the first computing device, the first physical address where the first data is located on the second computing device based on the first virtual address and the page table includes: In response to a page fault exception caused when a function running in the first computing device requests to access the first virtual address, the first computing device determines the first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining, by the first computing device, the first virtual address, the network address, and the page table includes: The first computing device receives the first virtual address, the network address, and a virtual address segment sent by a coordination device. The coordination device is used to obtain the virtual address of the data generated by the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; Based on the network address, the first computing device obtains the page table corresponding to the virtual address segment from the second computing device. The page table is used to record the mapping relationship between virtual addresses and physical addresses in the virtual address segment.

5. The method according to claim 4, wherein The obtaining, by the first computing device, the page table corresponding to the virtual address segment from the second computing device includes: The first computing device sends a page table reading request to the second computing device. The page table reading request includes the virtual address segment and is used to request to read the page table corresponding to the virtual address segment; The first computing device receives the page table sent by the second computing device.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: The first computing device receives a first virtual address space sent by a coordination device; The first computing device runs a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

7. The method according to claim 6, wherein The first virtual address space does not overlap with a second virtual address space, and the second virtual address space is the virtual address space used by the function running on the second computing device.

8. A data transmission method, characterized in that, Including: The second computing device sends a first virtual address and a virtual address segment to the coordination device. The first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located within the virtual address segment. The coordination device is used to forward the first virtual address and the virtual address segment to the first computing device; In response to the first computing device requesting to read the virtual address segment, the second computing device sends a page table to the first computing device. The page table is used to record the mapping relationship between the virtual addresses in the virtual address segment and the physical addresses, and the page table records the mapping relationship between the first virtual address and the first physical address; In response to the first computing device requesting to access the first physical address, the second computing device returns the first data on the first physical address to the first computing device.

9. The method according to claim 8, characterized in that The method is applied to a Serverless platform. The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

10. The method according to claim 8 or 9, characterized in that The method further includes: The second computing device sets the permission of the target physical address to copy-on-write. The target physical address is the physical address recorded in the page table and whose original permission is write.

11. The method according to any one of claims 8-10, characterized in that, The method further includes: The second computing device receives the second virtual address space sent by the coordination device; The second computing device runs a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

12. The method according to claim 11, wherein The first virtual address space does not overlap with the second virtual address space. The first virtual address space is the virtual address space used by the function running on the first computing device.

13. A first computing device, characterized in that, Includes: A transceiver module, used to obtain a first virtual address, a network address, and a page table. The first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device. The network address is the address of the second computing device. The page table is used to record the mapping relationship between the virtual addresses and the physical addresses in the second computing device; A processing module, used to determine the first physical address where the first data is located on the second computing device based on the first virtual address and the page table; The transceiver module is further used to read the first data from the first physical address of the second computing device based on the network address.

14. The device according to claim 13, characterized in that, The first computing device and the second computing device are respectively used to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

15. The device according to claim 13 or 14, characterized in that, The processing module is further used to: In response to a page fault exception caused by a function running in the first computing device requesting access to the first virtual address, the first computing device determines a first physical address where the first data is located on the second computing device based on the first virtual address and the page table.

16. The device according to any one of claims 13-15, characterized in that, The transceiver module is further configured to: Receive the first virtual address, the network address, and the virtual address segment sent by the coordination device, where the coordination device is configured to obtain the virtual address of the data generated by the second computing device, and the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device; Based on the network address, obtain the page table corresponding to the virtual address segment from the second computing device, where the page table is used to record the mapping relationship between the virtual address and the physical address in the virtual address segment.

17. The device according to claim 16, characterized in that, The transceiver module is further configured to: Send a page table read request to the second computing device, where the page table read request includes the virtual address segment and is used to request to read the page table corresponding to the virtual address segment; Receive the page table sent by the second computing device.

18. The device according to any one of claims 13-17, wherein: The transceiver module is further configured to receive the first virtual address space sent by the coordination device; The processing module is further configured to run a function according to the first virtual address space, and the first virtual address space is used to indicate the space where the virtual address used by the function is located.

19. The device according to claim 18, characterized in that, The first virtual address space does not overlap with the second virtual address space, and the second virtual address space is the virtual address space used by the function running on the second computing device.

20. A second computing device, characterized in that, Comprising: A transceiver module, configured to send a first virtual address and a virtual address segment to a coordination device, where the first virtual address is the virtual address corresponding to the first data generated by the second computing device on the second computing device, the virtual address segment is used to indicate the virtual address range corresponding to the data generated by the second computing device, and the first virtual address is located in the virtual address segment, and the coordination device is configured to forward the first virtual address and the virtual address segment to the first computing device; In response to the first computing device requesting to read the virtual address segment, the transceiver module is further configured to send a page table to the first computing device, where the page table is used to record the mapping relationship between the virtual address and the physical address in the virtual address segment, and the page table records the mapping relationship between the first virtual address and the first physical address; In response to the first computing device requesting to access the first physical address, the transceiver module is further configured to return the first data on the first physical address to the first computing device.

21. The device according to claim 20, characterized in that, The first computing device and the second computing device are respectively configured to run different functions on the Serverless platform, and the input of the function run by the first computing device includes the data generated by the function run by the second computing device.

22. The device according to claim 20 or 21, characterized in that, The device further includes: A processing module, configured to set the permission of a target physical address to copy-on-write, where the target physical address is a physical address recorded in the page table and whose original permission is write.

23. The device according to any one of claims 20-22, characterized in that, The apparatus further includes a processing module; The transceiver module is further configured to receive a second virtual address space sent by a coordination device; The processing module is configured to run a function according to the second virtual address space, and the second virtual address space is used to indicate the space where the virtual address used by the function is located.

24. The device according to claim 23, wherein The first virtual address space does not overlap with the second virtual address space, and the first virtual address space is the virtual address space used by the function running on the first computing device.

25. A computing device, characterized in that, Comprising a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the apparatus executes the method according to any one of claims 1 to 12.

26. A data transmission system, characterized in that, Comprising a first computing device according to any one of claims 13-19, a second computing device according to any one of claims 20-24, and a coordination device, where the coordination device is configured to implement data forwarding between the first computing device and the second computing device.

27. A computer storage medium, characterized in that, The computer storage medium stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

28. A computer program product, characterized in that, The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

Images