A data transmission method and related apparatus

Abstract

Embodiments of the present application disclose a data transmission method for improving data transmission efficiency. The method comprises the following steps: a cloud service platform provides a first configuration interface, the first configuration interface is used for receiving a subscription request sent by a first device, the subscription request comprises a specification of a computing resource, and the computing resource is used for caching data transmitted between the first device and a second device; the cloud service platform receives a data transmission request, the data transmission request indicates transmission of first data between the first device and the second device; the cloud service platform determines that the first data exists in the cache of the computing resource, and reads the first data; and the cloud service platform sends the first data to the first device or the second device according to the data transmission request.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of computer, and in particular, to a data transmission method and related apparatus. BACKGROUND

[0002] With the development of cloud computing technology, devices can transmit data through a network to obtain cloud services required by the devices. For example, in a cloud rendering scenario, a user device transmits data to be rendered to a cloud server through a network, or the cloud server obtains external resources required for rendering data from other cloud servers through the network.

[0003] In the process of using cloud services, the user device needs to transmit a large amount of data to the cloud server through the network for processing. Since a large amount of data needs to consume a large bandwidth when being transmitted through the network, the data transmission efficiency of the user device in the process of using cloud services is low in the case of limited transmission bandwidth. SUMMARY

[0004] Embodiments of the present application provide a data transmission method and related apparatus, for improving data transmission efficiency.

[0005] A first aspect of embodiments of the present application provides a data transmission method. The method can be executed by a cloud service platform, or by a component of the cloud service platform, such as a processor, a chip or a chip system of the cloud service platform, or by a logic module or software capable of realizing all or part of the functions of the cloud service platform. Taking the execution by the cloud service platform as an example, the method provided by the first aspect includes: the cloud service platform provides a first configuration interface, the first configuration interface is used to receive a subscription request sent by a first device, the subscription request includes a specification of a computing resource, the computing resource is used to cache data transmitted between the first device and a second device, and the computing resource includes one or more acceleration units. The cloud service platform receives a data transmission request, the data transmission request indicates transmission of first data between the first device and the second device. The cloud service platform determines that the first data exists in the cache of the computing resource, and then reads the first data. The cloud service platform sends the first data to the first device or the second device according to the data transmission request.

[0006] In the embodiments of the present application, the cloud service platform can provide acceleration services for data transmission for the first device or the second device, so that the first device can accelerate the data transmission process by subscribing to the acceleration services, and the flexibility of the first device in obtaining the acceleration services for data transmission is improved. Further, the cloud service platform can provide a cache space to store the first data transmitted between the first device and the second device, and when the first device or the second device transmits the first data again, the first data can be directly read from the cache of the cloud platform, thereby improving the data transmission speed between the first device and the second device.

[0007] In a possible implementation, the data transmission request indicates that the first device sends first data to the second device, and the data transmission request comprises an address of the second device. In a process in which the cloud service platform sends the first data to the second device according to the data transmission request, the cloud service platform sends the first data in the cache to the second device according to the address of the second device. Specifically, in a cloud rendering scenario, the first data comprises data that the first device needs to perform cloud rendering on the second device, and when the cloud service platform determines that the first data exists in the cache, the cloud service platform sends the first data in the cache to the second device according to the address of the second device.

[0008] In the scenario in which the first device sends the first data to the second device, the cloud service platform sends the first data in the cache to the second device, thereby improving the speed of sending the first data from the first device to the second device and reducing bandwidth consumption in the data transmission process.

[0009] In a possible implementation, the data transmission request indicates that the first device requests the first data from the second device. The data transmission request comprises an address of the second device. In a process in which the cloud service platform sends the first data to the first device or the second device according to the data transmission request, the cloud service platform sends the first data in the cache to the first device according to the address of the first device. Specifically, in a cloud rendering scenario, the first data comprises data that the first device needs to obtain from the second device to perform a cloud rendering task, and when the cloud service platform determines that the first data exists in the cache, the cloud service platform sends the first data in the cache to the first device according to the address of the first device.

[0010] In the scenario in which the first device requests the first data from the second device, when the cloud service platform caches the first data, the cloud service platform sends the first data in the cache to the first device, thereby improving the speed of obtaining the first data from the second device by the first device and reducing bandwidth consumption in the data transmission process.

[0011] In a possible implementation, the data transmission request comprises a hash value of the first data, and in a process in which the cloud service platform determines that the first data exists in the cache of the computing resource, the cloud service platform determines that the hash value is included in a data duplication table. The data duplication table is used to indicate data that has been cached in the cloud service platform, and the data duplication table comprises a correspondence between the first data and the hash value. Specifically, when the data proxy unit of the cloud service platform determines that the hash value of the first data is included in the data duplication table, the data proxy unit of the cloud service platform sends a data transmission request to the acceleration unit of the cloud service platform, and the data transmission request comprises the hash value of the first data.

[0012] When the cloud service platform determines that the hash value of the first data is included in the data duplication table, the data proxy unit of the cloud service platform sends the hash value of the first data to the acceleration unit of the cloud service platform, thereby reducing the transmission bandwidth between the data proxy unit and the acceleration unit and improving the data transmission speed.

[0013] In a possible implementation, after the acceleration unit of the cloud service platform receives the data transmission request sent by the data proxy unit, the cloud service platform reads the first data in the cache according to the hash value of the first data in the data transmission request and sends the first data to the first device or the second device.

[0014] In the cloud service platform according to the embodiment of the present application, the first data in the cache is read according to the hash value of the first data, thereby improving the realizability of the cloud service platform reading the first data in the embodiment of the present application.

[0015] In a possible implementation, before the data proxy unit of the cloud service platform sends the data transmission request to the acceleration unit, the data proxy unit of the cloud service platform performs a segmentation operation on the first data to obtain first data blocks. The data proxy unit calculates the hash values of the first data blocks and performs a deduplication operation on a plurality of data blocks with the same hash value. After the deduplication operation, the data proxy unit queries the data duplication table according to the hash values of the first data blocks. When the hash value of the first data block is not hit in the data duplication table, the data proxy unit of the cloud service platform sends the first data block to the acceleration unit of the cloud service platform.

[0016] In the cloud service platform according to the embodiment of the present application, the deduplication operation is performed on the first data, and the data duplication table is queried based on the first data blocks after the deduplication operation, thereby reducing the data of the transmitted data blocks and further improving the data transmission speed.

[0017] In a possible implementation, when the data duplication table does not include the hash value, the cloud service platform returns a first data transmission response to the first device, receives the first data sent by the first device, and stores the first data in the cache of the acceleration unit of the cloud service platform and updates the data duplication table; or when the data duplication table does not include the hash value, the cloud service platform returns a second data transmission response to the second device, receives the first data sent by the second device, and updates the data duplication table.

[0018] In the cloud service platform according to the embodiment of the present application, when the data duplication table does not include the hash value, the cloud service platform receives the first data, stores the first data in the cache of the acceleration unit, and updates the data duplication table in the data proxy unit, so that when the cloud service platform transmits the first data again, the cloud service platform can provide data transmission acceleration service.

[0019] In a possible implementation, in the process of updating the data duplication table, after the cloud service platform stores the first data in the cache of the acceleration unit, the acceleration unit sends a notification message to the data agent unit of the cloud service platform, and the notification message is used to notify the data agent unit to update the data duplication table, and the data duplication table is used to indicate the data that has been cached in the acceleration unit.

[0020] In the embodiments of the present application, the cloud service platform sends a notification message to the data agent unit after caching the first data, and updates the data duplication table in the data agent unit, thereby improving the realizability of the scheme.

[0021] In a possible implementation, the cloud service platform receives a data acquisition request of a first device, the data acquisition request indicates to acquire second data from a second device, the cloud service platform determines that the second data is cached in the acceleration unit of the cloud server, and sends the second data to the first device.

[0022] In the embodiments of the present application, in the scenario that the first device requests the second data from the second device, when the cloud service platform caches the second data, the second data in the cache is read and sent to the first device, thereby improving the speed of the first device to acquire the second data from the second device, and reducing the bandwidth consumption in the data transmission process.

[0023] The second aspect of the embodiments of the present application provides a cloud service platform, comprising a data acceleration management unit, a data agent unit and an acceleration unit. The data acceleration management unit is configured to provide a first configuration interface, the first configuration interface is configured to receive a subscription request sent by a first device, the subscription request comprises the specification of a computing resource, and the computing resource is configured to cache data transmitted between the first device and a second device. The data agent unit is configured to receive a data transmission request, the data transmission request indicates to transmit first data between the first device and the second device. The acceleration unit is configured to determine that the first data exists in the cache of the computing resource, and read the first data. The acceleration unit is further configured to send the first data to the first device or the second device according to the data transmission request.

[0024] In a possible implementation, the data transmission request indicates to send the first data from the first device to the second device, the data transmission request comprises an address of the second device, and the acceleration unit is specifically configured to send the first data in the cache to the second device according to the address of the second device.

[0025] In a possible implementation, the data transmission request indicates that the first device requests the first data from the second device, and the data transmission request comprises an address of the first device. The acceleration unit is specifically configured to send the first data in the cache to the first device.

[0026] In a possible implementation, the data transmission request includes a hash value of the first data, and the proxy unit is specifically configured to determine that the hash value is included in the data duplication table, and the data duplication table includes a correspondence between the first data and the hash value.

[0027] In a possible implementation, the acceleration unit is specifically configured to receive a hash value of the first data, and send the first data to the first device or the second device according to the hash value of the first data.

[0028] In a possible implementation, the data proxy unit is specifically further configured to return a first data transmission response to the first device when the hash value is not included in the data duplication table, receive first data sent by the first device, and update the data duplication table; or return a second data transmission response to the second device when the hash value is not included in the data duplication table, receive first data sent by the second device, and update the data duplication table.

[0029] The third aspect of the embodiment of the present application further provides a cloud service platform, including a processor, the processor is coupled with a memory, and the processor is used for storing instructions, when the instructions are executed by the processor, the computing device executes the method in the first aspect or any possible implementation of the first aspect or the first aspect.

[0030] The fourth aspect of the embodiment of the present application further provides a computer readable storage medium, instructions are stored on the computer readable storage medium, when the instructions are executed, the computer executes the method in the first aspect or any possible implementation of the first aspect.

[0031] The fifth aspect of the embodiment of the present application provides a computer program product, the computer program product includes instructions, when the instructions are executed, the computer implements the method in the first aspect or any possible implementation of the first aspect.

[0032] It can be understood that the beneficial effects that can be achieved by any of the cloud service platforms, computer readable media or computer program products provided above can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1a A system architecture schematic diagram of a data transmission system provided by the embodiment of the present application;

[0034] Figure 1b A schematic diagram of different implementation forms of a data proxy unit provided by the embodiment of the present application;

[0035] Figure 2 A schematic diagram of a data transmission method provided by the embodiment of the present application;

[0036] Figure 3Another schematic diagram of a data transmission method provided by an embodiment of the present application is shown in FIG. 2.

[0037] Figure 4 Another schematic diagram of a data transmission method provided by an embodiment of the present application is shown in FIG. 2.

[0038] Figure 5 A schematic diagram of a logical architecture of a cloud service platform provided by an embodiment of the present application is shown in FIG. 3.

[0039] Figure 6 A schematic diagram of a structure of a cloud service platform provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0040] The present application provides a data transmission method and related apparatus, which can improve the data transmission efficiency of cloud service.

[0041] The terms "first", "second", "third", "fourth" etc. (if any) in the description, claims, and drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments described herein can be carried out in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a list of steps or units is not necessarily limited to those steps or units that are clearly listed, but can include other steps or units that are not clearly listed or inherent to such processes, methods, products, or apparatus.

[0042] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0043] The data transmission method and related apparatus provided by the embodiments of the present application are described below in conjunction with the drawings.

[0044] Please refer to Figure 1a , Figure 1a A schematic diagram of a system architecture of a data transmission system provided by an embodiment of the present application is shown in FIG. 5. Figure 1aAs shown, the data transmission system 100 includes a data acceleration management unit 101, an acceleration unit 102, a data proxy unit 103, a first device 104, and a second device 105. The acceleration unit 102 includes a connection acceleration module 1021 and an external access acceleration module 1022. In the data transmission system 100, the data acceleration management unit 101, the acceleration unit 102, and the data proxy unit 103 constitute a cloud service platform, which provides the first device 104 with data transmission acceleration services. The data acceleration management unit 101, the acceleration unit 102, and the data proxy unit 103 in the cloud service platform can be integrated on one server of the cloud service platform, or distributed on multiple servers, which is not limited in particular.

[0045] It should be noted that in some scenarios, the cloud service platform further includes the first device 104 and the second device 105. For example, when the first device 104 is a user device and the second device 105 is a cloud server, the cloud server provides cloud services for the user device, and in this case, the cloud service platform includes the second device 105. For another example, when the first device 104 is a cloud server and the second device 105 is an external resource server, the cloud server requests the external resource server to provide external data required for providing cloud services, and in this case, the cloud service platform includes the first device 104.

[0046] In Figure 1a In the system architecture as shown, the first device 104 can subscribe to data transmission acceleration services from the data acceleration management unit 101, and after the first device 104 subscribes to the data transmission acceleration services, the first device 104 sends data to the second device 105 through the data proxy unit 103 and the acceleration unit 102, or the first device 104 can request the acceleration unit 102 to obtain data resources in the second device 105 through the data proxy unit 103.

[0047] The functions of the various parts of the data transmission system as shown will be described in detail below. Figure 1a

[0048] The data acceleration management unit 101 is configured to implement functions such as subscription of data transmission acceleration services, creation and deployment of acceleration units, etc. Specifically, the data acceleration management unit 101 can provide the first device 104 with a subscription service for data transmission acceleration through a UI interface or an application program interface (API), and the data acceleration management unit 101 can also create the acceleration unit 102 according to a subscription request and configure the acceleration unit 102, with the configuration information including an availability zone (AZ), a virtual private cloud (VPC), and a subnet, etc.

[0049] ​The acceleration unit 102 is configured to implement the function of data transmission acceleration. Specifically, the acceleration unit 102 is capable of creating a cache to accelerate the transmission of data, and the cache is created in a manner of hash table or tree. The acceleration unit 102 can be a physical machine, a virtual machine or a container.

[0050] The acceleration unit 102 includes a connection acceleration module 1021 and an external access acceleration module 1022. Both the connection acceleration module 1021 and the external access acceleration module 1022 have a data analysis module and a cache module. The data analysis module is configured to analyze whether the transmitted data meets the cache condition, and the cache module is configured to provide a cache space to accelerate the data transmission.

[0051] The connection acceleration module 1021 can provide data transmission acceleration service for the process of transmitting data from the user equipment to the cloud server. The external access acceleration module 1022 can provide data transmission acceleration service for the process of obtaining external data resources by the cloud service. The connection acceleration module 1021 and the external access acceleration module 1022 can also be two functional modules of the same software instance, that is, the acceleration unit 102 has two functions of connection acceleration and external access acceleration.

[0052] The data proxy unit 103 is configured to provide a data transmission proxy function to the first device 104, and cooperate with the acceleration unit 102 to reduce the transmission of duplicate data. The data transmission proxy function provided by the data proxy unit 103 includes data segmentation, data duplication query and data transmission function.

[0053] It should be noted that the data proxy unit 103 in the embodiments of the present application can be integrated in the first device 104 in the form of a software development kit (SDK), or can be an independently running proxy server, and the specific implementation is not limited.

[0054] Please refer to Figure 1b , Figure 1b The two implementation modes of the data proxy unit provided in the embodiments of the present application are shown in the schematic diagram. As Figure 1b shown, the data proxy unit 103 can be integrated in the first device 104 in the form of SDK and run with the application program in the first device 104. The data proxy unit 103 can also be independently run in the form of proxy server. When the data proxy unit 103 is independently run in the form of proxy, the data proxy unit 103 can be connected with multiple first devices and provide data transmission acceleration service for the application programs of the multiple first devices. When the data proxy unit 103 provides data transmission proxy service in the form of proxy in the embodiments of the present application, the proxy form includes http proxy, ftp proxy and socket proxy.

[0055] In this application embodiment, the first device 104 can be a user device or a cloud server, and the second device 105 can be a cloud server or an external resource server. In one example of this application, the first device 102 is a user device, and the second device 105 is a cloud server. The user device sends data to the cloud server through the proxy unit 103 and the acceleration unit 102, and the cloud server performs processing tasks on the data sent by the user device, such as cloud data rendering tasks.

[0056] In another example of this application, the first device 102 is a cloud server and the second device 105 is an external resource server. When the cloud server needs to obtain data from the external resource server, the cloud server obtains the data from the external resource server through the acceleration unit 102. For example, the cloud server needs to obtain the data resources required by the cloud rendering task from the external resource server during the execution of the cloud rendering task.

[0057] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data transmission method provided in an embodiment of this application. Figure 2 The data transmission method shown includes, but is not limited to, the following steps:

[0058] 201. The first device sends an order request to the cloud service platform, which includes the specifications of computing resources.

[0059] The first device sends a subscription request to the cloud service platform, which includes the specifications of computing resources. The computing resources are used to cache data transmitted between the first and second devices, and include one or more acceleration units. The specifications of the computing resources in the subscription request include one or more of the following: the specifications and configuration information of the acceleration units. Specifically, the specifications of the acceleration units may be one or more of the following, such as cache size or number of processor cores; the configuration information may be one or more of the following, such as VPC information or subnet information to which the acceleration units need to be associated.

[0060] Specifically, the first device sends an order request to the acceleration service management unit of the cloud service platform. The acceleration service management unit creates an acceleration unit based on the order request. The acceleration unit is used to cache the data transmitted between the first device and the second device. The transmitted data includes the data sent by the first device to the second device through the acceleration unit, as well as the data obtained by the first device from the second device through the acceleration unit.

[0061] In this embodiment of the application, the first device can send a subscription request to the acceleration service management unit in a variety of ways. For example, the first device can send a subscription request to the acceleration service management unit through a web page, an application, or an application programming interface (API).

[0062] After the acceleration service management unit receives the subscription request sent by the first device, the acceleration unit is created according to the subscription request. For example, the acceleration management service unit creates the acceleration unit in the manner of purchasing a virtual machine, a physical machine or a container according to the subscription information, and associates the acceleration unit to the corresponding VPC or subnet according to the configuration information in the subscription information.

[0063] 202. receiving a data transmission request, the data transmission request indicating transmission of first data between a first device and a second device.

[0064] The cloud service platform receives the data transmission request, the data transmission request indicating transmission of first data between a first device and a second device. Specifically, the acceleration unit in the cloud service platform receives the data transmission request sent by the first device through the data proxy unit.

[0065] In the scenario where the first device sends the first data to the second device, the data transmission request includes an address of the second device, and the data transmission request indicates that the cloud service platform sends the first data to the second device according to the address of the second device. In the scenario where the first device requests the first data from the second device, the data transmission request includes an address of the second device, and the data transmission request indicates that the cloud service platform obtains the first data from the second device according to the address of the second device.

[0066] The data transmission request in the embodiments of the present application further includes the first data or a hash value of the first data. For example, in the scenario where the first device sends the first data to the second device, before the acceleration unit receives the data transmission request, the first device calculates a hash value of the first data through the data proxy unit, and queries the hash value of the first data in the data duplication table of the data proxy unit, the data duplication table being used to indicate the first data that has been cached in the acceleration unit, and the data duplication table including the correspondence between the first data and the hash value.

[0067] When the hash value of the first data is hit in the data duplication table, it indicates that the first data exists in the cache of the acceleration unit, at this time, the data transmission request includes the hash value of the first data. When the hash value of the first data is not hit in the data duplication table, it indicates that the first data does not exist in the cache of the acceleration unit, at this time, the data transmission request includes the first data.

[0068] The first data in the embodiments of the present application includes a plurality of first data blocks after the data proxy unit performs segmentation operation, and the hash value of the first data includes hash values of the plurality of first data blocks. Specifically, in the scenario that the first device sends data to the second device, before the data proxy unit sends the first data to the acceleration unit, the first device performs data segmentation on the first data to be sent through the data proxy unit to obtain the first data block, the data proxy unit calculates the hash value of the first data block, and the data proxy unit queries the data redundancy table based on the hash value of the first data block.

[0069] If the hash value of the first data block is hit in the data redundancy table, the data transmission request includes the hash value of the first data block. If the hash value of the first data block is not hit in the data redundancy table, the data transmission request includes the first data block.

[0070] 203. Determine that the first data exists in the cache of the computing resource, and read the first data.

[0071] When the first data exists in the cache of the acceleration unit, the acceleration unit reads the first data from the cache. When the first data does not exist in the cache of the acceleration unit, the acceleration unit determines whether the first data meets the cache condition, and if the first data meets the cache condition, the first data is stored to the cache. The first data includes data to be transmitted between the first device and the second device.

[0072] That is, when the hash value of the first data is included in the data transmission request received by the acceleration unit, the cloud service platform determines that the first data exists in the cache of the acceleration unit, and reads the first data. When the first data is included in the data transmission request received by the acceleration unit, the cloud service platform determines that the first data does not exist in the cache of the acceleration unit, and then determines whether the first data meets the cache condition, and if the first data meets the cache condition, the first data is stored to the cache.

[0073] It should be noted that the first data in the embodiments of the present application includes one or more first data blocks after segmentation, when the acceleration unit receives the hash value of the first data block, the acceleration unit reads the first data block from the cache according to the hash value of the first data block. After the acceleration unit receives the first data block, it is determined whether the first data block meets the cache condition, and if the first data block meets the cache condition of the acceleration unit, the acceleration unit caches the first data block to the cache of the acceleration unit. If the first data block does not meet the cache condition of the acceleration unit, the acceleration unit sends the first data block to the second device. The cache condition is, for example, the data length of the first data block is greater than or equal to a preset threshold.

[0074] It can be understood that, in the embodiment of the application, after the acceleration unit caches the first data, the acceleration unit sends a notification message to the data proxy unit, and the notification message is used to notify the data proxy unit to update the data repetition table, and the data repetition table is used to indicate the data that has been cached in the acceleration unit.

[0075] 204. According to the data transmission request, the first data is sent to the first device or the second device.

[0076] According to the data transmission request, the first data is sent to the first device or the second device. In the scenario that the first device sends data to the second device, when the first data exists in the cache of the acceleration unit, the acceleration unit reads the first data from the cache, and then sends the first data to the second device according to the data transmission request. When the first data does not exist in the cache of the acceleration unit, the acceleration unit forwards the data transmission request to the second device.

[0077] In the scenario that the first device requests the second data from the second device, after the first device obtains the second data from the second device through the acceleration unit, the acceleration unit stores the transmitted second data in the cache of the acceleration unit. When the acceleration unit transmits data, if the data transmission request indicates that the first device requests the second data from the second device, the acceleration unit can directly read the second data in the cache, and send the second data cached in the acceleration unit to the first device.

[0078] In the embodiment of the application, the data acceleration management unit can create the acceleration unit based on the subscription request of the first device, and the acceleration unit can cache the data transmitted by the first device and the second device, so that when the first device and the second device perform data transmission again, the data transmission can be directly read from the cache of the acceleration unit, thereby providing acceleration service for the data transmission between the first device and the second device, and improving the data transmission efficiency between the first device and the second device.

[0079] The following describes a data transmission method provided by the embodiment of the application, taking the first device as an example of a user device and the second device as an example of a cloud server.

[0080] Please refer to Figure 3 , Figure 3 The flowchart of another data transmission method provided by the embodiment of the application is shown. The data transmission method includes but is not limited to one or more of the following steps:

[0081] 301. The user device sends first data to the data proxy unit.

[0082] The user device sends first data to the data proxy unit, and the first data includes data to be sent by the user device to the cloud server.

[0083] In one example of step 301, the user device needs to perform a cloud rendering task through the cloud server, and an application in the user device sends data that needs to be rendered to the data proxy unit.

[0084] 302. The data proxy unit segments and deduplicates the first data.

[0085] The data proxy unit segments and deduplicates the first data. Specifically, the data proxy unit segments the first data to obtain a plurality of first data blocks, and calculates hash values of the segmented first data blocks to obtain a plurality of hash values of the first data blocks. In the process of segmenting the first data, the data unit can segment the data based on a separator set in the deduplication session, and the data segmentation manner includes a fixed block manner and a variable length block manner.

[0086] It can be understood that after the deduplication analysis, the data proxy unit can perform a deduplication operation on the first data based on the hash values of the plurality of first data blocks. The proxy unit performs data deduplication on the first data blocks with the same hash value to obtain the first data after deduplication.

[0087] After the data proxy unit generates the hash value of the first data, it queries the hash value of the first data block in the data redundancy table, which is used to indicate the data that has been cached in the connection acceleration module. The data proxy unit can determine whether the first data block already exists in the cache of the connection acceleration module by querying the data redundancy table.

[0088] In the embodiment of the application, the data proxy unit can perform communication transmission on the first data based on a programmable deduplication session, wherein different programming programs can specify different deduplication data block manners. The following is an example of the deduplication session in the embodiment of the application:

[0089] Session s = Session.createOrOpen(...) / / Create or open a deduplication session.

[0090] Transport t = session.createTransport(...) / / Create a transmission channel based on the session. One session can support one or more transmission channels.

[0091] t.addDedupPrefix("}") / / Set the deduplication separator.

[0092] t.setDedupBlockSize("4M") / / Set the deduplication data block size, which is used for fixed block deduplication algorithm or maximum block limit of variable length block.

[0093] Stream stream = t.createOutputStream(...) / / Create a transport output stream based on the transport usually.

[0094] Stream.write(data...) / / Write data.

[0095] Stream.write(data,true) / / Write data and require deduplication processing for this write data.

[0096] In the embodiments of the present application, the data proxy unit can deduplicate data blocks in one deduplication session, or deduplicate data blocks in multiple deduplication sessions, which is not limited.

[0097] 303.If the data duplication table does not hit the first data block, send the first data block to the connection acceleration module.

[0098] If the hash value of the first data block is not hit in the data duplication table in the data proxy unit, the data proxy unit sends the first data block to the connection acceleration module.

[0099] 304.If the first data block meets the cache condition, cache the first data block.

[0100] After the connection acceleration module receives the first data block in the first data sent by the data proxy unit, it analyzes the first data block. Specifically, if the first data block meets the cache condition, the connection acceleration module stores the first data block in the cache. For example, when the data length of the first data block is greater than or equal to a preset threshold, the connection acceleration module stores the first data block in the cache and calculates the hash value of the first data block.

[0101] 305.The connection acceleration module forwards the first data block to the cloud server.

[0102] The connection acceleration module forwards the first data block to the cloud server. Specifically, the first data forwarded by the connection acceleration module to the cloud server includes the cached first data block and the first data block that does not meet the cache condition.

[0103] In the embodiments of the present application, the cloud server can receive the first data block through the software development kit (SDK) or through the TCP protocol, which is not limited.

[0104] 306.The connection acceleration module sends a notification message to the data proxy unit.

[0105] After the connection acceleration module stores the first data chunk to the cache of the connection acceleration module, the connection acceleration module sends a notification message to the data agent unit, the notification message is used to notify the data agent unit to update the data repetition table, the data repetition table is used to indicate the data that has been cached in the acceleration unit.

[0106] After the data agent unit receives the notification message sent by the connection acceleration module, the data agent unit updates the internal data repetition table according to the notification message. Specifically, the data agent unit identifies that the first data has been cached in the connection acceleration module in the data repetition table. For example, the data with the data segment hash value A has been cached in the connection acceleration module.

[0107] The execution sequence of steps 305 and 306 in the embodiments of the present application is not limited.

[0108] 307. The user equipment sends the first data again.

[0109] 308. The data agent unit performs segmentation and deduplication operations on the first data.

[0110] The steps 307 to 308 in the embodiments of the present application are similar to the methods performed in the steps 301 and 302 described above, and details are not repeated.

[0111] 309. If the data repetition table hits the first data chunk, the hash value of the first data is sent to the connection acceleration module.

[0112] If the hash value of the first data chunk is hit in the data repetition table in the data agent unit, it means that the first data chunk is cached in the connection acceleration unit. Therefore, the data agent unit sends the hash value of the first data chunk to the connection acceleration module.

[0113] 310. The connection acceleration module reads the first data chunk according to the hash value of the first data chunk.

[0114] After the connection acceleration module receives the hash value of the first data chunk, the connection acceleration module reads the first data chunk according to the hash value of the first data chunk.

[0115] 311. The connection acceleration module sends the first data chunk to the server.

[0116] After the connection acceleration module reads the first data chunk according to the hash value of the first data chunk in the first data, the connection acceleration module sends the first data chunk to the cloud server.

[0117] 312. The connection acceleration module performs cache data cleaning.

[0118] The connection acceleration module cleans up the data in the cache. Specifically, the connection acceleration module can clean up the cache data one by one, can clean up the cache data in batches, and the specific manner is not limited.

[0119] In the embodiment of the application, the connection acceleration module can trigger the cleaning of the cache data periodically, or can trigger the connection acceleration module to clean the cache data when the cache data reaches the preset capacity, and the specific manner is not limited.

[0120] 313. The connection acceleration module sends a cleaning notification message to the data proxy unit.

[0121] After the connection acceleration module cleans up the cache data, it sends a cleaning notification message to the data proxy unit, which is used to notify the data proxy unit to clean up the hash values of the corresponding cache data that are removed in the data duplication table.

[0122] 314. Clean the data duplication table according to the cleaning notification message.

[0123] The data proxy unit cleans the data duplication table according to the cleaning notification message. Specifically, the data proxy unit deletes the hash values corresponding to the data blocks that have been cleaned by the acceleration unit in the data duplication table.

[0124] The execution order of steps 312 to 314 in the embodiment of the application is not limited to the execution order of steps 301 to 310 described above.

[0125] In the scenario where the user equipment sends the first data to the cloud server, the connection acceleration module reads the first data in the cache and sends it to the cloud server, thereby improving the speed of sending the first data from the user equipment to the cloud server and reducing the bandwidth consumption in the data transmission process.

[0126] The following first device takes the cloud server as an example, and the second device takes the external resource server as an example to introduce the data transmission method provided by the embodiment of the application.

[0127] Please refer to Figure 4 , Figure 4 The flowchart of another data transmission method provided by the embodiment of the application. The data transmission method includes but is not limited to one or more of the following steps:

[0128] 401. The cloud server sends a first access request to the external access acceleration module through the data proxy unit, which is used to request the second data from the external resource server.

[0129] The cloud server sends a first access request to the external access acceleration module through the data proxy unit, and the first access request is used to request second data from an external resource server. The first access request includes source address information of the second data, for example, a same resource locator URL corresponding to the second data.

[0130] For example, in one example of step 401, when the cloud server needs to connect to an external resource server to obtain data related to a cloud rendering task during execution of the cloud rendering task, the cloud server sends a first access request to the external access acceleration module through the data proxy unit, and the source address information in the first access request is “https: / / ...”.

[0131] The cloud server in the embodiments of the present application can access the second data of the external resource server through a proxy mode or access the second data of the external resource server through an SDK mode, and the specific mode is not limited.

[0132] 402. The external access acceleration module queries whether the second data exists in the cache.

[0133] The external access acceleration module queries whether the second data exists in the cache. Specifically, after receiving the first access request, the external access acceleration module analyzes the source address information in the first access request, and queries whether the second data exists in the cache in the external access acceleration module according to the source address information.

[0134] 403. If not, the external access acceleration module forwards the first access request to the external resource server.

[0135] If the second data does not exist in the external access acceleration module, the external access acceleration module forwards the first access request to the external resource server. Specifically, the external access acceleration module forwards the first access request to the external resource server according to the source address information in the first access request.

[0136] It should be noted that when the second data exists in the cache of the external access acceleration module, the external access acceleration module also needs to determine whether to read the second data based on a preconfigured cache strategy. For example, when the cache data in the external access acceleration module expires, the external access acceleration module still forwards the first access request to the external resource server.

[0137] 404. The external resource server sends the second data to the external access acceleration module.

[0138] After receiving the first access request, the external resource server sends the second data to the external access acceleration module.

[0139] The external resource server in the embodiments of the present application can send the second data to the external access acceleration module through the hypertext transfer protocol (HTTP) or through other protocols, and the specific protocol is not limited.

[0140] 405. The external access acceleration module caches the second data.

[0141] After the external access acceleration module receives the second data sent by the external resource server, the external access acceleration module caches the second data. The external access acceleration module establishes a corresponding relationship between the second data and the source address information, so that the cloud server re-requests the second data according to the source address information to query the second data from the cache of the external access acceleration module.

[0142] 406. The external access acceleration module sends the second data to the cloud server.

[0143] After the external access acceleration module receives the second data sent by the external resource server, the external access acceleration module sends the second data to the cloud server.

[0144] The execution sequence of steps 405 to 406 in the embodiments of the present application is not limited.

[0145] 407. The cloud server sends a second access request to the external access acceleration module through the data proxy unit, and the second access request is used to re-request the second data with the same source.

[0146] The cloud server sends a second access request to the external access acceleration module through the data proxy unit, and the second access request is used to re-request the second data, that is, the source address information in the second access request is the same as that in the first access request.

[0147] 408. According to the second access request, it is queried whether the second data is cached in the external access acceleration module.

[0148] After the external access acceleration module receives the second access request, the source address information in the second access request is extracted, and it is queried whether the second data exists in the external access acceleration module according to the source address information in the first access request.

[0149] For example, in one example of step 408, the external access acceleration module queries whether the second data exists in the cache of the external access acceleration module according to the uniform resource locator (URL) in the second access request.

[0150] 409. The external access acceleration module triggers the detection of data resource changes.

[0151] The external access acceleration module can trigger a detection of a data resource change. Specifically, the external access acceleration module sends a detection message to the external resource server, and the detection message is used to update the data corresponding to the source address information in the cache.

[0152] For example, when the data corresponding to the source address information in the external access acceleration module changes, the external access acceleration module can trigger a detection of a data resource change, and obtain the changed data corresponding to the source address information from the external resource server. For another example, when the data corresponding to the source address information in the external access acceleration module expires, the external access acceleration module can also trigger a detection of a data resource change, and obtain the updated data corresponding to the source address information from the external resource server.

[0153] In the embodiment of the application, the external access acceleration module can trigger a detection of a resource change periodically, and can trigger a detection of a resource change based on a request message of the cloud server, and the specific implementation is not limited.

[0154] 410. If the second data exists, the external access acceleration module sends the cached second data to the cloud server.

[0155] If the second data exists in the external access acceleration module, the external access acceleration module reads the second data in the cache based on the source address information in the second access request, and the external access acceleration module sends the cached second data to the cloud server.

[0156] In the scenario where the cloud server requests the second data from the external resource server in the embodiment of the application, when the second data is cached in the external access acceleration module, the external access acceleration module reads the second data in the cache and sends it to the cloud server, thereby improving the speed of the cloud server obtaining the second data from the external access acceleration module, and reducing the bandwidth consumption in the data transmission process.

[0157] The data transmission method provided in the embodiment of the application is introduced above, and the service platform related to the embodiment of the application is introduced below in combination with the drawings.

[0158] Please refer to Figure 5 , Figure 5 A structural schematic diagram of a cloud service platform provided in the embodiment of the application. The device is used to implement each step performed by the cloud service platform in each embodiment, as shown in Figure 5 The cloud service platform 500 includes a data acceleration management unit 501, a data proxy unit 502, and an acceleration unit 503.

[0159] The data acceleration management unit 501 is configured to provide a first configuration interface, and the first configuration interface is configured to receive a subscription request sent by the first device, and the subscription request comprises a specification of a computing resource, and the computing resource is configured to cache data transmitted between the first device and the second device. The data proxy unit 502 is configured to receive a data transmission request, and the data transmission request indicates transmission of first data between the first device and the second device. The acceleration unit 503 is configured to determine that the first data exists in the cache of the computing resource, and read the first data. The acceleration unit 503 is further configured to send the first data to the first device or the second device according to the data transmission request.

[0160] In a possible implementation, the data transmission request indicates that the first data is sent from the first device to the second device, and the data transmission request comprises an address of the second device. The acceleration unit 503 is specifically configured to send the first data in the cache to the second device according to the address of the second device.

[0161] In a possible implementation, the data transmission request indicates that the first device requests the first data from the second device, and the data transmission request comprises an address of the second device. The acceleration unit 503 is specifically configured to send the first data in the cache to the first device.

[0162] In a possible implementation, the data transmission request comprises a hash value of the first data, and the data proxy unit 502 is specifically configured to determine that the hash value is included in a data duplication table, and the data duplication table comprises a correspondence between the first data and the hash value.

[0163] In a possible implementation, the acceleration unit 503 is specifically configured to receive the hash value of the first data, and send the first data to the first device or the second device according to the hash value of the first data.

[0164] In a possible implementation, the data proxy unit 502 is specifically further configured to return a first data transmission response to the first device when the hash value is not included in the data duplication table, receive the first data sent by the first device, and update the data duplication table; or return a second data transmission response to the second device when the hash value is not included in the data duplication table, receive the first data sent by the second device, and update the data duplication table.

[0165] It should be understood that the division of units in the above apparatus is only a logical functional division, and all or part of them can be integrated into a physical entity or physically separated when actually implemented. The units in the apparatus can all be implemented in the form of software invoked by a processing element; all in the form of hardware; or some units in the form of software invoked by a processing element and some units in the form of hardware. For example, each unit can be a separately established processing element, or can be integrated in a chip of the apparatus, in addition, it can also be stored in the form of a program in a memory, and the function of the unit is invoked and executed by a processing element of the apparatus. In addition, all or part of these units can be integrated together or independently implemented. The processing element described herein can be a processor, which can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each unit can be implemented by integrated logic circuits of hardware in the processing element or in the form of software invoked by the processing element.

[0166] It should be noted that, for the above method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the described actions, and those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0167] Other reasonable combinations of steps that those skilled in the art can think of based on the above description are also within the protection scope of the present application. In addition, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0168] Please refer to Figure 6 , Figure 6 A cloud service platform schematic diagram is provided for the embodiments of the present application. As shown in Figure 6 , the cloud service platform 600 includes a processor 610, a memory 620 and an interface 630, the processor 610, the memory 620 and the interface 630 are coupled through a bus (not labeled in the figure). The memory 620 stores instructions, when the execution instructions in the memory 620 are executed, the cloud service platform 600 executes the method executed by the cloud service platform in the above method embodiments.

[0169] The cloud service platform 600 can be one or more integrated circuits configured to implement the above methods, for example, one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For another example, when units in the apparatus can be implemented in the form of a processing element scheduler, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke a program. For another example, these units can be integrated together to be implemented in the form of a system-on-a-chip (SOC).

[0170] The processor 610 can be a central processing unit (CPU), and can also be other general purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, transistor logic device, a hardware component or any combination thereof. The general purpose processor can be a microprocessor or any conventional processor.

[0171] The memory 620 can include a read-only memory and a random access memory, and provide instructions and data to the processor 610. The memory 620 can also include a non-volatile random access memory. For example, the memory 620 can be provided with multiple partitions, each of which is used to store a private key of a different software module.

[0172] The memory 620 can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0173] The bus can include, in addition to the data bus, a power bus, a control bus, and a status signal bus, etc. The bus can be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0174] In another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores computer execution instructions. When the processor of the device executes the computer execution instructions, the device executes the method performed by the cloud service platform in the above method embodiment.

[0175] In another embodiment of the present application, a computer program product is also provided, which includes computer-executable instructions stored in a computer-readable storage medium. When the processor of the device executes the computer-executable instructions, the device performs the method performed by the cloud service platform in the above-described method embodiments.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0177] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the above-described device embodiments are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0178] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0179] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0180] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A data transmission method, characterized by, The method is applied to a cloud service platform, and comprises: providing a first configuration interface, the first configuration interface being configured to receive a subscription request sent by a first device, the subscription request comprising a specification of a computing resource, the computing resource being configured to cache data transmitted between the first device and a second device; receiving a data transmission request, the data transmission request indicating transmission of first data between the first device and the second device; determining that the first data exists in a cache of the computing resource, and reading the first data; sending the first data to the first device or the second device according to the data transmission request.

2. The method of claim 1, wherein, The data transmission request indicates that the first data is sent from the first device to the second device, and the data transmission request comprises an address of the second device. The sending of the first data to the first device or the second device according to the data transmission request comprises: sending the first data in the cache to the second device according to the address of the second device.

3. The method according to claim 1 or 2, characterized in that, The data transmission request indicates that the first data is requested by the first device from the second device. The sending of the first data to the first device or the second device according to the data transmission request comprises: sending the first data in the cache to the first device.

4. The method of claim 3, wherein, The data transmission request comprises the address of the second device.

5. The method according to any one of claims 2 to 4, characterized in that, The data transmission request comprises a hash value of the first data. The determination that the first data exists in the cache of the computing resource comprises: determining that the hash value is included in a data duplication table, the data duplication table comprising a correspondence between the first data and the hash value.

6. The method of claim 5, wherein, The sending of the first data to the first device or the second device according to the data transmission request comprises: receiving the hash value of the first data; sending the first data to the first device or the second device according to the hash value of the first data.

7. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: when the data duplication table does not include the hash value, returning a first data transmission response to the first device; receiving the first data sent by the first device and updating the data duplication table; or when the data duplication table does not include the hash value, returning a second data transmission response to the second device; receiving the first data sent by the second device and updating the data duplication table.

8. A cloud service platform, characterized by comprise: a data acceleration management unit configured to provide a first configuration interface, the first configuration interface being configured to receive a subscription request sent by a first device, the subscription request comprising a specification of a computing resource, the computing resource being configured to cache data transmitted between the first device and a second device; a data proxy unit configured to receive a data transmission request, the data transmission request indicating transmission of first data between the first device and the second device; an acceleration unit configured to determine that the first data exists in a cache of the computing resource, and read the first data; the acceleration unit is further configured to send the first data to the first device or the second device according to the data transmission request.

9. The cloud services platform of claim 8, wherein, The data transmission request indicates that the first data is sent from the first device to the second device, the data transmission request comprises an address of the second device, and the acceleration unit is specifically configured to: According to the address of the second device, the first data in the cache is sent to the second device.

10. The cloud services platform of claim 8 or 9, wherein, The data transmission request indicates that the first device requests the first data from the second device, and the acceleration unit is specifically configured to: The first data in the cache is sent to the first device.

11. The cloud services platform of claim 10, wherein, The data transmission request comprises an address of the second device.

12. The cloud services platform according to any of claims 9 to 11, characterized in that, The data transmission request comprises a hash value of the first data, and the proxy unit is specifically configured to: Determine that the hash value is included in a data duplication table, and the data duplication table comprises a correspondence between the first data and the hash value.

13. The cloud services platform of claim 12, wherein, The acceleration unit is specifically configured to: Receive the hash value of the first data; According to the hash value of the first data, the first data is sent to the first device or the second device.

14. The cloud services platform according to any of claims 8 to 11, characterized in that, The data proxy unit is specifically configured to: When the data duplication table does not include the hash value, return a first data transmission response to the first device; Receive the first data sent by the first device and update the data duplication table; Or, When the data duplication table does not include the hash value, return a second data transmission response to the second device; Receive the first data sent by the second device and update the data duplication table.

15. A cloud services platform, characterized by The processor is coupled with a memory, and the processor is configured to store instructions, and when the instructions are executed by the processor, the computing device executes the method in any one of claims 1 to 7.

16. A computer-readable storage medium having stored thereon instructions, The instructions are executed to enable the computer to execute the method in any one of claims 1 to 7.

17. A computer program product comprising instructions therein, characterised in that, The instructions are executed to enable the computer to execute the method in any one of claims 1 to 7.

Images