File transmission method and related equipment

Abstract

The embodiment of the invention discloses a file transmission method and related equipment, which are used for reducing the frequency of fragmenting and recombining a large file in cross-network transmission and improving the transmission efficiency. The method provided by the embodiment of the invention is applied to a first transmission server in a first network. The method comprises the following steps: acquiring a plurality of fragments sent by a first application in a first network, wherein the plurality of fragments are contained in a target file; and sending the plurality of fragments to a second transmission server, so that the second transmission server sends the plurality of fragments to a second storage server included in the second network, and the second storage server is used for recombining the plurality of fragments to obtain the target file. Wherein fragmented transmission between the first transmission server and the second transmission server is cross-network transmission, and the second transmission server and the second storage server are included in a second network.

Description

[0001] This application claims priority to the Chinese patent application filed on August 8, 2023, with application number 202310993151.0, and invention name “A method, system and device for file transmission”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of communications, and in particular to a file transmission method and related equipment. Background Art

[0003] Cloud platforms provide storage services that support the upload and download capabilities of local large files. With the development of information technology, storage services on different cloud platforms, or storage services in different regions on the same cloud platform, have the need to transfer large files across networks. How to achieve cross-network transmission of large files in cloud platform scenarios has become an urgent problem to be solved.

[0004] In the traditional file transfer method, the application transmits the large file in pieces to the local storage service, which reassembles the pieces and stores the large file. When transmitting large files across the network, the file transfer service sender reads the large file from the local storage service, splits it into pieces, and then sends the pieces to the file transfer service receiver. The file transfer service receiver reassembles the pieces to obtain the large file, then splits the large file into pieces and transmits the pieces to the remote storage service, so that the remote storage service receives the pieces and reassembles them to obtain the large file.

[0005] In this method, all nodes involved in file transfer (including the file transfer service sender, the file transfer service receiver, and the remote storage service) will be fragmented and / or reorganized, which makes the operation complicated and leads to low transmission efficiency. Summary of the invention

[0006] The embodiments of the present application provide a file transmission method and related equipment for reducing the number of times large files are fragmented and reassembled during cross-network transmission, reducing the complexity of operations, and thus improving transmission efficiency.

[0007] A first aspect of the present application provides a file transmission method, which is applied to a first transmission server, and the method includes:

[0008] The first transmission service end is included in the first network, and the first network also includes the first application. The first application can call the interface of the first transmission service end to transmit a large file to the second network. At this time, the first application will fragment the large file to obtain multiple fragments. Among them, the large file refers to a file whose data volume exceeds a threshold. The size of the threshold can be set according to the needs of the actual application, and is not specifically limited here. Exemplarily, take the target file as an example of a large file. The first transmission service end can obtain multiple fragments from the application, and these multiple fragments are included in the target file. Among them, the data volume size of each fragment in the multiple fragments can be the same or different, and is not specifically limited here. The first transmission service end transmits these multiple fragments across the network to the second transmission service end in the second network, that is, the fragment transmission between the first transmission service end and the second transmission service end is a cross-network transmission. The first transmission service end sends these multiple fragments to the second transmission service end, so that the second transmission service end sends these multiple fragments to the second storage service end included in the second network. The second storage service end reorganizes these multiple fragments to obtain the target file. Based on the above process, the transmission of the target file between the first network and the second network is realized, that is, the cross-network transmission of the target file is realized.

[0009] It can be seen from the above technical solutions that the present application has the following advantages: the first transmission server and the second transmission server establish a communication connection between the first network and the second network. After the first transmission server obtains multiple fragments of the target file of the first application from the first network, the multiple fragments are forwarded to the second transmission server, so that the second transmission server sends the multiple fragments to the second storage service server of the second network, and the second storage service server reassembles the multiple fragments to obtain the target file. In this process, the first transmission server and the second transmission server only transmit the fragments, and do not reassemble the multiple fragments. In addition, the fragment transmission between the first transmission server and the second transmission server is a cross-network transmission. In other words, the technical solution provided by the present application reduces the number of fragmentation and reorganization operations, reduces the complexity of the operation, and improves the transmission efficiency when the target file that needs to be fragmented is transmitted across the network, compared with the traditional method.

[0010] In some possible implementations of the first aspect, the first network and the second network are different networks, and there are multiple possible situations. Optionally, the first network corresponds to the first cloud platform, and the second network corresponds to the second cloud platform, that is, the first network and the second network are networks used by different cloud platforms. At this time, the first network and the second network can be networks used by the same tenant on different cloud platforms, or networks used by different tenants on different cloud platforms. In addition, when the first network and the second network correspond to different cloud platforms, the first network and the second network can be deployed in the same region or in different regions, which is not specifically limited here. Optionally, the first network and the second network can be networks of different tenants on the same cloud platform. Specifically, the first network is the network of the first tenant on the first cloud platform, and the second network is the network of the second tenant on the first cloud platform. When the first network and the second network correspond to different tenants of the same cloud platform, the first network and the second network can be deployed in the same region or in different regions, which is not specifically limited here. Optionally, the first network and the second network can also be deployed in different regions of the same cloud platform. Specifically, the first network is deployed in the first region of the first cloud platform, and the second network is deployed in the second region of the first cloud platform. At this time, the first network and the second network can correspond to the same tenant or different tenants, which is not limited here. In general, the first network and the second network are different networks, including corresponding to different cloud platforms, corresponding to different tenants, or deployed in different regions, which is not limited here.

[0011] In this application, different networks are connected through the file transfer system, and large files can be transferred between different networks. In addition, different networks have multiple possibilities, which can flexibly adapt to different scene requirements, enrich the application scenarios of the technical solution of this application, and enhance the flexibility of the technical solution of this application.

[0012] In some possible implementations of the first aspect, the first transmission server may apply a cache function, and before sending multiple fragments to the second transmission server, the first transmission server may store the multiple fragments in the first cache. That is, after the first transmission server receives multiple fragments from the first application, it caches the multiple fragments in the first cache. The first application calls the sending interface of the first transmission server, and the first transmission server sends the multiple fragments in the first cache to the second transmission server one by one. Similarly, the second transmission server may also apply a cache function, and store the multiple fragments received from the first transmission server in the second cache. The second transmission server sends the multiple fragments stored in the second cache to the second storage service server one by one.

[0013] In some possible implementations of the first aspect, considering network instability and other reasons, there may be a situation where a segment transmission fails between the first transmission server and the second transmission server. In this case, the second transmission server can send a retransmission request to the first transmission server, and the retransmission request carries the sequence number of the target segment, or the sequence number of the target segment and the file name of the target file, wherein the target segment is included in multiple segments of the target file. In response to the retransmission request, the first transmission server sends the target segment stored in the first cache to the second transmission server.

[0014] In the present application, in the fragment retransmission scheme, the first transmission server can directly obtain the target fragment that needs to be retransmitted from the cache, optimize the cross-network transmission mechanism, ensure the successful transmission of large files through the fragment retransmission function, and increase the stability of the technical solution.

[0015] In some possible implementations of the first aspect, the first transmission server may also send multiple slices to the first storage service client on the same first network, so that the first network may also store the target file. Specifically, the first transmission server may obtain a first instruction, and the first instruction instructs to store the target file on the first storage service client. The first instruction may also be issued by the first application to the first transmission server when calling the sending interface of the first transmission server, or may be issued after or before calling the sending interface, which is not specifically limited here. In response to the first instruction, the first transmission server sends multiple slices to the first storage service client, so that the first storage service client obtains the multiple slices, and reorganizes the multiple slices to obtain the target file.

[0016] In this application, the file transfer service provides a parameter setting for whether to perform local backup. When local backup is required, multiple shards are sent to the local first storage service client to achieve backup of the target file. Compared with the traditional solution, each file transfer must be backed up on the first storage service client, which reduces the storage space occupied and the number of reorganizations during the file transfer process. In addition, it can also be flexibly configured according to the needs of actual applications, which improves the practicality of the technical solution of this application.

[0017] In some possible implementations of the first aspect, the first storage service client provides an object storage service or a file storage service.

[0018] In some possible implementations of the first aspect, the second storage service client provides an object storage service or a file storage service.

[0019] In this application, there are multiple possibilities for storage services provided by each network, which further enriches the application scenarios of the technical solution of this application.

[0020] In some possible implementations of the first aspect, the first transmission server and the second transmission server are included in a file transmission system, and the file transmission system establishes a communication connection between the first network and the second network through the first transmission server and the second transmission server to implement file transmission between the first network and the second network. The first transmission server and the second transmission server can be provided by the same entity or by different entities, which is not specifically limited here.

[0021] A second aspect of the present application further provides a communication device, the communication device being a first transmission service end in a first network, the communication device comprising:

[0022] The transceiver unit is used to obtain multiple slices sent from the first application in the first network, and the multiple slices are included in the target file. The multiple slices are sent to the second transmission server, so that the second transmission server sends the multiple slices to the second storage service client, and the second storage service client is used to reassemble the multiple slices to obtain the target file. The slice transmission between the first transmission server and the second transmission server is a cross-network transmission. The second transmission server and the second storage service client are included in the second network.

[0023] The communication device is used to implement the file transmission method shown in the aforementioned first aspect, or any possible implementation of the first aspect.

[0024] A third aspect of the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster implements the method disclosed in the first aspect or any possible implementation of the first aspect.

[0025] A fourth aspect of the present application provides a computer program product comprising instructions, which, when executed by a computer device cluster, enables the computer device cluster to implement the method disclosed in the first aspect or any possible implementation of the first aspect.

[0026] In a fifth aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method disclosed in the first aspect or any possible implementation of the first aspect.

[0027] The beneficial effects shown in the second to fifth aspects of the present application are similar to those of the first aspect or any possible implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the system architecture provided for an embodiment of the present application;

[0029] Figure 2 Another schematic diagram of a system architecture provided for an embodiment of the present application;

[0030] Figure 3 A flowchart of a file transmission method provided in an embodiment of the present application;

[0031] Figure 4 Another flowchart of the file transmission method provided in the embodiment of the present application;

[0032] Figure 5 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The embodiments of the present application provide a file transmission method and related equipment for reducing the number of times large files are fragmented and reassembled during cross-network transmission, reducing the complexity of operations, and thus improving transmission efficiency.

[0036] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0037] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged in appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attribute in the embodiments of the present application when describing. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment containing a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or", describes the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B, can represent: A exists alone, A and B exist simultaneously, and B exists alone, wherein A, B can be singular or plural. The character " / " generally represents that the associated objects before and after are a kind of "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0038] First, see Figure 1 , Figure 1 A schematic diagram of the system architecture provided for an embodiment of the present application.

[0039] With the help of Figure 1 A brief description of the cloud platform scenario is given below. Figure 1 As shown, the cloud platform 30 may provide interfaces related to cloud computing services, such as a configuration page (i.e., an interface) or an application programming interface (API) for users to access cloud services. As the top-level object for managing cloud services and / or cloud resources, the user uses the client 10 to log in to the cloud platform 30 via the Internet 20 using the account and password registered on the cloud platform 30 to configure and use cloud services, which are provided by the infrastructure managed by the cloud platform 30. The cloud platform 30 manages the infrastructure, which includes multiple cloud data centers located in different regions, and each region is provided with at least one cloud data center. For example Figure 1 The region 1 shown includes cloud data center 1 and cloud data center 2, and the region 2 includes cloud data center 3 and cloud data center 4. Each cloud data center is provided with multiple servers, and service instances are running on the servers.

[0040] Among them, the business instance is an instance belonging to the user for running its business, including one or any combination of virtual machines, containers, and dedicated hosts. A virtual machine (VM) refers to a complete computer system with complete hardware system functions that runs in a completely isolated environment and is simulated by software. When creating a virtual machine in a server, part of the hard disk and memory capacity of the physical machine needs to be used as the hard disk and memory capacity of the virtual machine. Each virtual machine has an independent hard disk and operating system, and the user of the virtual machine can operate the virtual machine like using a server. Container is a virtualization technology in computer operating systems. This technology allows processes to run in a relatively independent and isolated environment (including independent file systems, namespaces, resource views, etc.), thereby simplifying the software deployment process, enhancing the portability and security of the software, and improving the utilization of system resources. Dedicated host (DeH) refers to dedicated physical host resources that users can enjoy exclusively. Users create cloud servers on dedicated hosts to meet their higher requirements for isolation, security, and performance. Users of dedicated hosts do not need to share the physical resources of the host with other users.

[0041] In the embodiment of the present application, storage services and file transfer services are deployed in the business instance, and users can use the storage services and file transfer services of the cloud platform 30 through the client 10. The storage service is used to store files, and the file transfer service is used to realize file transfer between different networks.

[0042] Combine the following Figure 2 , further illustrating the system architecture provided in the embodiments of the present application.

[0043] like Figure 2 As shown, the first network and the second network are different networks, and the difference is that they are deployed on different cloud platforms, in different regions, or belong to different tenants. The region is the region mentioned above. Belonging to different tenants means being used by different tenants. The tenants mentioned here can be enterprise-level users or individual users, and the specific details are not limited here.

[0044] The first network includes a first storage service end and a first transmission service end, and the second network includes a second storage service end and a second transmission service end. The first storage service end and the second storage service end provide storage services, including reorganizing the fragments of large files, storing large files, etc. The first transmission service end and the second transmission service end provide file transfer services for receiving or sending fragments of large files. The file transfer system provided in an embodiment of the present application includes a first transmission service end and a second transmission service end, which are used to realize file transfer between the first network and the second network. Exemplarily, it is assumed that the first application calls the first transmission service end to send a file to the second network, the first transmission service end acts as a sender, and the second network includes the second transmission service end as a receiver, thereby realizing the transmission of files between the first network and the second network.

[0045] In some optional implementations, the first network and the second network may further include a distributed cache service, and the file transfer system may apply the distributed cache service to cache segments of a large file.

[0046] In some optional implementations, the second network may further include a second application, and the second application may call the second transmission server to obtain segments of the large file to obtain a complete large file.

[0047] The following is a detailed description of the above process with the help of the flowchart. Figure 3 , Figure 3 A flowchart of a file transmission method provided in an embodiment of the present application includes:

[0048] 301. A first transmission server obtains a plurality of fragments sent from a first application in a first network, wherein the plurality of fragments are included in a target file.

[0049] The first application included in the first network segments the target file to be transmitted to the second network, and calls the segment upload interface of the first transmission server to send the multiple segments included in the target file to the first transmission server in the first network. The target file is a large file, and a large file refers to a file whose data volume exceeds a threshold. The size of the threshold can be set according to the needs of the actual application, and is not specifically limited here. Optionally, an application program interface (API) configuration interface can be provided, and the user can set the threshold. Exemplarily, a file with a data volume exceeding 10MB can be set as a large file.

[0050] In addition, when the first application slices the target file, the data volume of each slice can be set to be the same, different, or partially the same, which is not limited here. For example, assuming that the data volume of the target file is 24MB, the first application can divide the target file into 6 slices on average, and the data volume of each slice is 4MB. It is also possible to set the data volume of each of the first 4 slices to 5MB, and the data volume of the last slice to 4MB.

[0051] It should be noted that the embodiment of the present application does not limit the first application, and the first application is an application that communicates with the first transmission server and can segment large files. For example, the first application can be a data statistics application, and the target file includes various data collected by the first application during the statistical period.

[0052] 302. The first transport server sends multiple fragments to the second transport server.

[0053] The first transmission server receives multiple fragments from the first application one by one, and the first application calls the sending interface of the first transmission server, so that the first transmission server sends the multiple fragments to the second transmission server in the second network one by one. Among them, a one-to-one corresponding channel is configured between the sending interface of the first transmission server and the receiving interface of the second transmission server for transmitting the fragments. At this time, the first transmission server acts as a sending end and the second transmission server acts as a receiving end.

[0054] It should be noted that a one-to-one channel is configured between the sending interface of the first transmission server and the receiving interface of the second transmission server. When the first application calls the sending interface of the first transmission server, it will transmit the fragment to the receiving interface of the second transmission server that uniquely corresponds to the sending interface, thereby ensuring the correct transmission of the fragment. Among them, the channel configuration between the sending interface and the receiving interface of different transmission servers can be preset or modified in the application, which is not limited here. For example, the channel between the sending interface and the receiving interface can be configured according to the needs of the tenant.

[0055] In an embodiment of the present application, the first transmission server and the second transmission server may be included in a file transfer system, and the file transfer system is used to connect the first network and the second network. In other words, the first network and the second network were originally not interoperable, and a communication channel between the first network and the second network was opened up through the file transfer system. In addition, the first transmission server and the second transmission server are respectively included in different networks. Specifically, the first transmission server is included in the first network, and the second transmission server is included in the second network. This means that in an embodiment of the present application, the shard transmission between the first transmission server and the second transmission server is a cross-network transmission.

[0056] It should be noted that the first transmission server and the second transmission server can be included in the file transfer system and the provider of the first transmission server and the second transmission server (such as a manufacturer, a seller, etc.) is not limited. In actual applications, the first transmission server and the second transmission server can be provided by the same entity or by different entities, and no specific limitation is made here.

[0057] The first network and the second network are different networks, including being deployed on different cloud platforms, in different regions, or belonging to different tenants. There are many possibilities:

[0058] Optionally, the first network and the second network may correspond to different cloud platforms respectively. For example, the first network corresponds to the first cloud platform, and the second network corresponds to the second cloud platform. In this solution, the first cloud platform and the second cloud platform may be different cloud platforms used by the same tenant, that is, a heterogeneous cloud scenario. The first cloud platform and the second cloud platform may also be different cloud platforms used by different tenants, in which case the file transfer between the first network and the second network is cross-tenant and cross-cloud transfer. In addition, when the first network and the second network correspond to different cloud platforms, the first network and the second network may be deployed in the same region or in different regions, which is not specifically limited here.

[0059] Optionally, the first network and the second network may correspond to different tenants respectively. For example, the first network may be a network deployed by the first tenant on the first cloud platform, and the second network may be a network deployed by the second tenant on the first cloud platform. In this solution, the first network and the second network correspond to the same cloud platform, which is a homogeneous cloud scenario. The first network and the second network may be deployed in the same region or in different regions, which is not specifically limited here.

[0060] Optionally, the first network can be deployed in the first region of the first cloud platform, and the second network can be deployed in the second region of the first cloud platform. In this solution, the first network and the second network correspond to the same cloud platform, which is a homogeneous cloud scenario. The first network and the second network can correspond to the same tenant or different tenants, and the specific details are not limited here. If the first network and the second network correspond to the same tenant, the file transfer between the first network and the second network is a cross-regional transfer. If the first network and the second network correspond to different tenants, the file transfer between the first network and the second network is a cross-regional and cross-tenant transfer.

[0061] In the embodiments of the present application, there are multiple possibilities for the relationship between the first network and the second network, which can flexibly adapt to different scenario requirements, enrich the application scenarios of the technical solution of the present application, and enhance the flexibility of the technical solution of the present application.

[0062] In some optional implementations, the first transmission server may apply a distributed cache service. Specifically, when receiving multiple fragments one by one, the first transmission server may store the multiple fragments in the first cache. At this time, the first transmission server sends the multiple fragments to the second transmission server, which includes the first transmission server sending the multiple fragments stored in the first cache to the second transmission server.

[0063] It is understandable that the first transmission server sends multiple fragments stored in the first cache to the second transmission server, which does not limit the first transmission server to store multiple fragments in the first cache before sending the multiple fragments to the second transmission server. The first transmission server stores the fragments in the first cache, and it is emphasized that after receiving the fragments from the first application, the first transmission server can store the fragments first, and then send the fragments to the second transmission server. In other words, the first transmission server applies a cache mechanism for backup, which can prevent the inconvenience caused by fragment transmission failures.

[0064] Optionally, if the first transmission server determines that the transmission of a target segment among multiple segments has failed, the first transmission server may send a retransmission request to the first application, and the retransmission request carries the sequence number of the target segment, or the sequence number of the target segment and the file name of the target file. Furthermore, the retransmission request indicates the retransmission of the target segment. Exemplarily, it is assumed that the first application simultaneously sends segments of multiple large files to the first transmission server, that is, a large file concurrency scenario. In this scenario, the retransmission request sent by the first transmission server to the first application carries not only the sequence number of the target segment, but also the file name of the target file, so that the first application determines that the segment that failed to be transmitted is the target segment in the target file, thereby ensuring successful retransmission of the segment.

[0065] It should be noted that the first transmission server requesting the first application to retransmit the target fragment does not affect the process of the first transmission server sending other fragments except the target fragment among the multiple fragments to the second transmission server. In other words, the first transmission server can simultaneously send other fragments except the target fragment among the multiple fragments to the second transmission server and request the first application to retransmit the target fragment.

[0066] It is understandable that the above description does not limit the first transmission server to send all the fragments included in the target file to the second transmission server after obtaining them, so as to ensure the transmission efficiency. Of course, in actual applications, the first transmission server can also send these fragments to the second transmission server after determining that all the fragments included in the target file have been obtained. The specific application scheme can be determined according to the needs of the actual application, and is not specifically limited here.

[0067] 303. The second transmission server sends multiple fragments to the second storage service server included in the second network.

[0068] The second transmission server can receive multiple slices from the first transmission server one by one through the receiving interface, and send these multiple slices to the second storage service. The second transmission server and the second storage service are located in the same region of the same cloud platform and provide services to the same tenant. In addition, as described above, a one-to-one corresponding channel is configured between the sending interface of the first transmission server and the receiving interface of the second transmission server, which means that in an embodiment of the present application, a corresponding transmission channel is configured for each tenant to ensure the accuracy of large file transmission across the network.

[0069] In some optional implementations, the second transmission server may also apply a distributed cache service. Specifically, when receiving multiple fragments one by one, the second transmission server may store the multiple fragments in the second cache. At this time, the second transmission server sends the multiple fragments to the second storage service client, which includes the second transmission server sending the multiple fragments stored in the second cache to the second storage service client.

[0070] It is understandable that the second transmission server sends the multiple fragments stored in the second cache to the second storage service client, which does not limit the second transmission server to store the multiple fragments in the second cache before sending the multiple fragments to the second storage service client. The second transmission server stores the fragments in the second cache, and it is emphasized that after receiving the fragments from the first transmission service, the second transmission server can first store the fragments and then send the fragments to the second storage service client. In other words, the second transmission server applies a cache mechanism for backup, which can prevent the inconvenience caused by fragment transmission failures.

[0071] 304. The second storage service client reorganizes the multiple slices to obtain the target file.

[0072] After receiving the multiple slices, the second storage service client can reorganize the multiple slices to obtain the target file. Based on this, the transmission of the target file between the first network and the second network is realized, that is, the cross-network transmission of the large file is realized.

[0073] Optionally, if the second storage service determines that the transmission of the target slice among the multiple slices has failed, the second storage service may send a retransmission request to the second transmission service, and the retransmission request carries the sequence number of the target slice, or the sequence number of the target slice and the file name of the target file. Moreover, the retransmission request indicates the retransmission of the target slice. The second transmission service may respond to the retransmission request and send the target slice to the second storage service. If the second cache of the second transmission service includes the target slice, the second transmission service sends the target slice stored in the second cache to the second storage service; if the second cache does not include the target slice, the second transmission service may forward the retransmission request to the first transmission service so that the first transmission service retransmits the target slice to the second transmission service.

[0074] For example, assume that the second transmission server sends multiple large file segments to the second storage service client at the same time, that is, in a large file concurrency scenario. In this scenario, the retransmission request sent by the first transmission server to the first application carries not only the sequence number of the target segment, but also the file name of the target file, so that the first application determines that the segment that failed to be transmitted is the target segment in the target file, thereby ensuring successful retransmission of the segment.

[0075] Based on the foregoing description, it can be known that in an embodiment of the present application, the first transmission service end and the second transmission service end establish a communication connection between the first network and the second network. After the first transmission service end obtains multiple fragments of the target file of the first application from the first network, the multiple fragments are forwarded to the second transmission service end, so that the second transmission service end sends the multiple fragments to the second storage service end of the second network, and the second storage service end reorganizes the multiple fragments to obtain the target file. In this process, the first transmission service end and the second transmission service end only transmit the fragments, and do not reorganize the multiple fragments. In addition, the fragment transmission between the first transmission service end and the second transmission service end is a cross-network transmission. In other words, the technical solution provided by the present application reduces the number of fragmentation and reorganization operations, reduces the complexity of the operation, and improves the transmission efficiency when the target file that needs to be fragmented is transmitted across the network compared with the traditional method.

[0076] In some optional implementations, when the first transmission server sends multiple fragments to the second transmission server, fragment transmission failure may occur due to factors such as network instability. To solve this problem, the file transmission method provided in the embodiment of the present application also establishes a retransmission mechanism for the first transmission server and the second transmission server.

[0077] In summary, if the second transmission server determines that the transmission of the target segment among the multiple segments fails, the second transmission server can send a retransmission request to the first transmission server application. The retransmission request carries the sequence number of the target segment, or carries the sequence number of the target segment and the file name of the target file, and indicates to retransmit the target segment among the multiple segments. The first transmission server obtains the retransmission request from the second transmission server, and in response to the retransmission request, sends the target segment stored in the first cache to the second transmission server.

[0078] For example, assume that the first transmission server sends multiple large file segments to the second transmission server at the same time, that is, a large file concurrent scenario. In this scenario, the retransmission request sent by the second transmission server to the first transmission server carries not only the sequence number of the target segment, but also the file name of the target file, so that the first transmission server determines that the segment that failed to be transmitted is the target segment in the target file, thereby ensuring successful retransmission of the segment.

[0079] It should be noted that the second transmission service end's request to the first transmission service end to retransmit the target fragment does not affect the process of the second transmission service end sending the other fragments except the target fragment among the multiple fragments to the second storage service end. In other words, the second transmission service end can simultaneously send the other fragments except the target fragment among the multiple fragments to the second storage service end and request the first transmission service end to retransmit the target fragment.

[0080] It is understandable that the above description does not limit the second transmission server to send these fragments to the second storage service client after obtaining all the fragments included in the target file, so as to ensure the transmission efficiency. Of course, in actual applications, the second transmission server can also send these fragments to the second storage service client after determining that all the fragments included in the target file have been obtained. The specific application scheme can be determined according to the needs of the actual application, and is not specifically limited here.

[0081] In the present application, in the fragment retransmission scheme, the first transmission server can directly obtain the target fragment that needs to be retransmitted from the cache, optimize the cross-network transmission mechanism, ensure the successful transmission of large files through the fragment retransmission function, and increase the stability of the technical solution.

[0082] In some optional implementations, the first network can cache the target file locally, that is, the first storage service included in the first network can be applied to store the target file in the first storage service. In this solution, the first application can send a first instruction to the first transmission service, and the first instruction instructs to store the target file in the first storage service. This means that the local backup parameter of the file transfer service is set to yes. The first transmission service responds to the first instruction and sends multiple slices to the first storage service so that the first storage service reorganizes the multiple slices to obtain the target file. Among them, the first storage service and the first transmission service are located in the same region of the same cloud platform and provide services to the same tenant.

[0083] There are many possible ways for the first transmission server to send multiple shards to the first storage service client, which are described below respectively:

[0084] Optionally, a distributed cache service is applied on the first transmission server, and a plurality of shards are stored in the first cache. If the first application sends a first instruction to the first transmission server when calling the shard upload interface of the first transmission server, or after calling the shard upload interface of the first transmission server. At this time, the first transmission server can send the plurality of shards stored in the first cache to the first storage service client. If the first application sends a first instruction to the first transmission server before calling the shard upload interface of the first transmission server. At this time, the first transmission server needs to wait until the first cache stores the shards included in the target file, and then send the plurality of shards stored in the first cache to the first storage service client.

[0085] It is understandable that the first transmission server sends the multiple slices stored in the first cache to the first storage service client, which does not limit the first transmission server to store the multiple slices in the first cache before sending the multiple slices to the first storage service client. The first transmission server stores the slices in the first cache, and it is emphasized that after receiving the slices from the first application, the first transmission server can store the slices first, and then send the slices to the first storage service client. In other words, the first transmission server applies the cache mechanism for backup.

[0086] It should be noted that the first transmission service sends multiple fragments to the first storage service, which does not affect the process of the first transmission service sending multiple fragments to the second transmission service. In other words, the embodiment of the present application does not limit the execution order of the first transmission service sending multiple fragments to the first storage service and the first transmission service sending multiple fragments to the second transmission service.

[0087] In some optional implementations, for solutions that require local backup, the first application may also directly send multiple fragments included in the target file to the first storage service client.

[0088] In this application, the file transfer service provides a parameter setting for whether to perform local backup. When local backup is required, multiple shards are sent to the local first storage service client to achieve backup of the target file. Compared with the traditional solution, each file transfer must be backed up on the first storage service client, which reduces the storage space occupied and the number of reorganizations during the file transfer process. In addition, it can also be flexibly configured according to the needs of actual applications, which improves the practicality of the technical solution of this application.

[0089] In the embodiment of the present application, the first storage service end and the second storage service end can both provide multiple types of storage services. Specifically, the first storage service end provides object storage services, or provides file storage services. The second storage service end provides object storage services, or provides file storage services.

[0090] Among them, file storage services are generally in the form of directories and files (for example, C:\User\Program Files\Common Files). Data is stored and accessed in the form of files and organized according to directory results. File storage has its own file system. Each client is the same file system, which can be used by multiple clients and is easy to share. The difference between object storage services and file storage services is that objects in a layer structure will no longer have a hierarchical structure, but are characterized by extended metadata. Data and metadata are stored as a whole object, and data is found through a universally unique identifier (UUID).

[0091] In this application, there are multiple possibilities for storage services provided by each network, which further enriches the application scenarios of the technical solution of this application.

[0092] In some optional implementations, after the second storage service successfully reorganizes the multiple slices to obtain the target file, that is, after step 304, the second storage service may feedback that the transmission is completed. At this time, the first transmission service and the second transmission service may delete the multiple slices corresponding to the target file cached by each, thereby reducing the storage space occupied.

[0093] Similarly, in a large file concurrent scenario, the second storage service client can provide feedback after each large file is successfully reorganized, or provide unified feedback on the successful reorganization of multiple files, so that the first transmission server and the second transmission server can delete the multiple fragments corresponding to the large files cached by each of them to reduce storage space occupancy.

[0094] In general, the business process of the file transmission method provided in the embodiment of the present application is as follows: Figure 4 See Figure 4 , Figure 4 A flowchart of a file transfer method provided in an embodiment of the present application.

[0095] During the file transfer process, the first application can call the slice upload interface of the first transmission server and execute step 1. Send multiple slices of the file to the first transmission server. The first transmission server executes step 2. Receives and caches multiple slices. These multiple slices are stored in the first cache. The first application can call the sending interface of the first transmission server and execute step 3. File transfer. The file transfer mentioned here refers to cross-network file transfer, that is, the first transmission server transfers the file to other networks. The first transmission server, as the sending end of the file transfer, responds to step 3 and executes step 4. Sends the slices to the second transmission server across the network. The slices here include multiple slices of the file. The second transmission server, as the receiving end of the file transfer, executes step 5. Receives and caches the slices. These multiple slices are stored in the second cache. The second transmission server can also execute step 6. Upload the slices to the second storage service end. The second transmission server can upload multiple slices of the file to the second storage service end one by one. The second storage service end receives the multiple slices and executes step 7. Reorganizes the multiple slices to obtain the file uploaded by the first application.

[0096] In some optional implementations, if the second storage service determines that a fragment is missing when executing step 7, so that the file uploaded by the first application cannot be obtained, the second storage service can feedback the transmission failure to the second transmission service. If the second cache of the second transmission server does not include the missing fragment, the second transmission server executes step 8. Sends a retransmission request to the first transmission server. The first transmission server responds to the retransmission request and executes step 9. Resends the missing fragment to the second transmission server. The second transmission server receives and caches the missing fragment, and resends the missing fragment to the second storage service, so that the second storage service obtains multiple fragments of the file to ensure successful reorganization.

[0097] In some optional implementations, if the second storage service determines that multiple fragments are successfully reassembled when executing step 7 to obtain the file uploaded by the first application. Then the second storage service executes step 10. Feedback the completion of the transmission to the second transmission server. The second transmission server can respond to step 10 and delete multiple fragments of the file stored in the second cache to free up storage space. In addition, the second transmission server can also execute step 11. Feedback the success of the transmission to the first transmission server. In a scenario where the first transmission server does not need to perform local backup (that is, the local backup parameter is set to no), or in a scenario where the first transmission server needs local backup and the local backup is successful, the first transmission server can respond to step 11 and delete multiple fragments of the file stored in the first cache to free up storage space.

[0098] In some optional implementations, if there is a local backup requirement, that is, in the solution where the local backup parameter is set to yes, the first transmission service end may also perform step 12. Upload the slices to the first storage service end. The first transmission service end may upload multiple slices of the file to the first storage service end one by one. The first storage service end receives multiple slices and performs step 13. Reorganizes multiple slices to obtain the files uploaded by the first application. Similarly, if the first storage service end determines that a slice is missing when performing step 13, so that the file uploaded by the first application cannot be obtained, the first storage service end may feedback the transmission failure to the first transmission service end, so that the first transmission service end initiates a retransmission request to the first application to obtain the missing slices. If the first storage service end reorganizes successfully, the file uploaded by the first application is obtained. The first storage service end may feedback the transmission success to the first transmission service end. If the second storage service end also feedbacks the transmission completion at this time, the first transmission service end may delete the multiple slices of the file stored in the first cache to release storage space.

[0099] It should be noted that Figure 4 The specific implementation process of each step has been described in detail in the previous text, so it will not be repeated here.

[0100] In some optional implementations of the present application, the second network may further include a second application. The second application may directly read multiple slices included in the target file from the second transmission service end or the second storage service end, and reorganize the multiple slices to obtain the target file. The target file may also be obtained from the second storage service end. In other words, after obtaining the multiple slices, the second transmission service end or the second storage service end may notify the second application to read the slices. After obtaining the target file, the second storage service end may notify the second application to perform application reading. This is not specifically limited here.

[0101] The following is a description of the related devices provided in the embodiments of the present application. Figure 5 , Figure 5 This is a schematic diagram of the structure of a communication device provided in an embodiment of the present application. Figure 5 As shown, the communication device 500 is a first transmission service end in a first network, and the communication device 500 includes a transceiver unit 501 .

[0102] In some optional implementations, the transceiver unit 501 is used to: obtain multiple slices sent from the first application in the first network, and the multiple slices are included in the target file. Send the multiple slices to the second transmission server, so that the second transmission server sends the multiple slices to the second storage service, and the second storage service is used to reassemble the multiple slices to obtain the target file. The slice transmission between the first transmission server and the second transmission server is a cross-network transmission, and the second transmission server and the second storage service are included in the second network.

[0103] In some optional implementations, the relationship between the first network and the second network satisfies any of the following: the first network corresponds to the first cloud platform, and the second network corresponds to the second cloud platform. Alternatively, the first network is the network of the first tenant on the first cloud platform, and the second network is the network of the second tenant on the first cloud platform. Alternatively, the first network is deployed in the first region of the first cloud platform, and the second network is deployed in the second region of the first cloud platform.

[0104] In some optional implementations, the communication device 500 further includes a processing unit 502, configured to store a plurality of fragments in the first cache.

[0105] The second transmission server sends a plurality of fragments stored in a second cache to the second storage service client, and the second cache is used to store the fragments received by the second transmission server.

[0106] In some optional implementations, the transceiver unit 501 is further configured to: obtain a retransmission request from the second transmission server, the retransmission request indicating retransmission of a target segment among multiple segments, the retransmission request carrying a sequence number of the target segment, or carrying the sequence number of the target segment and a file name of the target file, and in response to the retransmission request, send the target segment stored in the first cache to the second transmission server.

[0107] In some optional implementations, the transceiver unit 501 is further configured to: obtain a first instruction, the first instruction instructing to store the target file in a first storage service client, the first storage service client being included in the first network, and send a plurality of slices to the first storage service client, so that the first storage service client reassembles the plurality of slices to obtain the target file.

[0108] In some optional implementations, the first storage service client provides an object storage service, or provides a file storage service.

[0109] In some optional implementations, the second storage service client provides an object storage service, or provides a file storage service.

[0110] The communication device 500 is used to execute the operations performed by the first transmission server in the aforementioned embodiment to implement the file transmission method provided in the embodiment of the present application, and the details will not be repeated here.

[0111] See also Figure 6 , Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0112] like Figure 6 As shown, computing device 600 includes: bus 603, memory 604, processor 605 and communication interface 606. Processor 605, memory 604 and communication interface 606 communicate with each other through bus 603. It should be understood that the present invention does not limit the number of processors and memories in computing device 600.

[0113] The bus 603 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 603 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 604, the processor 605, and the communication interface 606).

[0114] The processor 605 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0115] The memory 604 may include a volatile memory, such as a random access memory (RAM). The processor 605 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0116] The memory 604 stores executable program codes, and the processor 605 executes the executable program codes to respectively implement the functions of the transceiver unit 601 (corresponding to the aforementioned transceiver unit 501) and the processing unit 602 (corresponding to the aforementioned processing unit 502), thereby implementing the file transmission method. That is, the memory 604 stores instructions for executing the file transmission method provided in the embodiment of the present application.

[0117] The communication interface 606 uses a transceiver unit such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0118] The computing device 600 is used to execute the operations performed by the first server in the aforementioned embodiment to implement the screen sharing method provided in the embodiment of the present application.

[0119] An embodiment of the present invention further provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0120] See also Figure 7 , Figure 7 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application.

[0121] like Figure 7 As shown, the computing device cluster includes at least one computing device 600. Instructions may be stored in the memory 604 in one or more computing devices 600 in the computing device cluster, and are used to execute the instructions of the file transmission method provided in the embodiment of the present application.

[0122] It should be noted that the memory 604 in different computing devices 600 in the computing device cluster can store different instructions for executing some functions. That is, the instructions stored in the memory 604 in different computing devices 600 can implement the functions of one or more modules in the transceiver unit 601 and the processing unit 602.

[0123] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the instructions are run on at least one computer device included in the computing device cluster, the at least one computer device included in the computing device cluster executes the above-mentioned file transfer method.

[0124] The embodiment of the present application also provides a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes computer program instructions, which are used to execute the above-mentioned file transfer method. Specifically, when the computer program instructions are executed by a computing device cluster, the computing device cluster implements the above-mentioned file transfer method.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A file transmission method, characterized in that: The method is applied to a first transmission server in a first network, and the method includes: Acquire a plurality of fragments sent by a first application from the first network, wherein the plurality of fragments are included in a target file; Sending the multiple slices to a second transmission server, so that the second transmission server sends the multiple slices to a second storage service client, the second storage service client is used to reassemble the multiple slices to obtain the target file, and the second transmission server and the second storage service client are included in a second network; Among them, the segment transmission between the first transmission server and the second transmission server is cross-network transmission.

2. The method according to claim 1, characterized in that: The relationship between the first network and the second network satisfies at least one of the following: The first network corresponds to a first cloud platform, and the second network corresponds to a second cloud platform; or, The first network is a network of the first tenant on the first cloud platform, and the second network is a network of the second tenant on the first cloud platform; or, The first network is deployed in the first region of the first cloud platform, and the second network is deployed in the second region of the first cloud platform.

3. The method according to claim 1 or 2, characterized in that: Before the first transmission server sends the plurality of fragments to the second transmission server, the method further includes: The first transmission server stores the plurality of fragments in a first cache; The second transmission server sends the multiple slices to the second storage service client, including: the second transmission server sends the multiple slices stored in a second cache to the second storage service client, and the second cache is used to store the slices received by the second transmission server.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining a retransmission request from the second transmission server, the retransmission request indicating retransmission of a target segment among the multiple segments, the retransmission request carrying a sequence number of the target segment, or carrying the sequence number of the target segment and a file name of the target file; In response to the retransmission request, the target segment stored in the first cache is sent to the second transmission server.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining a first instruction, wherein the first instruction instructs to store the target file in a first storage service end, wherein the first storage service end is included in the first network; The multiple slices are sent to the first storage service client, so that the first storage service client reorganizes the multiple slices to obtain the target file.

6. The method according to claim 5, characterized in that The first storage service client provides object storage service or file storage service.

7. The method according to any one of claims 1 to 6, characterized in that The second storage service client provides an object storage service or a file storage service.

8. A communication device, characterized in that: The communication device is a first transmission service end in a first network, and the communication device includes: Transceiver unit, used for: Acquire a plurality of fragments sent by a first application from the first network, wherein the plurality of fragments are included in a target file; Sending the multiple slices to a second transmission server, so that the second transmission server sends the multiple slices to a second storage service client, the second storage service client is used to reassemble the multiple slices to obtain the target file, and the second transmission server and the second storage service client are included in a second network; Among them, the segment transmission between the first transmission server and the second transmission server is cross-network transmission.

9. The device according to claim 8, characterized in that The relationship between the first network and the second network satisfies any one of the following: The first network corresponds to a first cloud platform, and the second network corresponds to a second cloud platform; or, The first network is a network of the first tenant on the first cloud platform, and the second network is a network of the second tenant on the first cloud platform; or, The first network is deployed in the first region of the first cloud platform, and the second network is deployed in the second region of the first cloud platform.

10. The device according to claim 8 or 9, characterized in that The communication device also includes a processing unit, further configured to store the plurality of fragments in a first cache; The second transmission server sends the multiple fragments stored in the second cache to the second storage service client, and the second cache is used to store the fragments received by the second transmission server.

11. The device according to claim 10, characterized in that The transceiver unit is further used for: Obtaining a retransmission request from the second transmission server, the retransmission request indicating retransmission of a target segment among the multiple segments, the retransmission request carrying a sequence number of the target segment, or carrying the sequence number of the target segment and a file name of the target file; In response to the retransmission request, the target segment stored in the first cache is sent to the second transmission server.

12. The device according to any one of claims 8 to 11, characterized in that The transceiver unit is further used for: Obtaining a first instruction, wherein the first instruction instructs to store the target file in a first storage service end, wherein the first storage service end is included in the first network; The multiple slices are sent to the first storage service client, so that the first storage service client reorganizes the multiple slices to obtain the target file.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 7.

15. A computer-readable storage medium, characterized in that: includes computer program instructions; When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method of any one of claims 1 to 7.

Images