Data transmission method, device and equipment

Abstract

The present invention provides a data transmission method, device and equipment, which can be applied to the field of distributed database technology. The method includes: obtaining data to be transmitted; establishing a shared transmission link between any two data transmission nodes; calling a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data transmission to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed databases, and more specifically to a data transmission method, device, equipment, medium and program product. Background Art

[0002] A database cluster usually consists of multiple data nodes, each of which has multiple data shards. When querying data, communication channels need to be established between data shards for data transmission.

[0003] In the related art, when a data transmission request is received, a transmission link is often established between data slices. When a new data transmission request is received, a new transmission link still needs to be established between data slices. As the amount of data to be transmitted increases, the number of transmission links constructed increases accordingly, which places an excessive burden on the system. Summary of the invention

[0004] In view of the above problems, the present invention provides a data transmission method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present invention, a data transmission method is provided, comprising: obtaining data to be transmitted; establishing a shared transmission link between any two data transmission nodes; calling a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, transmitting the data to be transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data transmission to be transmitted is completed, clearing the transmission session corresponding to the data to be transmitted.

[0006] According to an embodiment of the present invention, the shared transmission links include multiple links; based on the shared transmission links and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, including: determining the number of shared transmission links between any two data transmission nodes; based on the number of shared transmission links, determining a target shared transmission link from multiple shared transmission links; based on the target shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the target transmission link and the data transmission session.

[0007] According to an embodiment of the present invention, a target shared transmission link is determined among multiple shared transmission links based on the number of shared transmission links, including: when the number of shared transmission links reaches a first preset number threshold, determining the number of data transmission sessions corresponding to each shared transmission link; and determining the target shared transmission link among the multiple shared transmission links, wherein the number of data transmission sessions corresponding to the target shared transmission link is the smallest among the number of data transmission sessions corresponding to each shared transmission link.

[0008] According to an embodiment of the present invention, determining a target transmission link from a plurality of shared transmission links according to the number of shared transmission links also includes: when the number of shared transmission links does not reach a first preset number threshold, creating a new shared transmission link, and using the new shared transmission link as the target shared transmission link.

[0009] According to an embodiment of the present invention, based on a shared transmission link and a data transmission session, data to be transmitted is transmitted between any two data transmission nodes, and also includes: obtaining emergency identification information of the data to be transmitted, the emergency identification information characterizing the urgency of transmission of the data to be transmitted; and performing data transmission on the data to be transmitted according to the emergency identification information.

[0010] According to an embodiment of the present invention, each shared transmission link includes a receiving end and a sending end; the data to be transmitted includes multiple data packets; the above method also includes: determining the preset number of received data in the receiving window of the receiving end; determining the number of data packets corresponding to the receiving end; when the number of data packets and the preset number of received data in the receiving window do not meet the preset conditions, delaying the transmission of the data transmission session corresponding to the data packet.

[0011] According to an embodiment of the present invention, the above method also includes: before the data to be transmitted is transmitted, using the sending end of the shared transmission link to compress the data to be transmitted; after the data to be transmitted is transmitted, using the receiving end of the shared transmission link to decompress the data to be transmitted.

[0012] According to an embodiment of the present invention, the method further comprises: performing status analysis on the data to be transmitted to obtain an analysis result; and when the analysis result indicates an abnormal status, clearing a data transmission session corresponding to the data to be transmitted.

[0013] The second aspect of the present invention provides a data transmission device, including: an acquisition module, used to acquire data to be transmitted; an establishment module, used to establish a shared transmission link between any two data transmission nodes; a creation module, used to call a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; a transmission module, used to transmit the data to be transmitted between any two data transmission nodes according to the shared transmission link and the data transmission session, wherein there is a mapping relationship between the shared transmission link and the data transmission session; and a first clearing module, used to clear the data transmission session corresponding to the data to be transmitted after the transmission of the data to be transmitted is completed.

[0014] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0016] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0017] According to an embodiment of the present invention, data to be transmitted is obtained; a shared transmission link is established between any two data transmission nodes; a session manager component is called to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared. Due to the use of a technical means of using a shared transmission link and a data transmission session to transmit the data to be transmitted between any two data transmission nodes, the problem of frequent establishment of communication links and too many generated communication links, which results in excessive burden on the system, is avoided, and the technical effect of reducing the system burden and supporting high-concurrency tasks of large-scale database clusters is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 An application scenario diagram of a data transmission method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown;

[0020] Figure 2 A flow chart of a data transmission method according to an embodiment of the present invention is shown;

[0021] Figure 3 A schematic diagram showing a data transmission method of the related art;

[0022] Figure 4 A schematic diagram showing a data transmission method according to an embodiment of the present invention is shown;

[0023] Figure 5 A performance display diagram of a shared transmission link according to an embodiment of the present invention is shown;

[0024] Figure 6 A performance display diagram of multiple shared transmission links according to an embodiment of the present invention is shown;

[0025] Figure 7 A schematic diagram showing a data transmission method according to another embodiment of the present invention is shown;

[0026] Figure 8 It shows a structural block diagram of a data transmission device according to an embodiment of the present invention; and

[0027] Fig. 9 A block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] An embodiment of the present invention provides a data transmission method, which includes acquiring data to be transmitted; establishing a shared transmission link between any two data transmission nodes; calling a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data transmission to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared.

[0033] Figure 1 An application scenario diagram of a data transmission method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0035] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0037] The server 105 may be a server that provides various services, such as a background management server (only as an example) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may perform data transmission processing between database clusters for received user requests and other data.

[0038] It should be noted that the data transmission method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the data transmission device provided in the embodiment of the present invention can generally be set in the server 105. The data transmission method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the data transmission device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0040] Figure 2 A flow chart of a data transmission method according to an embodiment of the present invention is shown.

[0041] like Figure 2 As shown, the data transmission method 200 of this embodiment includes operations S210 to S250. The method is applied to a database cluster, and the database cluster includes multiple data transmission nodes.

[0042] In operation S210, data to be transmitted is obtained.

[0043] According to an embodiment of the present invention, a database cluster (Massively Parallel Processing DatabaseCluster) is a distributed database architecture for processing large-scale data and executing complex queries. It distributes data and computing tasks to multiple data transmission nodes for parallel processing to improve query performance and scalability.

[0044] According to an embodiment of the present invention, the data to be transmitted may be data that needs to be transmitted in a database cluster when performing a data query.

[0045] In operation S220, a shared transmission link between any two data transmission nodes is established.

[0046] According to an embodiment of the present invention, a database cluster includes multiple data transmission nodes, wherein multiple data shards are distributed on the data transmission nodes.

[0047] According to an embodiment of the present invention, the data transmission nodes may be independent database servers constituting a database cluster, and are used to process and store data.

[0048] According to an embodiment of the present invention, data sharding may be used to classify and store data on a data transmission node, or to back up data on a data transmission node.

[0049] According to an embodiment of the present invention, the shared transmission link may be a Transmission Control Protocol (TCP) link.

[0050] In operation S230, the session manager component is called to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted.

[0051] According to an embodiment of the present invention, a data transmission session may be a link for data transmission between a data slice on a data transmission node and a shared transmission link corresponding to the data transmission node, and is used to enable the data slice on the data transmission node to receive and send data to be transmitted.

[0052] According to an embodiment of the present invention, when data to be transmitted sent by data shard 1 of data transmission node 1 to data shard 2 of data transmission node 2 is obtained, a data transmission session a1 associated with data shard 1 and a data transmission session a2 associated with data shard 2 can be constructed, the data transmission session a1 is used to receive the data to be transmitted transmitted by data shard 1 of data transmission node 1 and transmit the data to be transmitted to the data transmission session a2, and the data transmission session a2 is used to send the data to be transmitted to data shard 2 of data transmission node 2. Both the data transmission session a1 and the data transmission session a2 correspond to the data to be transmitted.

[0053] According to an embodiment of the present invention, the session manager component may be used to create a data transmission session and manage the created data transmission session, such as clearing the data transmission session.

[0054] According to an embodiment of the present invention, the data to be transmitted and the data transmission session have a one-to-one correspondence. For example, when the data to be transmitted a is received, a data transmission session a corresponding to it is created. When the data to be transmitted b is obtained, the data to be transmitted b cannot use the data transmission session a, and a new data transmission session b needs to be created.

[0055] According to the embodiment of the present invention, the data transmission session can reduce the burden on the system while ensuring the security of the data transmission process.

[0056] In operation S240, data to be transmitted is transmitted between any two data transmission nodes according to the shared transmission link and the data transmission session, wherein there is a mapping relationship between the shared transmission link and the data transmission session.

[0057] According to an embodiment of the present invention, a data transmission session and a shared transmission link have a mapping relationship. The data transmission node transmits the data to be transmitted to the shared transmission link through the data transmission session. The shared transmission link transmits the data to be transmitted to another data transmission node and transmits it to the corresponding data slice through the data transmission session.

[0058] According to an embodiment of the present invention, when the data to be transmitted sent by the data slice 1 of the data transmission node 1 to the data slice 2 of the data transmission node 2 is obtained, a mapping relationship is established between the data transmission session a1 and the data transmission session a2 of the data to be transmitted and the shared transmission link between the data transmission node 1 and the data transmission node 2 respectively. When data transmission is performed, the data slice 1 of the data transmission node 1 transmits the data to be transmitted to the shared transmission link through the data transmission session a1, and then transmits it to the data transmission session a2 through the shared transmission link. The data transmission session a2 sends the data to be transmitted to the data slice 2 of the data transmission node 2.

[0059] According to an embodiment of the present invention, when new data to be transmitted is obtained, it is only necessary to create a data transmission session corresponding to it, and establish a mapping relationship between the data transmission session and the shared transmission link. The data transmission of the new data to be transmitted between any two data transmission nodes can be realized through the shared transmission link, without creating a new transmission link to perform data transmission on the new data to be transmitted, so as to reduce the burden on the system.

[0060] In operation S250, after the transmission of the data to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared.

[0061] According to an embodiment of the present invention, a data transmission session is provided with identity identification information, and the identity identification information corresponds one-to-one to the data to be transmitted. When it is determined that the data to be transmitted is completed, the identity identification information corresponding to it is determined, and the corresponding data transmission session is cleared according to the identity identification information to release the data transmission session resources and avoid the impact of too many useless data session resources on other data session resources.

[0062] According to an embodiment of the present invention, data to be transmitted is obtained; a shared transmission link is established between any two data transmission nodes; a session manager component is called to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared. Due to the use of a technical means of using a shared transmission link and a data transmission session to transmit the data to be transmitted between any two data transmission nodes, the problem of frequent establishment of communication links and too many generated communication links, which results in excessive burden on the system, is avoided, and the technical effect of reducing the system burden and supporting high-concurrency tasks of large-scale database clusters is achieved.

[0063] Figure 3 A schematic diagram showing a data transmission method of the related art; Figure 4 A schematic diagram of a data transmission method according to an embodiment of the present invention is shown.

[0064] Database cluster 300 Figure 3 As shown, for example, data shard 2 of data transmission node 2 and data shard 5 of data transmission node 2 both need to send data to data shard 1 of data transmission node 1, and a transmission link needs to be established between data shard 2 of data transmission node 2 and data shard 1 of data transmission node 1, and data transmission between data shard 2 and data shard 1 is achieved based on the transmission link; a transmission link is established between data shard 5 of data transmission node 2 and data shard 1 of data transmission node 1, and data transmission between data shard 5 and data shard 1 is achieved based on the transmission link.

[0065] If data shard 2 of data transmission node 2 and data shard 5 of data transmission node 2 both need to send data to data shard 4 of data transmission node 1, it is necessary to establish a transmission link between data shard 2 of data transmission node 2 and data shard 4 of data transmission node 1, and realize data transmission between data shard 2 and data shard 4 based on the transmission link; establish a transmission link between data shard 5 of data transmission node 2 and data shard 4 of data transmission node 1, and realize data transmission between data shard 5 and data shard 4 based on the transmission link.

[0066] It can be seen that in the related art, when data shards between data transmission nodes are used for data transmission, a transmission link needs to be established each time data transmission is performed so as to realize data transmission between the data transmission nodes. Therefore, as the number of data transmission nodes N, the number of data shards P, and the number of data to be transmitted S increase, the number of transmission links C created can be expressed as: , the number of transmission links created will also increase exponentially, which will increase the system burden.

[0067] Database cluster 400 Figure 4 As shown, for example, data slice 2 of data transmission node 2 and data slice 5 of data transmission node 2 both need to send data to data slice 1 of data transmission node 1, and a shared transmission link can be established between data transmission node 2 and data transmission node 1. Data slice 2 establishes a data transmission session corresponding to the data to be transmitted, and the data transmission session transmits the data of data slice 2 to the shared transmission link, transmits it to the corresponding data transmission session through the shared transmission link, and then transmits it to data slice 1; data slice 5 establishes a data transmission session corresponding to the data to be transmitted, and the data transmission session transmits the data of data slice 5 to the shared transmission link, transmits it to the corresponding data transmission session through the shared transmission link, and then transmits it to data slice 1. If data slice 2 of data transmission node 2 and data slice 5 of data transmission node 2 both need to send data to data slice 4 of data transmission node 1, the manner in which data slice 2 of data transmission node 2 and data slice 5 of data transmission node 2 send data to data slice 1 of data transmission node 1 is similar, and will not be repeated here.

[0068] Data shard 3 of data transmission node 3 and data shard 6 of data transmission node 3 both need to send data to data shard 1 of data transmission node 1, and a shared transmission link can be established between data transmission node 3 and data transmission node 1. Data shard 3 establishes a data transmission session corresponding to its data to be transmitted, and the data transmission session transmits the data of data shard 3 to the shared transmission link, transmits it to the corresponding data transmission session through the shared transmission link, and then transmits it to data shard 1; data shard 6 establishes a data transmission session corresponding to its data to be transmitted, and the data transmission session transmits the data of data shard 6 to the shared transmission link, transmits it to the corresponding data transmission session through the shared transmission link, and then transmits it to data shard 1.

[0069] It can be seen that when there is only one shared transmission link between data transmission node 2 and data transmission node 1, and there is only one shared transmission link between data transmission node 3 and data transmission node 1, the number of shared transmission links C created between the data transmission nodes can be expressed as: C=M(N-1), where M is the number of shared transmission links between two data transmission nodes. Therefore, when new data to be transmitted is obtained, there is no need to create a new transmission link. Data transmission is performed only through the shared transmission link created between the data transmission nodes, and the data shards establish a connection relationship with the shared transmission link through the data transmission session.

[0070] For example, if the database cluster size is 100 (the number of data transmission nodes is 100, that is, N=100), the number of data shards on each data transmission node is 2 (that is, P=2), and the number of data to be transmitted is 200 (that is, S=200). The number of transmission links in the related technology is , which exceeds the system port number range (65535) and cannot support large-scale database clusters and high concurrent tasks. However, when the data transmission method of the present invention is used, only one shared transmission link is constructed between two data transmission nodes. The number of constructed shared transmission links is C=N–1=99, which is much smaller than 79600, and there is no need to create them frequently, which reduces the system burden and improves the stability of the system.

[0071] According to an embodiment of the present invention, the shared transmission links include multiple links; based on the shared transmission links and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, including: determining the number of shared transmission links between any two data transmission nodes; based on the number of shared transmission links, determining a target shared transmission link from multiple shared transmission links; based on the target shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the target transmission link and the data transmission session.

[0072] According to an embodiment of the present invention, the number of shared transmission links between any two data transmission nodes may be one or more.

[0073] According to the embodiment of the present invention, since a data transmission session only establishes a mapping relationship with one shared transmission link, one shared transmission link among multiple shared transmission links may be confirmed as a target shared transmission link.

[0074] According to an embodiment of the present invention, the data transmission node transmits the data to be transmitted to the target shared transmission link through a data transmission session. The target shared transmission link transmits the data to be transmitted to another data transmission node and transmits it to the corresponding data slice through the data transmission session.

[0075] According to an embodiment of the present invention, when there is only one shared transmission link between any two data transmission nodes, the burden on the system is minimal. For long-distance transmission in different places, in order to avoid network instability, a shared transmission link may be affected by network fluctuations, resulting in reduced transmission efficiency and data transmission congestion. Therefore, the shared transmission link between any two data transmission nodes can preferably be set to multiple links within a certain value range.

[0076] Figure 5 A performance display diagram of a shared transmission link according to an embodiment of the present invention is shown; Figure 6 A performance display diagram of multiple shared transmission links according to an embodiment of the present invention is shown.

[0077] like Figure 5 As shown, it can be seen from the performance display diagram 500 that when there is only one shared transmission link, as the network delay increases, the transmission rate decreases. Under a normal network, the transmission rate is about 500Mbps. When the network delay is 10ms, the transmission rate is about 200Mbps. When the network delay is 20ms, the transmission rate is about 100Mbps.

[0078] When there are multiple shared transmission connections, it can avoid the problem of transmission delay or blockage caused by network fluctuations on a shared transmission link. Figure 6 As shown in the performance display diagram 600, under normal network conditions, the transmission rate is about 500 Mbps. As the amount of data to be transmitted increases, when the network delay is 10 ms and 20 ms, it also approaches 500 Mbps. Therefore, compared with one shared transmission link, multiple shared transmission links are less affected by network fluctuations.

[0079] According to an embodiment of the present invention, a target shared transmission link is determined among multiple shared transmission links based on the number of shared transmission links, including: when the number of shared transmission links reaches a first preset number threshold, determining the number of data transmission sessions corresponding to each shared transmission link; and determining the target shared transmission link among the multiple shared transmission links, wherein the number of data transmission sessions corresponding to the target shared transmission link is the smallest among the number of data transmission sessions corresponding to each shared transmission link.

[0080] According to an embodiment of the present invention, the first preset number threshold may be a maximum number of shared transmission links between two data transmission nodes.

[0081] According to an embodiment of the present invention, when the number of shared transmission links reaches a first preset number threshold, there is no need to construct a new shared transmission link, and a target shared transmission connection is selected from the current multiple shared transmission links.

[0082] According to an embodiment of the present invention, each shared transmission link corresponds to a different number of data transmission sessions, and it is necessary to determine a target shared transmission link from multiple shared transmission links based on the number of data transmission sessions to ensure that there is no situation where the number of data transmission sessions corresponding to a certain shared transmission link is too many or too few.

[0083] According to an embodiment of the present invention, a shared transmission link with the least number of data transmission sessions is used as a target shared transmission link. For example, there are three shared transmission links between two data transmission nodes, among which the number of data transmission sessions corresponding to shared transmission connection A is 3, the number of data transmission sessions corresponding to shared transmission link B is 5, and the number of data transmission sessions corresponding to shared transmission link C is 2. Then, a mapping relationship is established between the new data transmission session and the shared transmission link C.

[0084] According to an embodiment of the present invention, if multiple data transmission sessions are obtained at the same time, the number of data sessions corresponding to each shared transmission link is sorted, and multiple data transmission sessions are allocated to at least one target shared transmission link according to the sorting. For example, the order of three shared transmission links between two data transmission nodes can be: shared transmission link C, shared transmission connection A, and shared transmission link B. If two data transmission sessions are obtained, they can be randomly allocated to shared transmission link C and shared transmission connection A. If four data transmission sessions are obtained (i.e., the number of shared connections is exceeded), shared transmission link C is randomly allocated to two data transmission sessions, shared transmission connection A is randomly allocated to one data transmission session, and shared transmission link B is randomly allocated to one data transmission session.

[0085] For another example, allocation may be performed based on the ranking position, and multiple data transmission sessions acquired at the same time may be randomly allocated to the shared transmission links corresponding to the first two ranked ones (ie, shared transmission link C, shared transmission connection A).

[0086] According to an embodiment of the present invention, by determining a target shared transmission link based on the number of data transmission sessions corresponding to each shared transmission link, it is avoided that there are too many or too few data transmission sessions corresponding to a certain shared transmission link.

[0087] According to an embodiment of the present invention, determining a target transmission link from a plurality of shared transmission links according to the number of shared transmission links also includes: when the number of shared transmission links does not reach a first preset number threshold, creating a new shared transmission link, and using the new shared transmission link as the target shared transmission link.

[0088] According to an embodiment of the present invention, when the number of shared transmission links does not reach a first preset number threshold, a new shared transmission link is created between two transmission nodes. Since the new shared transmission link does not have a corresponding data transmission session, the new shared transmission link is used as the target shared transmission link, and a mapping relationship is established between the acquired new data transmission session and the new shared transmission link.

[0089] According to an embodiment of the present invention, based on a shared transmission link and a data transmission session, data to be transmitted is transmitted between any two data transmission nodes, and also includes: obtaining emergency identification information of the data to be transmitted, the emergency identification information characterizing the urgency of transmission of the data to be transmitted; and performing data transmission on the data to be transmitted according to the emergency identification information.

[0090] According to an embodiment of the present invention, the emergency identification information represents the urgency of the transmission of the data to be transmitted, wherein the emergency identification information may be expressed in the form of a numerical value. The task of the data to be transmitted may automatically set or manually set a numerical value as the emergency identification information of the data to be transmitted when it is acquired.

[0091] According to an embodiment of the present invention, the emergency identification information can be divided into emergency priority, high priority, medium priority and low priority according to the numerical range corresponding to the emergency identification information. For example, emergency identification information with a value greater than or equal to 10 is an emergency priority, emergency identification information with a value in the range of 7 to 9 is a high priority, emergency identification information with a value in the range of 4 to 6 is a medium priority, and emergency identification information with a value in the range of 1 to 3 is a low priority.

[0092] According to an embodiment of the present invention, data transmission of data to be transmitted may be performed according to the emergency identification information, and the data to be transmitted may be queued according to the emergency priority, high priority, medium priority and low priority. The queuing strategy may be to first send all data to be transmitted corresponding to the emergency priority, then send the data to be transmitted corresponding to the high priority of the first preset value, then send the data to be transmitted corresponding to the medium priority of the second preset value, and then send the data to be transmitted corresponding to the low priority of the third preset value. For example, after all the data to be transmitted corresponding to the emergency priority are sent, after every three data to be transmitted corresponding to the high priority are sent, two data to be transmitted corresponding to the medium priority are sent, and then one data to be transmitted corresponding to the low priority is sent.

[0093] According to an embodiment of the present invention, the value corresponding to the emergency identification information can be changed dynamically. When new data to be transmitted is obtained, the emergency identification information corresponding to the data to be transmitted in the current queue is adjusted. For example, the value corresponding to the emergency identification information of the data to be transmitted A in the current queue is 3. When new data to be transmitted B is obtained, the value corresponding to the emergency identification information of the data to be transmitted A is adjusted to 4. After the emergency identification information corresponding to the data to be transmitted in the current queue is adjusted, the data to be transmitted A in the current queue and the new data to be transmitted B are re-prioritized until new data to be transmitted is obtained.

[0094] Alternatively, after receiving the adjustment instruction, the urgency identification information of the data to be transmitted in the current queue may be adjusted.

[0095] According to an embodiment of the present invention, by setting emergency identification information for the data to be transmitted and dynamically adjusting it, problems such as data transmission errors caused by congestion of multiple data to be transmitted can be avoided, so as to end the data transmission session as soon as possible and release resources.

[0096] According to an embodiment of the present invention, each shared transmission link includes a receiving end and a sending end; the data to be transmitted includes multiple data packets; the above method also includes: determining the preset number of received data in the receiving window of the receiving end; determining the number of data packets corresponding to the receiving end; when the number of data packets and the preset number of received data in the receiving window do not meet the preset conditions, delaying the transmission of the data transmission session corresponding to the data packet.

[0097] According to an embodiment of the present invention, the window size of the receiving window of the receiving end is fixed and is used to receive a continuous data stream.

[0098] According to an embodiment of the present invention, the sending end can simultaneously send the to-be-transmitted data corresponding to a plurality of data transmission sessions to the receiving end via a shared transmission link.

[0099] According to an embodiment of the present invention, the preset condition may be the sum of the numbers of data packets transmitted to the receiving end by multiple data transmission sessions, which sum does not exceed the preset number of received data in the receiving window of the receiving end.

[0100] According to an embodiment of the present invention, when the receiving end determines that the number of data packets and the preset number of received data in the receiving window do not meet the preset conditions, a heartbeat packet is sent to the corresponding sending end. The heartbeat packet includes the window size of the receiving window of the receiving end and the number of data packets corresponding to the receiving end, which can be specifically the number of data packets corresponding to each data transmission session when there are multiple data transmission sessions for data transmission.

[0101] According to an embodiment of the present invention, after receiving a heartbeat packet, the sending end determines the data transmission session with the largest number of corresponding data packets among the number of data packets corresponding to multiple data transmission sessions, and performs delay processing on the data transmission session. For example, data transmission session A, data transmission session B, and data transmission session C simultaneously perform data transmission through a shared transmission link, wherein the receiving window interface of the receiving end has received 10 data packets a, 20 data packets b, and 5 data packets c, a total of 35 data packets, exceeding the preset number of received data of 30 in the receiving window. At this time, the receiving end sends a heartbeat packet to the sending end, and the sending end analyzes the heartbeat packet and determines that data packet b accounts for the largest proportion in the total number of all data packets corresponding to the receiving end, and the sending end performs delay processing on the data transmission session B corresponding to data packet b.

[0102] According to an embodiment of the present invention, delayed transmission processing may be delaying processing of data packets that have not yet been sent to the receiving end in a data transmission session, may be extending the transmission time interval between any two data packets; or may be blocking some data packets at the sending end.

[0103] According to an embodiment of the present invention, the delayed transmission process may also be to adjust the emergency identification information of the data to be transmitted corresponding to the current transmitting end. Specifically, when the transmitting end receives a heartbeat packet sent by the corresponding receiving end, an adjustment instruction is issued, and based on the adjustment instruction, the value of the emergency identification information of the data to be transmitted corresponding to the current transmitting end is adjusted downward.

[0104] For example, the above-mentioned data transmission session B has transmitted 20 data packets b, and the remaining 30 data packets b have not been transmitted. Delay processing is performed on the remaining 30 data packets b, which can be done by expanding the transmission interval between each data packet b in the remaining 30 data packets b, or by reducing the value 4 corresponding to the emergency identification information of the data packet b to 3. According to the adjusted emergency identification information of the data packet b, all the current data to be transmitted at the sending end are re-prioritized, and data transmission is performed according to the new priority sorting.

[0105] According to the embodiment of the present invention, by delaying the transmission of the to-be-transmitted data corresponding to the data transmission session, the problem of congestion at the receiving end is avoided, the data transmission efficiency is improved, and the data transmission is made more flexible.

[0106] According to an embodiment of the present invention, the above method also includes: before the data to be transmitted is transmitted, using the sending end of the shared transmission link to compress the data to be transmitted; after the data to be transmitted is transmitted, using the receiving end of the shared transmission link to decompress the data to be transmitted.

[0107] According to an embodiment of the present invention, the data to be transmitted between any two data transmission nodes are mostly data with similar column structures and have a high compression ratio. Therefore, before data transmission, the data to be transmitted is compressed, which can reduce the amount of data occupied by the data to be transmitted in the shared transmission link, thereby reducing the transmission burden of the shared transmission link.

[0108] According to an embodiment of the present invention, after the data to be transmitted is transmitted to the receiving end, the data to be transmitted is decompressed.

[0109] According to an embodiment of the present invention, the method further includes: performing status analysis on the data to be transmitted to obtain an analysis result; and when the analysis result indicates that the status is abnormal, clearing the data transmission session corresponding to the data to be transmitted.

[0110] According to an embodiment of the present invention, the status analysis of the data to be transmitted may be to analyze the format of the data to be transmitted. If the format of the data to be transmitted does not meet a preset format condition, the data transmission session corresponding to the data to be transmitted is cleared.

[0111] Alternatively, the duration information of the data to be transmitted is recorded, the duration information including the waiting time of the data to be transmitted in the queue and the transmission time during the data transmission process. If the duration information of the data to be transmitted exceeds the preset duration, the data transmission session corresponding to the data to be transmitted is cleared.

[0112] Alternatively, all data transmission sessions may be traversed to determine whether each data transmission session is complete, for example, whether there is a data transmission session that has a link relationship established with only one data slice, or whether there is a data transmission session that has not established a mapping relationship with a shared transmission link. If so, clear the data transmission session corresponding to the data to be transmitted.

[0113] According to the embodiment of the present invention, by performing status analysis on the data to be transmitted, session resources of the data to be transmitted in an abnormal state are released, thereby avoiding the impact of the abnormal data to be transmitted on the system and reducing the system burden.

[0114] Figure 7A schematic diagram of a data transmission method according to another embodiment of the present invention is shown.

[0115] Data transmission method 700 Figure 7 As shown, perform operation S701 to obtain data to be transmitted. Perform operation S702 to create a data transmission session corresponding to the data to be transmitted obtained. Perform operation S703 to determine whether the number of shared transmission links between two data transmission nodes corresponding to the data to be transmitted reaches a first preset number threshold. If not, perform operation S704 to create a new shared transmission link, and then perform operation S706 to determine the new shared transmission link as the target shared transmission link; if satisfied, perform operation S705 to determine the number of transmission sessions corresponding to the shared transmission link, and determine the shared transmission link with the least number of data transmission sessions among the number of data transmission sessions corresponding to the multiple shared transmission links among the multiple shared transmission links. Perform operation S706 to use the shared transmission link with the least number of corresponding data transmission sessions determined in operation S705 as the target shared transmission link. After determining the shared transmission link, perform operation S707 to determine the emergency identification information of the data to be transmitted, prioritize the data to be transmitted according to the emergency identification information, and perform data transmission on the data to be transmitted according to the priority queuing. After the data transmission is completed, operation S708 is performed to clear the transmission data session corresponding to the data to be transmitted.

[0116] Based on the above data transmission method, the present invention also provides a data transmission device. Figure 8 The device is described in detail.

[0117] Figure 8 A structural block diagram of a data transmission device according to an embodiment of the present invention is shown.

[0118] like Figure 8 As shown, the data transmission device 800 of this embodiment includes an acquisition module 810 , an establishment module 820 , a creation module 830 , a transmission module 840 and a first clearing module 850 .

[0119] The acquisition module 810 is used to acquire the data to be transmitted. In one embodiment, the acquisition module 810 can be used to perform the operation S210 described above, which will not be described in detail here.

[0120] The establishing module 820 is used to establish a shared transmission link between any two of the data transmission nodes. In one embodiment, the establishing module 820 can be used to perform the operation S220 described above, which will not be described in detail here.

[0121] The creation module 830 is used to call the session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted has a one-to-one correspondence with the data transmission session, and the data transmission session is used to receive and send the data to be transmitted. In one embodiment, the creation module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0122] The transmission module 840 is used to transmit the data to be transmitted between the two data transmission nodes according to the shared transmission link and the data transmission session, wherein there is a mapping relationship between the shared transmission link and the data transmission session. In one embodiment, the transmission module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0123] The first clearing module 850 is used to clear the transmission session corresponding to the data to be transmitted after the data to be transmitted is completed. In one embodiment, the clearing module 850 can be used to perform the operation S250 described above, which will not be described in detail here.

[0124] According to an embodiment of the present invention, data to be transmitted is obtained; a shared transmission link is established between any two data transmission nodes; a session manager component is called to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; according to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; after the data to be transmitted is completed, the transmission session corresponding to the data to be transmitted is cleared. Due to the use of a technical means of using a shared transmission link and a data transmission session to transmit the data to be transmitted between any two data transmission nodes, the problem of frequent establishment of communication links and too many generated communication links, which results in excessive burden on the system, is avoided, and the technical effect of reducing the system burden and supporting high-concurrency tasks of large-scale database clusters is achieved.

[0125] According to an embodiment of the present invention, any multiple modules among the acquisition module 810, the establishment module 820, the creation module 830, the transmission module 840 and the first clearing module 850 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 810, the establishment module 820, the creation module 830, the transmission module 840 and the first clearing module 850 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 810 , the establishment module 820 , the creation module 830 , the transmission module 840 and the first clearing module 850 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0126] According to an embodiment of the present invention, the transmission module 840 includes a first determination submodule, a second determination submodule and a first transmission submodule.

[0127] The first determining submodule is configured to determine the number of the shared transmission links between any two of the data transmission nodes.

[0128] The second determining submodule is configured to determine a target shared transmission link from among the multiple shared transmission links according to the number of the shared transmission links.

[0129] The first transmission submodule is used to transmit the data to be transmitted between any two data transmission nodes according to the target shared transmission link and the data transmission session, wherein there is a mapping relationship between the target transmission link and the data transmission session.

[0130] According to an embodiment of the present invention, the second determining submodule includes a first determining unit and a second determining unit.

[0131] The first determining unit is configured to determine the number of the data transmission sessions corresponding to each of the shared transmission links when the number of the shared transmission links reaches a first preset number threshold.

[0132] The second determining unit is configured to determine the target shared transmission link among the multiple shared transmission links, wherein the number of the data transmission sessions corresponding to the target shared transmission link is the smallest among the numbers of the data transmission sessions corresponding to each of the shared transmission links.

[0133] According to an embodiment of the present invention, the second determining submodule further includes a creating unit.

[0134] The creating unit is configured to create a new shared transmission link if the number of the shared transmission links does not reach a first preset number threshold, and use the new shared transmission link as the target shared transmission link.

[0135] According to an embodiment of the present invention, the transmission module 840 further includes an acquisition submodule and a second transmission submodule.

[0136] The acquisition submodule is used to acquire the emergency identification information of the data to be transmitted, wherein the emergency identification information represents the urgency of the transmission of the data to be transmitted.

[0137] The second transmission submodule is used to perform data transmission on the data to be transmitted according to the emergency identification information.

[0138] According to an embodiment of the present invention, the data transmission device 800 further includes a first determination module, a second determination module and a processing module.

[0139] The first determining module is used to determine the preset number of received data in the receiving window of the receiving end.

[0140] The second determining module is used to determine the number of the data packets corresponding to the receiving end.

[0141] The processing module is used to perform delayed transmission processing on the data transmission session corresponding to the data packet when the number of the data packets and the preset number of received data in the receiving window do not meet a preset condition.

[0142] According to an embodiment of the present invention, the data transmission device 800 further includes a compression module and a decompression module.

[0143] The compression module is used to compress the data to be transmitted by using the sending end of the shared transmission link before transmitting the data to be transmitted.

[0144] The decompression module is used to decompress the data to be transmitted by using the receiving end of the shared transmission link after the data to be transmitted is completed.

[0145] According to an embodiment of the present invention, the data transmission device 800 further includes an analysis module and a second clearing module.

[0146] The analysis module is used to perform status analysis on the data to be transmitted to obtain analysis results.

[0147] The second clearing module is used to clear the data transmission session corresponding to the data to be transmitted when the analysis result indicates that the state is abnormal.

[0148] Fig. 9 A block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention is shown.

[0149] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0150] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0151] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0152] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the data transmission method according to the embodiment of the present invention is implemented.

[0153] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0154] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data transmission method provided by the embodiment of the present invention.

[0155] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0157] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0158] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0159] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in a variety of ways, even if such combinations and / or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in a variety of ways without departing from the spirit and teachings of the present invention. All of these combinations and / or couplings fall within the scope of the present invention.

[0161] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data transmission method, applied to a database cluster, wherein the database cluster includes a plurality of data transmission nodes, each of which has a plurality of data shards, characterized in that: The method comprises: Get the data to be transmitted; Establishing a shared transmission link between any two of the data transmission nodes; Calling a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; According to the shared transmission link and the data transmission session, the data to be transmitted is transmitted between the any two data transmission nodes, wherein there is a mapping relationship between the shared transmission link and the data transmission session; After the data transmission to be transmitted is completed, clearing the data transmission session corresponding to the data to be transmitted; Among them, the data transmission session is a link for data transmission between the data slice on the data transmission node and the shared transmission link corresponding to the data transmission node, which is used to enable the data slice on the data transmission node to receive and send data to be transmitted.

2. The method according to claim 1, characterized in that The shared transmission link includes a plurality of links; The transmitting the data to be transmitted between the any two data transmission nodes according to the shared transmission link and the data transmission session includes: Determining the number of the shared transmission links between any two of the data transmission nodes; Screening the shared transmission links according to the number of the shared transmission links, and determining a target shared transmission link from the multiple shared transmission links; According to the target shared transmission link and the data transmission session, the data to be transmitted is transmitted between the any two data transmission nodes through the target shared transmission link and the data transmission session, wherein there is a mapping relationship between the target shared transmission link and the data transmission session.

3. The method according to claim 2, characterized in that The determining, according to the number of the shared transmission links, a target shared transmission link from the plurality of shared transmission links comprises: When the number of the shared transmission links reaches a first preset number threshold, determining the number of the data transmission sessions corresponding to each of the shared transmission links; The target shared transmission link is determined among the plurality of shared transmission links, wherein the number of the data transmission sessions corresponding to the target shared transmission link is the smallest among the numbers of the data transmission sessions corresponding to each of the shared transmission links.

4. The method according to claim 3, characterized in that The step of determining a target shared transmission link from a plurality of shared transmission links according to the number of the shared transmission links further includes: When the number of the shared transmission links does not reach a first preset number threshold, A new shared transmission link is created, and the new shared transmission link is used as the target shared transmission link.

5. The method according to claim 1, characterized in that The step of transmitting the to-be-transmitted data between the any two data transmission nodes according to the shared transmission link and the data transmission session further includes: Acquire emergency identification information of the data to be transmitted, where the emergency identification information indicates the urgency of transmission of the data to be transmitted; The data to be transmitted is transmitted according to the emergency identification information.

6. The method according to claim 1, characterized in that Each of the shared transmission links includes a receiving end and a transmitting end; the data to be transmitted includes a plurality of data packets; The method further comprises: Determining a preset number of received data in a receiving window of the receiving end; Determine the number of the data packets corresponding to the receiving end; When the number of the data packets and the preset number of received data in the receiving window do not satisfy a preset condition, a delayed transmission process is performed on the data transmission session corresponding to the data packet.

7. The method according to claim 6, characterized in that The method further comprises: Before transmitting the data to be transmitted, compressing the data to be transmitted using the sending end of the shared transmission link; After the transmission of the data to be transmitted is completed, the receiving end of the shared transmission link is used to decompress the data to be transmitted.

8. The method according to claim 1, characterized in that The method further comprises: Performing status analysis on the data to be transmitted to obtain analysis results; When the analysis result indicates an abnormal state, the data transmission session corresponding to the data to be transmitted is cleared.

9. A data transmission device, applied to a database cluster, wherein the database cluster comprises a plurality of data transmission nodes, each of which has a plurality of data shards, characterized in that: The device comprises: An acquisition module, used for acquiring data to be transmitted; An establishing module, used to establish a shared transmission link between any two of the data transmission nodes; A creation module, used to call a session manager component to create a data transmission session corresponding to the data to be transmitted, wherein the data to be transmitted and the data transmission session have a one-to-one correspondence, and the data transmission session is used to receive and send the data to be transmitted; a transmission module, configured to transmit the data to be transmitted between the arbitrary two data transmission nodes according to the shared transmission link and the data transmission session, wherein a mapping relationship exists between the shared transmission link and the data transmission session; and A first clearing module, configured to clear the data transmission session corresponding to the data to be transmitted after the data to be transmitted is completed; Among them, the data transmission session is a link for data transmission between the data slice on the data transmission node and the shared transmission link corresponding to the data transmission node, which is used to enable the data slice on the data transmission node to receive and send data to be transmitted.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 8.

Images