A method to improve concurrent access performance of MPP data warehouse based on high availability connection pool

Abstract

The present invention relates to the technical field of high-availability connection pools, and in particular to a method for improving the access concurrency performance of an MPP data warehouse based on a high-availability connection pool, comprising: a central control module determines whether a running connection meets a preset standard according to an average response time; determines whether to adjust a maximum allowable value of flow transmission in the running connection or whether to call several new connections to build several data channels between the user end and the secondary data warehouse according to the average response time, and corrects the comprehensive flow transmission value ratio in each running connection; the central control module determines whether to add a new connection to the connection pool according to the total number of connections to be used, and performs a secondary correction on the comprehensive flow transmission value ratio when the number of connections in the connection pool exceeds a preset number of connections, and handles the situation where multiple users access a single database at the same time, so as to improve the access concurrency performance of the MPP data warehouse.

Description

Technical Field

[0001] The present invention relates to the technical field of high-availability connection pools, and in particular to a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool. Background Art

[0002] The customer base of mobile services is very large. When processing user access to mobile user-related systems, there is the problem of handling a large number of customers accessing at the same time. When the number of users grows to a certain level, it will be unable to withstand the pressure of concurrent user access. Users go to the connection pool to obtain an exclusive connection for multiplexing as needed. The present invention processes electronic digital data and plans the construction of data channels for the secondary data warehouse.

[0003] Chinese patent publication number: CN111597173A, discloses a data warehouse system, including a data mobile unit, a data storage unit and a data access unit, which can provide data storage and access services to various internal business systems of Internet companies, send business data related to the business system to the data warehouse system, and the data warehouse system will perform unified cleaning and storage, and perform unified authentication processing when each business system accesses these data. It can be seen that the following problems exist: the situation where multiple users access a single database at the same time is not handled, and it is very easy to cause network crashes due to multiple users accessing a single database at the same time. Summary of the invention

[0004] To this end, the present invention provides a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool, so as to overcome the problem in the prior art that network crashes are caused by multiple users accessing a single database at the same time.

[0005] To achieve the above object, the present invention provides a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool, comprising:

[0006] When a single terminal in the user end issues an access request, the central control module calls a single connection from the connection pool according to the information in the access request to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse; the central control module marks the connection as a running connection when the connection completes the construction of the data channel between the user end and the secondary data warehouse;

[0007] When the central control module completes the construction of the data channel between the user terminal and the secondary data warehouse, the central control module sequentially records the access time node when the terminal issues an access request, the output time node when the secondary data warehouse outputs data, and the receiving time node when the user terminal receives data. The central control module obtains the response time of the running connection for this access request based on the access time node and the output time node, and obtains the transmission time of the running connection for this access request based on the output time node and the receiving time node.

[0008] The central control module counts the response time and transmission time of each access request of a single connection within a preset time to obtain the average response time and average transmission time of the running connection between the user terminal and the secondary data warehouse within the preset time, and determines whether the running connection meets the preset standard according to the average response time;

[0009] When determining that a single connection does not meet the preset standard, the central control module determines whether to adjust the maximum allowed value of the flow transmission in the running connection or whether to call several new connections to build several data channels between the user terminal and the secondary data warehouse according to the average response time, and after completing the determination, determines whether to correct the comprehensive flow transmission value ratio in each running connection according to the comprehensive transmission time of each running connection;

[0010] When determining that a new connection needs to be called, the central control module determines whether to add a new connection to the connection pool based on the total number of connections to be used, and determines whether to perform a secondary correction on the comprehensive traffic transmission value ratio of each running connection when the number of connections in the connection pool exceeds the preset connection number standard.

[0011] Further, the central control module calculates the average response time Ta0 of the running connection connected to a single secondary data warehouse within the preset time under the first preset condition, and sets Wherein, n is the number of access requests transmitted by the running connection within the preset time period, Tai is the i-th response time period corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset time period, i=1, 2, 3, ..., n, ei is the i-th time period correction coefficient for the i-th response time period;

[0012] The first preset condition is that the central control module determines that the running time of a single running connection reaches a preset time.

[0013] Furthermore, the central control module determines a determination method for the running connection according to the average response time Ta0 under the first preset condition, wherein:

[0014] The first determination method is that the central control module determines that the running connection of the running connection meets the preset standard; the first determination method satisfies that the average response time Ta0 of the running connection in the preset time is less than or equal to the first preset average response time Ta1 preset in the central control module;

[0015] The second determination method is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module adjusts the maximum allowable value of traffic transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1; the second determination method satisfies that the average response time Ta0 is greater than the first preset response time Ta1 and Ta0 is less than the second preset response time Ta2 preset by the central control module;

[0016] The third judgment method is that the central control module determines that the running connection of the secondary data warehouse connection does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse; the third judgment method satisfies that the average response time Ta0 is greater than the second preset average response time Ta2.

[0017] Further, the central control module determines the assignment method of each duration correction coefficient in turn according to the instantaneous data transmission volume of the running connection at the time node when the running connection transmits the access request to each terminal to the secondary data warehouse within the preset duration under the second preset condition, wherein, for the i-th duration correction coefficient ei,

[0018] The first assignment method is that the central control module assigns ei a value of 1.15; the first assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is less than or equal to the first preset instantaneous data transmission volume Qa preset in the central control module;

[0019] The second assignment method is that the central control module assigns ei a value of 1; the second assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the first preset instantaneous data transmission volume Qa preset in the central control module and Qi is less than or equal to the second preset instantaneous data transmission volume Qb preset in the central control module;

[0020] The third assignment method is that the central control module assigns ei a value of 0.95; the third assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the second preset instantaneous data transmission volume Qb;

[0021] The second preset condition is that the central control module calculates the average response time of the running connection within the preset time.

[0022] Further, the central control module calculates the difference △Tax between the average response time Ta0 and the first preset average response time Ta1 under the third preset condition, and sets △Tax=Ta1-Ta0. The central control module records △Tax as the first-level response time difference of the running connection and determines the adjustment method for the maximum allowable value of the flow transmission of the running connection according to △Tax, wherein,

[0023] The first adjustment method is that the central control module uses a preset first preset adjustment coefficient β1 to adjust the maximum allowable value C of the flow transmission to a corresponding value, and 1<β1<1.2 is set; the first adjustment method satisfies that the first-level difference △Tax is less than or equal to the first preset first-level response time difference △Tax1 set by the central control module;

[0024] The second adjustment method is that the central control module uses the preset second preset adjustment coefficient β2 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β1<β2<1.3; the second adjustment method satisfies that the first-level difference △Tax is greater than the first preset first-level response time difference △Tax1 and △Tax is less than or equal to the second preset first-level response time difference △Tax2 set by the central control module;

[0025] The third adjustment method is that the central control module uses the preset third preset adjustment coefficient β3 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β2<β3<1.5; the third adjustment method satisfies that the first-level difference △Tax is greater than the second preset first-level response time difference △Tax2;

[0026] The central control module records the adjusted maximum allowed value of the flow transmission for the running connection as C', and sets C'=C×βj, where j=1, 2, 3;

[0027] The third preset condition is that the central control module adjusts the maximum allowed value of flow transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1.

[0028] Further, the central control module detects the average transmission time of the running connection between the user terminal and the secondary data warehouse in the preset time length under the fourth preset condition, and for the kth running connection connected to the user terminal and the secondary data warehouse respectively, the central control module records the average transmission time of the running connection in the preset time length as Tbk0, wherein k=1, 2, 3, ..., m, m is the total number of running connections connected to the user terminal and the secondary data warehouse, and is set Wherein, Tbi is the i-th response duration corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset duration;

[0029] The fourth preset condition is that the central control module adjusts the maximum allowed value of the flow transmission for the running connection to C'.

[0030] Further, the central control module sequentially counts the average transmission time of each of the running connections within the preset time period to transmit data under the fourth preset condition to obtain the comprehensive transmission time Tb of each running connection within the preset time period, and determines the correction method for the comprehensive flow transmission value ratio in each running connection according to Tb, wherein the setting

[0031] The first correction method is that the central control module sets the comprehensive flow transmission value ratio for each of the running connections to a preset transmission value ratio B0, where B0 is the ratio of the maximum data transmission value allowed in a single running connection to the maximum data transmission value that the running connection can withstand; the first correction method satisfies that the comprehensive transmission time Tb is less than or equal to the first preset comprehensive transmission time Tb1 set in the central control module;

[0032] The second correction method is that the central control module uses the first correction coefficient e1 to set the comprehensive flow transmission value ratio B1 for each running connection, 1.2<e1<1.4, to B1=B0×e1; the second correction method satisfies that the comprehensive transmission time Tb is greater than the first preset comprehensive transmission time Tb1 and is less than or equal to the second preset comprehensive transmission time Tb2 set in the central control module;

[0033] The third correction method is that the central control module uses the second correction coefficient to set the comprehensive flow transmission value ratio B2 for each running connection, e1<e2<1.5, and sets B2=B0×e2; the third correction method satisfies that the comprehensive transmission time Tb is greater than the second preset comprehensive transmission time Tb2.

[0034] Further, the central control module calculates the difference △Tay between the average response time Ta0 and the second preset average response time Ta2 under the fifth preset condition, and sets △Tay=Ta0-Ta2. The central control module records △Tay as the secondary response time difference of the running connection and determines the calling method for the number of connections connected to the secondary data warehouse according to △Tay, wherein,

[0035] The first calling method is that the central control module uses the preset first adjustment coefficient γ1 to change the number of connections connected to the secondary data warehouse to the first number M1, 1.3<γ1<1.5, setting Wherein, M is the initial number of connections connected to the secondary data warehouse; the first calling mode satisfies that the secondary difference △Tay is less than or equal to the preset secondary response time difference △Tay0 set by the central control module;

[0036] The second calling method is that the central control module uses the preset second adjustment coefficient γ2 to change the number of connections connected to the secondary data warehouse to the first number M2, γ1<γ2<1.6, and sets The second calling mode satisfies that the second-level difference ΔTay is greater than the preset second-level response time difference ΔTay0;

[0037] The fifth preset condition is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse.

[0038] Further, the central control module determines a secondary correction method for the comprehensive flow transmission value ratio of each running connection according to the difference between the total number of connections after the addition and the preset connection number standard under the sixth preset condition, wherein:

[0039] The first secondary correction method is that the central control module uses the corrected comprehensive flow transmission value ratio Bz, where z=1, 2; the first secondary correction method satisfies the quantity difference being less than or equal to the quantity difference standard preset by the central control module;

[0040] The second secondary correction method is to correct the corrected comprehensive flow transmission value ratio to Bz', setting Bz'=1.05×Bz; the second secondary correction method satisfies the quantity difference being greater than the quantity difference standard preset by the central control module;

[0041] The sixth preset condition is that the central control module determines that the number of newly added connections is greater than a preset connection number standard in the connection pool.

[0042] Furthermore, the central control module obtains corresponding keywords according to the access request sent by the terminal to connect the terminal with the corresponding secondary data warehouse using a connection to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse.

[0043] Compared with the prior art, the beneficial effects of the present invention lie in that the central control module determines whether the running connection meets the preset standard based on the average response time; determines whether to adjust the maximum allowed value of traffic transmission in the running connection or whether to call several new connections to build several data channels between the user end and the secondary data warehouse based on the average response time, and corrects the comprehensive traffic transmission value ratio in each running connection; the central control module determines whether to add a new connection to the connection pool based on the total number of connections to be used, and performs a secondary correction on the comprehensive traffic transmission value ratio when the number of connections in the connection pool exceeds the preset number of connections, and handles the situation where multiple users access a single database at the same time, so as to improve the access concurrency performance of the MPP data warehouse.

[0044] Furthermore, when the central control module determines that the running time of a single running connection reaches the preset time, it calculates the average response time Ta0 of the running connection connected to the single secondary data warehouse within the preset time according to the number of access requests transmitted by the running connection within the preset time and the i-th response time corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset time. While ensuring accurate and timely acquisition of the average response time of the running connection connected to the single secondary data warehouse within the preset time, the access concurrency performance of the MPP data warehouse is further improved.

[0045] Furthermore, when the central control module determines that the running time of a single running connection reaches the preset time, it determines whether the running connection meets the preset standard based on the average response time Ta0. When the central control module determines that the running connection does not meet the preset standard, it selects a corresponding method to adjust the maximum allowed value of the flow transmission of the running connection or calls a corresponding number of new connections to build a number of data channels between the user end and the secondary data warehouse. While ensuring the accurate determination of the working status of the running connection, it further determines the solution when the running connection does not meet the preset standard based on the average response time, thereby further improving the access concurrency performance of the MPP data warehouse.

[0046] Furthermore, when the central control module calculates the average response time of the running connection within the preset time length, the corresponding correction coefficient ei is selected according to the instantaneous data transmission volume of the running connection at the time node when the running connection transmits an access request to each terminal to the secondary data warehouse within the preset time length. While ensuring accurate acquisition of the average response time of the running connection connected to a single secondary data warehouse within the preset time length, the access concurrency performance of the MPP data warehouse is further improved.

[0047] Furthermore, when the average response time Ta0 is greater than the first preset response time Ta1 and Ta0 is less than the second preset response time Ta2 preset by the central control module, the central control module adjusts the maximum allowed value of traffic transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1, thereby further improving the access concurrency performance of the MPP data warehouse while ensuring timely response to the access request issued by the terminal.

[0048] Furthermore, the central control module adjusts the maximum allowed value of the traffic transmission for the running connection to C', and then calculates the average transmission time between the user terminal and the secondary data warehouse within the preset time period according to the i-th instantaneous data transmission volume Qi of the running connection, the first preset instantaneous data transmission volume Qa, and the i-th response time corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset time period. While ensuring the accurate acquisition of the average transmission time, the access concurrency performance of the MPP data warehouse is further improved.

[0049] Furthermore, the central control module calculates the comprehensive transmission time Tb of each running connection within the preset time according to the average transmission time of the running connection within the preset time, and corrects the comprehensive traffic transmission value ratio in each running connection to the corresponding value according to Tb. When it is found that the average transmission time is too high, the corresponding comprehensive traffic transmission value ratio is selected to further improve the access concurrency performance of the MPP data warehouse.

[0050] Furthermore, under the third determination method, the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user end and the secondary data warehouse, thereby ensuring that the response time is effectively reduced while further improving the access concurrency performance of the MPP data warehouse.

[0051] Furthermore, when the central control module determines that the number of newly added connections is greater than the preset connection number standard in the connection pool, it will perform a secondary correction on the comprehensive traffic transmission value ratio of each running connection based on the difference between the total number of newly added connections and the preset connection number standard. While ensuring that the normal data transmission between the user terminal and the secondary data warehouse is not affected by the excessive number of connections in the connection pool, the access concurrency performance of the MPP data warehouse is further improved.

[0052] Furthermore, the central control module obtains the corresponding keyword according to the access request sent by the terminal to connect the terminal with the corresponding secondary data warehouse using a connection to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse, while ensuring that the user is connected to the corresponding secondary database, further improving the access concurrency performance of the MPP data warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of the steps of a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to an embodiment of the present invention;

[0054] Figure 2 This is a flow chart of an embodiment of the present invention for determining a determination method for the running connection according to the average response time;

[0055] Figure 3 A flowchart of an embodiment of the present invention for determining an adjustment method for a maximum allowable value of flow transmission for the running connection according to a difference between an average response time and a first preset average response time;

[0056] Figure 4 This is a flowchart of an embodiment of the present invention for determining a calling method for the number of connections connected to the secondary data warehouse according to a difference between an average response time and a second preset average response time. DETAILED DESCRIPTION

[0057] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0059] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0060] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] See also Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, there are respectively a flowchart of the steps of a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool in an embodiment of the present invention, a flowchart of determining a determination method for the running connection according to an average response time, a flowchart of determining an adjustment method for a maximum allowable value of flow transmission for the running connection according to a difference between an average response time and a first preset average response time, and a flowchart of determining a calling method for the number of connections connected to the secondary data warehouse according to a difference between an average response time and a second preset average response time; a flowchart of the steps of a method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool in an embodiment of the present invention, comprising:

[0062] When a single terminal in the user end issues an access request, the central control module calls a single connection from the connection pool according to the information in the access request to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse; the central control module marks the connection as a running connection when the connection completes the construction of the data channel between the user end and the secondary data warehouse;

[0063] When the central control module completes the construction of the data channel between the user terminal and the secondary data warehouse, the central control module sequentially records the access time node when the terminal issues an access request, the output time node when the secondary data warehouse outputs data, and the receiving time node when the user terminal receives data. The central control module obtains the response time of the running connection for this access request based on the access time node and the output time node, and obtains the transmission time of the running connection for this access request based on the output time node and the receiving time node.

[0064] The central control module counts the response time and transmission time of each access request of a single connection within a preset time to obtain the average response time and average transmission time of the running connection between the user terminal and the secondary data warehouse within the preset time, and determines whether the running connection meets the preset standard according to the average response time;

[0065] When determining that a single connection does not meet the preset standard, the central control module determines whether to adjust the maximum allowed value of the flow transmission in the running connection or whether to call several new connections to build several data channels between the user terminal and the secondary data warehouse according to the average response time, and after completing the determination, determines whether to correct the comprehensive flow transmission value ratio in each running connection according to the comprehensive transmission time of each running connection;

[0066] When determining that a new connection needs to be called, the central control module determines whether to add a new connection to the connection pool based on the total number of connections to be used, and determines whether to perform a secondary correction on the comprehensive traffic transmission value ratio of each running connection when the number of connections in the connection pool exceeds the preset connection number standard.

[0067] Specifically, the central control module calculates the average response time Ta0 of the running connection connected to a single secondary data warehouse within the preset time under the first preset condition, and sets Wherein, n is the number of access requests transmitted by the running connection within the preset time length, Tai is the i-th response time length corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset time length, i=1, 2, 3, ..., n, ei is the i-th time length correction coefficient for the i-th response time length;

[0068] The first preset condition is that the central control module determines that the running time of a single running connection reaches a preset time.

[0069] Specifically, the central control module determines the determination method for the running connection according to the average response time Ta0 under the first preset condition, wherein:

[0070] The first determination method is that the central control module determines that the running connection of the running connection meets the preset standard; the first determination method satisfies that the average response time Ta0 of the running connection in the preset time is less than or equal to the first preset average response time Ta1 preset in the central control module;

[0071] The second determination method is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module adjusts the maximum allowable value of traffic transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1; the second determination method satisfies that the average response time Ta0 is greater than the first preset response time Ta1 and Ta0 is less than the second preset response time Ta2 preset by the central control module;

[0072] The third determination method is that the central control module determines that the running connection of the secondary data warehouse connection does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse; the third determination method satisfies that the average response time Ta0 is greater than the second preset average response time Ta2;

[0073] Among them, Ta1=120, Ta2=2048.

[0074] Specifically, the central control module determines the assignment method of each duration correction coefficient in turn according to the instantaneous data transmission volume of the running connection at the time node when the running connection transmits the access request to each terminal to the secondary data warehouse within the preset duration under the second preset condition, wherein, for the i-th duration correction coefficient ei,

[0075] The first assignment method is that the central control module assigns ei a value of 1.15; the first assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is less than or equal to the first preset instantaneous data transmission volume Qa preset in the central control module;

[0076] The second assignment method is that the central control module assigns ei a value of 1; the second assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the first preset instantaneous data transmission volume Qa preset in the central control module and Qi is less than or equal to the second preset instantaneous data transmission volume Qb preset in the central control module;

[0077] The third assignment method is that the central control module assigns ei a value of 0.95; the third assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the second preset instantaneous data transmission volume Qb;

[0078] The second preset condition is that the central control module calculates the average response time of the running connection within the preset time;

[0079] Among them, Qa=33, Qb=155.

[0080] Specifically, the central control module calculates the difference △Tax between the average response time Ta0 and the first preset average response time Ta1 under the third preset condition, and sets △Tax=Ta1-Ta0. The central control module records △Tax as the first-level response time difference of the running connection and determines the adjustment method for the maximum allowable value of the flow transmission of the running connection according to △Tax, wherein,

[0081] The first adjustment method is that the central control module uses a preset first preset adjustment coefficient β1 to adjust the maximum allowable value C of the flow transmission to a corresponding value, and 1<β1<1.2 is set; the first adjustment method satisfies that the first-level difference △Tax is less than or equal to the first preset first-level response time difference △Tax1 set by the central control module;

[0082] The second adjustment method is that the central control module uses the preset second preset adjustment coefficient β2 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β1<β2<1.3; the second adjustment method satisfies that the first-level difference △Tax is greater than the first preset first-level response time difference △Tax1 and △Tax is less than or equal to the second preset first-level response time difference △Tax2 set by the central control module;

[0083] The third adjustment method is that the central control module uses the preset third preset adjustment coefficient β3 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β2<β3<1.5; the third adjustment method satisfies that the first-level difference △Tax is greater than the second preset first-level response time difference △Tax2;

[0084] The central control module records the adjusted maximum allowed value of the flow transmission for the running connection as C', and sets C'=C×βj, where j=1, 2, 3;

[0085] The third preset condition is that the central control module adjusts the maximum allowed value of flow transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1;

[0086] Among them, β1=1.15, β2=1.26, β3=1.44, Tax1=276, Tax2=552.

[0087] Specifically, the central control module detects the average transmission time of the running connection between the user terminal and the secondary data warehouse in the preset time length under the fourth preset condition. For the kth running connection connected to the user terminal and the secondary data warehouse respectively, the central control module records the average transmission time of the running connection in the preset time length as Tbk0, where k=1, 2, 3, ..., m, m is the total number of running connections connected to the user terminal and the secondary data warehouse, and is set Wherein, Tbi is the i-th response duration corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset duration;

[0088] The fourth preset condition is that the central control module adjusts the maximum allowed value of the flow transmission for the running connection to C'.

[0089] Specifically, the central control module sequentially counts the average transmission time of each running connection within the preset time to transmit data under the fourth preset condition to obtain the comprehensive transmission time Tb of each running connection within the preset time, and determines the correction method for the comprehensive flow transmission value ratio in each running connection according to Tb, wherein the setting

[0090] The first correction method is that the central control module sets the comprehensive flow transmission value ratio for each of the running connections to a preset transmission value ratio B0, where B0 is the ratio of the maximum data transmission value allowed in a single running connection to the maximum data transmission value that the running connection can withstand; the first correction method satisfies that the comprehensive transmission time Tb is less than or equal to the first preset comprehensive transmission time Tb1 set in the central control module;

[0091] The second correction method is that the central control module uses the first correction coefficient e1 to set the comprehensive flow transmission value ratio B1 for each running connection, 1.2<e1<1.4, to B1=B0×e1; the second correction method satisfies that the comprehensive transmission time Tb is greater than the first preset comprehensive transmission time Tb1 and is less than or equal to the second preset comprehensive transmission time Tb2 set in the central control module;

[0092] The third correction method is that the central control module uses the second correction coefficient to set the comprehensive flow transmission value ratio B2 for each running connection, e1<e2<1.5, and sets B2=B0×e2; the third correction method satisfies that the comprehensive transmission time Tb is greater than the second preset comprehensive transmission time Tb2;

[0093] Among them, B0=0.61, e1=1.26, e2=1.45, Tb1=0.38, Tb2=1.2.

[0094] Specifically, the central control module calculates the difference △Tay between the average response time Ta0 and the second preset average response time Ta2 under the fifth preset condition, and sets △Tay=Ta0-Ta2. The central control module records △Tay as the secondary response time difference of the running connection and determines the calling method for the number of connections connected to the secondary data warehouse according to △Tay, wherein,

[0095] The first calling method is that the central control module uses the preset first adjustment coefficient γ1 to change the number of connections connected to the secondary data warehouse to the first number M1, 1.3<γ1<1.5, setting Wherein, M is the initial number of connections connected to the secondary data warehouse; the first calling mode satisfies that the secondary difference △Tay is less than or equal to the preset secondary response time difference △Tay0 set by the central control module;

[0096] The second calling method is that the central control module uses the preset second adjustment coefficient γ2 to change the number of connections connected to the secondary data warehouse to the first number M2, γ1<γ2<1.6, and sets The second calling mode satisfies that the second-level difference ΔTay is greater than the preset second-level response time difference ΔTay0;

[0097] The fifth preset condition is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse;

[0098] Among them, △Tay0=3084, γ1=1.22, γ2=1.52.

[0099] Specifically, the central control module determines a secondary correction method for the comprehensive flow transmission value ratio of each running connection according to the difference between the total number of newly added connections and the preset connection number standard under the sixth preset condition, wherein:

[0100] The first secondary correction method is that the central control module uses the corrected comprehensive flow transmission value ratio Bz, where z=1, 2; the first secondary correction method satisfies the quantity difference being less than or equal to the quantity difference standard preset by the central control module;

[0101] The second secondary correction method is to correct the corrected comprehensive flow transmission value ratio to Bz', setting Bz'=1.05×Bz; the second secondary correction method satisfies the quantity difference being greater than the quantity difference standard preset by the central control module;

[0102] The sixth preset condition is that the central control module determines that the number of newly added connections is greater than the preset connection number standard in the connection pool;

[0103] Among them, the quantity difference standard is equal to 10.

[0104] Specifically, the central control module obtains corresponding keywords according to the access request sent by the terminal to connect the terminal with the corresponding secondary data warehouse using a connection to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse.

[0105] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool, characterized in that: include: When a single terminal in the user end issues an access request, the central control module calls a single connection from the connection pool according to the information in the access request to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse; The central control module marks the connection as a running connection when the connection completes the construction of the data channel between the user terminal and the secondary data warehouse; When the central control module completes the construction of the data channel between the user terminal and the secondary data warehouse, the central control module sequentially records the access time node when the terminal issues an access request, the output time node when the secondary data warehouse outputs data, and the receiving time node when the user terminal receives data. The central control module obtains the response time of the running connection for this access request based on the access time node and the output time node, and obtains the transmission time of the running connection for this access request based on the output time node and the receiving time node. The central control module counts the response time and transmission time of each access request of a single connection within a preset time to obtain the average response time and average transmission time of the running connection between the user terminal and the secondary data warehouse within the preset time, and determines whether the running connection meets the preset standard according to the average response time; When determining that a single connection does not meet the preset standard, the central control module determines whether to adjust the maximum allowed value of the flow transmission in the running connection or whether to call several new connections to build several data channels between the user terminal and the secondary data warehouse according to the average response time, and after completing the determination, determines whether to correct the comprehensive flow transmission value ratio in each running connection according to the comprehensive transmission time of each running connection; When determining that a new connection needs to be called, the central control module determines whether to add a new connection to the connection pool based on the total number of connections to be used, and determines whether to perform a secondary correction on the comprehensive traffic transmission value ratio of each running connection when the number of connections in the connection pool exceeds the preset connection number standard.

2. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 1 is characterized in that: The central control module calculates the average response time Ta0 of the running connection connected to a single secondary data warehouse within the preset time under the first preset condition, and sets Wherein, n is the number of access requests transmitted by the running connection within the preset time length, Tai is the i-th response time length corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset time length, i=1, 2, 3, ..., n, ei is the i-th time length correction coefficient for the i-th response time length; The first preset condition is that the central control module determines that the running time of a single running connection reaches a preset time.

3. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 2 is characterized in that: The central control module determines a determination method for the running connection according to the average response time Ta0 under the first preset condition, wherein: The first determination method is that the central control module determines that the running connection of the running connection meets the preset standard; the first determination method satisfies that the average response time Ta0 of the running connection in the preset time is less than or equal to the first preset average response time Ta1 preset in the central control module; The second determination method is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module adjusts the maximum allowable value of the flow transmission for the running connection to the corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1; the second determination method satisfies that the average response time Ta0 is greater than the first preset average response time Ta1 and Ta0 is less than the second preset average response time Ta2 preset by the central control module; The third judgment method is that the central control module determines that the running connection of the secondary data warehouse connection does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse; the third judgment method satisfies that the average response time Ta0 is greater than the second preset average response time Ta2.

4. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 3 is characterized in that: The central control module determines the assignment method of each duration correction coefficient in sequence according to the instantaneous data transmission volume of the running connection at the time node when the running connection transmits the access request to each terminal to the secondary data warehouse in the preset duration under the second preset condition, wherein, for the i-th duration correction coefficient ei, The first assignment method is that the central control module assigns ei a value of 1.15; the first assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is less than or equal to the first preset instantaneous data transmission volume Qa preset in the central control module; The second assignment method is that the central control module assigns ei a value of 1; the second assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the first preset instantaneous data transmission volume Qa preset in the central control module and Qi is less than or equal to the second preset instantaneous data transmission volume Qb preset in the central control module; The third assignment method is that the central control module assigns ei a value of 0.95; the third assignment method satisfies that when the terminal transmits the i-th access request to the secondary data warehouse, the i-th instantaneous data transmission volume Qi of the running connection is greater than the second preset instantaneous data transmission volume Qb; The second preset condition is that the central control module calculates the average response time of the running connection within the preset time.

5. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 4 is characterized in that: The central control module calculates the difference △Tax between the average response time Ta0 and the first preset average response time Ta1 under the third preset condition, and sets △Tax=Ta1-Ta0. The central control module records △Tax as the first-level response time difference of the running connection and determines the adjustment method for the maximum allowable value of the flow transmission of the running connection according to △Tax, wherein, The first adjustment method is that the central control module uses a preset first preset adjustment coefficient β1 to adjust the maximum allowable value C of the flow transmission to a corresponding value, and 1<β1<1.2 is set; the first adjustment method satisfies that the first-level response time difference △Tax is less than or equal to the first preset first-level response time difference △Tax1 set by the central control module; The second adjustment method is that the central control module uses the preset second preset adjustment coefficient β2 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β1<β2<1.3; the second adjustment method satisfies that the first-level response time difference △Tax is greater than the first preset first-level response time difference △Tax1 and the first-level response time difference △Tax is less than or equal to the second preset first-level response time difference △Tax2 set by the central control module; The third adjustment method is that the central control module uses the preset third preset adjustment coefficient β3 to adjust the maximum allowable value C of the flow transmission to the corresponding value, and sets β2<β3<1.5; the third adjustment method satisfies that the first-level response time difference △Tax is greater than the second preset first-level response time difference △Tax2; The central control module records the adjusted maximum allowed value of the flow transmission for the running connection as C', and sets C'=C×βj, where j=1, 2, 3; The third preset condition is that the central control module adjusts the maximum allowed value of flow transmission for the running connection to a corresponding value according to the difference between the average response time Ta0 and the first preset average response time Ta1.

6. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 5 is characterized in that: The central control module detects the average transmission time of the running connection between the user terminal and the secondary data warehouse in the preset time under the fourth preset condition. For the kth running connection connected to the user terminal and the secondary data warehouse respectively, the central control module records the average transmission time of the running connection in the preset time as Tbk0, where k=1, 2, 3, ..., m, m is the total number of running connections connected to the user terminal and the secondary data warehouse, and is set Wherein, Tbi is the i-th response duration corresponding to the i-th access request transmitted by the running connection to the secondary data warehouse within the preset duration; The fourth preset condition is that the central control module adjusts the maximum allowed value of the flow transmission for the running connection to C'.

7. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 6 is characterized in that: The central control module sequentially counts the average transmission time of each running connection within the preset time period to transmit data under the fourth preset condition to obtain the comprehensive transmission time Tb of each running connection within the preset time period, and determines the correction method for the comprehensive flow transmission value ratio in each running connection according to Tb, wherein the setting The first correction method is that the central control module sets the comprehensive flow transmission value ratio for each of the running connections to a preset transmission value ratio B0, where B0 is the ratio of the maximum data transmission value allowed in a single running connection to the maximum data transmission value that the running connection can withstand; the first correction method satisfies that the comprehensive transmission time Tb is less than or equal to the first preset comprehensive transmission time Tb1 set in the central control module; The second correction method is that the central control module uses the first correction coefficient e1 to set the comprehensive flow transmission value ratio B1 for each running connection, 1.2<e1<1.4, to B1=B0×e1; the second correction method satisfies that the comprehensive transmission time Tb is greater than the first preset comprehensive transmission time Tb1 and is less than or equal to the second preset comprehensive transmission time Tb2 set in the central control module; The third correction method is that the central control module uses the second correction coefficient to set the comprehensive flow transmission value ratio B2 for each running connection, e1<e2<1.5, and sets B2=B0×e2; the third correction method satisfies that the comprehensive transmission time Tb is greater than the second preset comprehensive transmission time Tb2.

8. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 7, characterized in that: The central control module calculates the difference △Tay between the average response time Ta0 and the second preset average response time Ta2 under the fifth preset condition, and sets △Tay=Ta0-Ta2. The central control module records △Tay as the secondary response time difference of the running connection and determines the calling method for the number of connections connected to the secondary data warehouse according to △Tay, wherein: The first calling method is that the central control module uses the preset first adjustment coefficient γ1 to change the number of connections connected to the secondary data warehouse to the first number M1, 1.3<γ1<1.5, setting Wherein, M is the initial number of connections connected to the secondary data warehouse; the first calling mode satisfies that the secondary difference △Tay is less than or equal to the preset secondary response time difference △Tay0 set by the central control module; The second calling method is that the central control module uses the preset second adjustment coefficient γ2 to change the number of connections connected to the secondary data warehouse to the first number M2, γ1<γ2<1.6, and sets The second calling mode satisfies that the second-level difference ΔTay is greater than the preset second-level response time difference ΔTay0; The fifth preset condition is that the central control module determines that the running connection connected to the secondary data warehouse does not meet the preset standard, and the central control module calls a corresponding number of new connections according to the difference between the average response time Ta0 and the second preset average response time Ta2 to build a number of data channels between the user terminal and the secondary data warehouse.

9. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 8, characterized in that: The central control module determines a secondary correction method for the comprehensive flow transmission value ratio of each running connection according to the difference between the total number of connections after the addition and the preset connection number standard under the sixth preset condition, wherein: The first secondary correction method is that the central control module uses the corrected comprehensive flow transmission value ratio Bz, where z=1, 2; the first secondary correction method satisfies the quantity difference being less than or equal to the quantity difference standard preset by the central control module; The second secondary correction method is to correct the corrected comprehensive flow transmission value ratio to Bz', setting Bz'=1.05×Bz; the second secondary correction method satisfies the quantity difference being greater than the quantity difference standard preset by the central control module; The sixth preset condition is that the central control module determines that the number of newly added connections is greater than a preset connection number standard in the connection pool.

10. The method for improving the concurrent performance of MPP data warehouse access based on a high-availability connection pool according to claim 1, characterized in that: The central control module obtains the corresponding keyword according to the access request sent by the terminal to connect the terminal with the corresponding secondary data warehouse using a connection to build a data channel between the user end and the corresponding secondary data warehouse in the MPP data warehouse.

Images