Data transmission capacity reduction and expansion method and related device

By using ordered collection cache and sorting data indexes between servers and adjusting the data transmission amount according to preset rules, the server downtime caused by data transmission pressure is solved, and the efficiency and stability of data transmission are improved.

CN115484273BActive Publication Date: 2024-11-19SHENZHEN SIWEI SOFTWARE TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202211071449.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-02
Publication Date
2024-11-19
Estimated Expiration
2042-09-02

AI Technical Summary

Technical Problem

In large-scale data transmission scenarios, the server is prone to crash due to sudden increase in visits or requests, and after simply cleaning up the wrong data, it may still crash due to excessive data pressure in the next transmission. How to optimize the data transmission method becomes urgent The problem solved.

Method used

By establishing an ordered set between servers, cache and sort data indexes, and adjust the data transmission amount according to the preset data transmission rules and capacity reduction ratio, ensuring that the data transmission amount is automatically adjusted when data transmission is abnormal, and improving the number of data transmission between servers Data synchronization efficiency and stability.

Benefits of technology

It improves the flexibility and efficiency of server data transmission, ensures stable operation of the server, and avoids downtime caused by excessive data pressure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115484273B_ABST
    Figure CN115484273B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a method for scaling data transmission and a related device. The scaling method is applied to a first server, the first server is connected to a second server in communication, the first server is used to send data, and the second server is used to receive data. The method includes: in the case of abnormal data transmission in the current data transmission cycle, N first data indexes are cached again in an ordered set for sorting, N is adjusted according to a preset scaling ratio, the adjusted first preset number is M, M second data indexes in the ordered set are determined, M data to be transmitted corresponding to the M second data indexes are determined, and M data to be transmitted are sent to the second server. The use of the embodiment of the present application helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure the stable operation of the server.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a method for scaling data transmission and a related device. Background Art

[0002] In scenarios where there is a large-scale data transmission demand, the server is prone to downtime due to a sudden increase in the number of visits or requests. When an error occurs in data transmission, if the erroneous data is simply cleaned up without any other processing, the server may still downtime due to excessive pressure to receive data during the next data transmission. Therefore, how to optimize the data transmission method has become an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a method for scaling data transmission and a related device, which helps to improve the efficiency of data synchronization between servers and helps to ensure the stable operation of the servers.

[0004] In a first aspect, an embodiment of the present application provides a method for scaling data transmission, which is applied to a first server, the first server is communicatively connected to a second server, the first server is used to send data, and the second server is used to receive data, the method comprising:

[0005] Determine a data index corresponding to at least one data to be transmitted, wherein the data index is at least one;

[0006] caching at least one data index into an ordered set, the ordered set sorting the at least one data index by score;

[0007] According to a preset data sending rule, determine N first data indexes in the ordered set, where N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission period, and N is a positive integer greater than or equal to 2;

[0008] Determine N data to be transmitted corresponding to the N first data indexes;

[0009] Sending the N data to be transmitted to the second server;

[0010] In the case where an exception occurs in the data transmission of the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting;

[0011] N is adjusted according to a preset shrinkage ratio, and the adjusted first preset number is M, where M is a positive integer greater than or equal to 1;

[0012] Determine M second data indexes in the ordered set, where the M second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0013] Determine M data to be transmitted corresponding to the M second data indexes;

[0014] Send the M data to be transmitted to the second server.

[0015] In a second aspect, an embodiment of the present application provides a device for scaling data transmission, which is applied to a first server, wherein the first server is communicatively connected to a second server, the first server is used to send data, and the second server is used to receive data. The device includes: a determination unit, a cache unit, a sending unit, and an adjustment unit, wherein:

[0016] The determining unit is used to determine a data index corresponding to at least one data to be transmitted, wherein the data index is at least one;

[0017] The cache unit is used to cache at least one data index into an ordered set, and the ordered set sorts the at least one data index by score;

[0018] The determining unit is further used to determine N first data indexes in the ordered set according to a preset data sending rule, where N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission period, and N is a positive integer greater than or equal to 2;

[0019] The determining unit is further used to determine N data to be transmitted corresponding to the N first data indexes;

[0020] The sending unit is used to send the N data to be transmitted to the second server;

[0021] The cache unit is further configured to cache the N first data indexes again into the ordered set for sorting when an abnormality occurs in the data transmission in the current data transmission cycle;

[0022] The adjusting unit is used to adjust N according to a preset shrinkage ratio, and the first preset number after adjustment is M, where M is a positive integer greater than or equal to 1;

[0023] The determining unit is further used to determine M second data indexes in the ordered set, where the M second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0024] The determining unit is further used to determine the M data to be transmitted corresponding to the M second data indexes;

[0025] The sending unit is further used to send the M data to be transmitted to the second server.

[0026] In a third aspect, an embodiment of the present application provides a server, comprising a processor, a memory, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to implement the steps in the first aspect of the embodiment of the present application.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute instructions of some or all of the steps described in the first aspect of the embodiment of the present application.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to enable a computer to execute part or all of the steps described in the first aspect of the embodiment of the present application.

[0029] The implementation of the embodiments of the present application has the following beneficial effects:

[0030] It can be seen that the scaling method and related device for data transmission described in the embodiments of the present application, the scaling method is applied to a first server, the first server is communicatively connected to the second server, the first server is used to send data, and the second server is used to receive data, the method includes: determining a data index corresponding to at least one data to be transmitted, the data index is at least one, caching at least one data index into an ordered set, the ordered set sorts at least one data index by a score, and according to a preset data sending rule, determining N first data indexes in the ordered set, N is a preset number of first data indexes, the N first data indexes are indexes corresponding to the data transmitted in the current data transmission cycle, N is a positive integer greater than or equal to 2, determining N data to be transmitted corresponding to the N first data indexes, sending N data to be transmitted to the second server, and receiving the data in the current data transmission cycle. In the event of a transmission exception, the N first data indexes are cached again in an ordered set for sorting, and N is adjusted according to a preset shrinkage ratio. The adjusted first preset number is M, where M is a positive integer greater than or equal to 1. The M second data indexes in the ordered set are determined, and the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle. The M data to be transmitted corresponding to the M second data indexes are determined, and the M data to be transmitted are sent to the second server. In this way, the data indexes are sorted through an ordered set. In the event of a data transmission exception, M data to be transmitted are selected from the ordered set according to the adjusted first preset number M and then sent to the second server. The data transmission volume is automatically adjusted when data transmission is abnormal, which helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure stable operation of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1A is a schematic diagram of interaction between a first server and a second server provided in an embodiment of the present application;

[0033] Figure 1B It is a structural diagram of a server provided in an embodiment of the present application;

[0034] Figure 1C It is a flowchart of a method for scaling data transmission provided by an embodiment of the present application;

[0035] Figure 2It is a structural diagram of another server provided in an embodiment of the present application;

[0036] Figure 3 This is a block diagram of the functional units of a data transmission capacity reduction and expansion device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The term "plurality" may refer to two or more, which will not be repeated later.

[0038] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] The key concepts and terms involved in this application include but are not limited to the following:

[0041] (1) Data index, which can be used to quickly access data in the database. The data and data index in the database are stored in pairs, and the data index and data correspond one to one.

[0042] (2) Ordered set, a data storage type in the redis database. Ordered sets cache data indexes. Each data index in an ordered set is associated with a score, and the ordered set sorts the data indexes according to the score.

[0043] (3) The server may be a portable electronic device equipped with an IOS system, an Android system, a Microsoft system or other operating systems. The portable electronic device may also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the server may not be a portable electronic device, but a desktop computer. The server may also include a server, etc., which is not limited here.

[0044] At present, in scenarios where there is a large-scale data transmission demand, the server is prone to crash due to a sudden increase in the number of visits or requests. When an error occurs in data transmission, if the erroneous data is simply cleaned up without any other processing, the server may still crash during the next data transmission due to excessive pressure on receiving data. Therefore, how to optimize the data transmission method has become an urgent problem to be solved.

[0045] Based on the above problems, the present application provides a method for scaling data transmission and a related device, which are described in detail below with reference to the accompanying drawings.

[0046] Figure 1A A schematic diagram of interaction between a first server and a second server provided in an embodiment of the present application: The first server 10 and the second server 20 maintain a communication connection, the first server 10 is used to send data, and the second server 20 is used to receive data.

[0047] Exemplarily, the first server 10 may determine a data index corresponding to at least one data to be transmitted, cache at least one data index in an ordered set, sort at least one data index by a score through the ordered set, and determine N first data indexes in the ordered set according to a preset data sending rule, and then determine N data to be transmitted corresponding to the N first data indexes, and send the N data to be transmitted to the second server 20. In the event of an abnormality in data transmission during the current data transmission cycle, the first server 10 caches the N first data indexes again in the ordered set for sorting, and adjusts N according to a preset shrinkage ratio, the adjusted first preset number is M, determines M second data indexes in the ordered set, and then determines M data to be transmitted corresponding to the M second data indexes, and sends the M data to be transmitted to the second server 20.

[0048] In this way, the data index is sorted by an ordered set. In the event of an abnormality in data transmission, M data to be transmitted are selected from the ordered set according to the adjusted first preset number M and then sent to the second server. The data transmission volume is automatically adjusted when an abnormality in data transmission occurs. This helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure the stable operation of the server.

[0049] See also Figure 1B , Figure 1B : It is a structural diagram of a server provided in an embodiment of the present application. The server may be a first server. The server includes a processor and a memory, etc. Among them, the memory is connected to the processor. The processor is the control center of the server, which uses various interfaces and lines to connect various parts of the entire server, and executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory, and calling data stored in the memory, thereby monitoring the server as a whole. The processor may be a central processing unit (CPU), a graphics processing unit (GPU) or a network processor (Neural-network Processing Unit, NPU).

[0050] Furthermore, the processor may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor.

[0051] The memory is used to store software programs and / or modules, and the processor executes various functional applications of the server by running the software programs and / or modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a software program required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0052] See also Figure 1C , Figure 1C : is a flow chart of a data transmission scaling method provided by an embodiment of the present application, as shown in the figure, which is applied to Figure 1B The server shown may be a first server, the first server is communicatively connected with the second server, the first server is used to send data, and the second server is used to receive data; the data transmission scaling method includes:

[0053] Step 101: Determine a data index corresponding to at least one data to be transmitted.

[0054] There is at least one data index, and there is a one-to-one correspondence between the data index and the data to be transmitted.

[0055] Step 102: Cache at least one data index into an ordered set, wherein the ordered set sorts the at least one data index by score.

[0056] The score corresponding to the data index may be determined by the time when the data was generated, the time when the data was changed, the size of the data, etc., and is not limited here.

[0057] Step 103: Determine N first data indexes in the ordered set according to a preset data sending rule.

[0058] Among them, N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission period, and N is a positive integer greater than or equal to 2.

[0059] The data transmission period may refer to a time interval for data synchronization between the first server and the second server, such as 100 milliseconds, 300 milliseconds, 500 milliseconds, 1 second, etc., which is not limited here.

[0060] The preset data transmission rule may be pre-stored in the first server or obtained by the first server from the second server, which is not limited here. The preset number may be a pre-set number or a number that changes randomly according to the communication conditions between the first server and the second server, which is not limited here.

[0061] Step 104: Determine N data to be transmitted corresponding to the N first data indexes.

[0062] Step 105: Send the N data to be transmitted to the second server.

[0063] Step 106: When an exception occurs in the data transmission of the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting.

[0064] The above-mentioned data transmission abnormality includes the first server failing to successfully send the data to be transmitted, the second server failing to successfully receive the data to be transmitted, and the like.

[0065] Step 107 : N is adjusted according to a preset shrinkage ratio, and the adjusted first preset number is M.

[0066] The preset reduction ratio may be a preset ratio or a ratio that changes randomly according to the communication conditions between the first server and the second server, which is not limited here. The preset reduction ratio may be pre-stored in the first server or obtained by the first server from the second server, which is not limited here. For example, the reduction ratio may be 90%, 80%, 75%, 60%, 55%, 50%, etc.

[0067] Wherein, M is a positive integer greater than or equal to 1. It can be understood that M is the product of the shrinkage ratio and N.

[0068] Step 108: Determine M second data indexes in the ordered set.

[0069] The M second data indexes are indexes corresponding to data to be transmitted in the next data transmission period.

[0070] Step 109: Determine M data to be transmitted corresponding to the M second data indexes.

[0071] Step 110: Send the M data to be transmitted to the second server.

[0072] In a specific implementation, the first server processes the data indexes in the ordered set according to their arrangement order. For example, the data indexes included in the ordered set A are a1, a2, a3, a4...a10. In the current data transmission cycle, the first server determines a1, a2, and a3 as the first data indexes, and determines the data to be transmitted corresponding to a1, a2, and a3, respectively, and sends the data to be transmitted to the second server. If an exception occurs in the data transmission of the current data transmission cycle, a1, a2, and a3 are cached again in the ordered set A. At this time, the data indexes in the ordered set A are a4...a10, a1, a2, and a3. In the next data transmission cycle, the first server transmits the data to be transmitted corresponding to a4 and a5. That is to say, when an exception occurs in the data transmission of the current data transmission cycle, the corresponding data indexes are cached in the ordered set and reordered, and in the next data transmission cycle, the data corresponding to the data index arranged in front in the ordered set is transmitted.

[0073] It should be noted that in the above example, in the next data transmission cycle, the data corresponding to the data index arranged in the front in the ordered set is transmitted. In actual operation, in the next data transmission cycle, the data index arranged in the back in the ordered set can also be selected, or M second data indexes can be selected according to the scores corresponding to the data indexes, and the data corresponding to the selected data indexes can be determined and then transmitted.

[0074] It can be seen that in the embodiment of the present application, the first server can determine a data index corresponding to at least one data to be transmitted, the data index is at least one, and the at least one data index is cached in an ordered set. The ordered set sorts the at least one data index by a score, and according to a preset data sending rule, N first data indexes in the ordered set are determined, N is a preset number of first data indexes, the N first data indexes are indexes corresponding to the data transmitted in the current data transmission cycle, N is a positive integer greater than or equal to 2, and N data to be transmitted corresponding to the N first data indexes are determined, and the N data to be transmitted are sent to the second server. In the event that an abnormality occurs in the data transmission in the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting, and the data is sorted according to the order. N is adjusted according to the preset shrinkage ratio, and the adjusted first preset number is M, where M is a positive integer greater than or equal to 1. M second data indexes in the ordered set are determined, and the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle. M data to be transmitted corresponding to the M second data indexes are determined, and the M data to be transmitted are sent to the second server. In this way, the data indexes are sorted through the ordered set. In the event of an abnormality in data transmission, M data to be transmitted are selected from the ordered set according to the adjusted first preset number M and then sent to the second server. In the event of an abnormality in data transmission, the data transmission volume is automatically adjusted, which helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure stable operation of the server.

[0075] In a possible example, in the above step 102, the ordered set sorts the at least one data index by score, which may include the following steps:

[0076] Step 1021: Determine the timestamp corresponding to each data index.

[0077] Step 1022: Determine the score corresponding to each data index through the timestamp.

[0078] Step 1023: Sort the at least one data index according to the size of the score.

[0079] In a specific implementation, the score corresponding to the data index can be determined by a timestamp, and the timestamp can be determined by the generation time of the data corresponding to the data index. For example, the generation time of the data a to be transmitted in the first server is t1, and the generation time of the data b to be transmitted is t2, and t1 is earlier than t2, then the score s1 of the data index of the data a to be transmitted is less than the score s2 of the data index of the data b to be transmitted.

[0080] When sorting the data indexes corresponding to the data a and b to be transmitted in the ordered set, the data index of the data a to be transmitted can be placed before the data index of the data b to be transmitted, or the data index of the data a to be transmitted can be placed after the data index of the data b to be transmitted, according to the size of the scores, and there is no limitation here.

[0081] It can be seen that in the embodiment of the present application, the score corresponding to the data index is determined by the timestamp, and the data index is sorted according to the size of the score, which helps to ensure the orderliness of the data index in the ordered set, helps to improve the efficiency of retrieving the data index, and further helps to improve the efficiency of data transmission.

[0082] In a possible example, the above step 103, determining the N first data indexes in the ordered set according to a preset data sending rule, may include the following steps:

[0083] Step 1031: Determine the current time and the time period to which the current time belongs, wherein the time period includes an idle time period and a busy time period.

[0084] Step 1032: Determine the transmission cardinality according to the time period to which the current time belongs.

[0085] Step 1033: Determine the number of the second servers.

[0086] Step 1034: Determine the influence coefficient of the transmission cardinality according to the number of the second servers.

[0087] Step 1035: Determine a preset number N of the first data indexes according to the influence coefficient and the transmission cardinality.

[0088] The above-mentioned influence coefficient can change with the change of the number of second servers. For example, when the number of second servers is in the range of (0,50], the influence coefficient is 0.85; when the number of second servers is in the range of (50,100], the influence coefficient is 0.9; when the number of second servers is in the range of (100,500], the influence coefficient is 0.92; when the number of second servers is in the range of (500,1000], the influence coefficient is 0.95; when the number of second servers is greater than 1000, the influence coefficient is 1.1. In this way, when the number of second servers is large, the preset number N of the first data index also increases accordingly. Adjusting the influence coefficient according to the number of second servers helps to improve the flexibility of data transmission of the first server.

[0089] The transmission cardinality can be determined based on the history of data transmission between the first server and the second server, and is not limited here. For example, if the first server and the second server usually transmit 1000 pieces of data per data transmission cycle, the transmission cardinality can be determined to be 1000.

[0090] In the specific implementation, it is considered that in certain specific time periods, the demand for data transmission between servers is relatively large. For example, in the scenario of house sales, the number of inquiries about houses for sale will increase on weekends or holidays, and the demand for data transmission on weekends or holidays is relatively large. Before the relevant preferential policies are introduced, the number of inquiries about houses for sale will increase, and after the policies are introduced, the number of house transactions will increase. Therefore, before and after the policies are introduced, the demand for data transmission between servers is relatively large. Therefore, the data transmission period can be divided into idle period and busy period. The transmission cardinality corresponding to the idle period is small, and the transmission cardinality corresponding to the busy period is large.

[0091] For example, when the first server transmits data to the second server, the current time is an idle period, then the transmission cardinality of the idle period is determined to be B1, and the influence coefficient α is determined according to the number of second servers, then the preset number of first data indexes N=α*B1.

[0092] It can be seen that in the embodiment of the present application, the preset number of first data indexes is determined based on the current time of data transmission and the number of second servers, and the number of servers receiving data and the time period of data transmission are taken into consideration when setting the number of data transmissions. This helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, while helping to reduce the pressure on the server to send and receive data, avoid downtime, and help ensure stable operation of the server.

[0093] In a possible example, if there are multiple second servers, each second server corresponds to at least one ordered set, and each ordered set corresponds to only one second server.

[0094] For example, the second server A corresponds to the ordered sets A, B, C and D, and the second server B corresponds to the ordered set E. The ordered sets A, B, C and D only store data indexes corresponding to data sent to the second server A, and the ordered set E only stores data indexes corresponding to data sent to the second server B.

[0095] In a possible example, the above step 106, caching the N first data indexes again into the ordered set for sorting, may include the following steps:

[0096] Step 1061: Determine a second server where an exception occurs, where the second server where the exception occurs is a target server, and there is at least one target server.

[0097] Step 1062: Execute the following steps AB for each target server:

[0098] Step A, determining an ordered set corresponding to a target server currently being processed, wherein the ordered set corresponding to the target server currently being processed is a target ordered set, and the target server currently being processed corresponds to at least one target ordered set;

[0099] Step B: Cache the N first data indexes into each target ordered set for sorting.

[0100] In a specific implementation, for the second server (target server) where an exception occurs, one or more corresponding target ordered sets are determined, and the first data indexes corresponding to the N to-be-transmitted data where the transmission exception occurs are re-cached into each target ordered set for sorting.

[0101] In the case where a target server corresponds to multiple ordered sets, the sorting rules for the data indexes in the multiple ordered sets can be the same or different, and this is not limited here. For example, target server A corresponds to ordered sets A and B, and the data indexes with smaller scores are processed first in ordered set A, and the data indexes with larger scores are processed first in ordered set B. In this way, different target servers correspond to different data transmission mechanisms, which helps to improve the flexibility of data transmission between servers.

[0102] In a possible example, the method may further include the following steps (not shown):

[0103] Step 111: When no abnormality occurs in the data transmission in the current data transmission cycle, N is adjusted according to a preset expansion ratio, and the adjusted second preset number is P, where P is a positive integer greater than N.

[0104] Step 112: Determine P second data indexes in the ordered set, where the P second data indexes are indexes corresponding to data to be transmitted in the next data transmission period.

[0105] Step 113: Determine P data to be transmitted corresponding to the P data indexes.

[0106] Step 114: Send the P data to be transmitted to the second server.

[0107] Among them, the above-mentioned preset expansion ratio can be a pre-set ratio, or a ratio that changes randomly according to the communication conditions of the first server and the second server, which is not limited here. The preset expansion ratio can be pre-stored in the first server, or obtained by the first server from the second server, which is not limited here. For example, the expansion ratio can be 110%, 120%, 125%, 130%, 155%, 180%, 200%, etc. It can be understood that P is the product of the expansion ratio and N.

[0108] For example, the data indexes included in the ordered set A are a1, a2, a3, a4...a10. In the current data transmission cycle, the first server determines a1 and a2 as the first data indexes, and determines the data to be transmitted corresponding to a1 and a2, respectively, and sends the data to be transmitted to the second server. If there is no abnormality in the data transmission in the current data transmission cycle, then in the next data transmission cycle, the first server adjusts N according to the preset expansion ratio and transmits the data to be transmitted corresponding to a3, a4, a5, a6, a7, a8, and a9.

[0109] In this way, when no abnormality occurs in data transmission, the preset number N of the first data index is adjusted according to the expansion ratio to increase the amount of data transmitted by the first server, which helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers.

[0110] In accordance with the above embodiment, please refer to Figure 2 , Figure 2 is a schematic diagram of the structure of another server provided in an embodiment of the present application, and the server may be a first server. Figure 2 As shown, the server includes a processor, a memory, and a computer program or instruction stored in the memory, wherein the computer program or instruction is stored in the memory and is configured to be executed by the processor. The server may also include a communication interface. In the embodiment of the present application, the program includes instructions for executing the following steps:

[0111] Determine a data index corresponding to at least one data to be transmitted, wherein the data index is at least one;

[0112] caching at least one data index into an ordered set, the ordered set sorting the at least one data index by score;

[0113] According to a preset data sending rule, determine N first data indexes in the ordered set, where N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission period, and N is a positive integer greater than or equal to 2;

[0114] Determine N data to be transmitted corresponding to the N first data indexes;

[0115] Sending the N data to be transmitted to the second server;

[0116] In the case where an exception occurs in the data transmission of the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting;

[0117] N is adjusted according to a preset shrinkage ratio, and the adjusted first preset number is M, where M is a positive integer greater than or equal to 1;

[0118] Determine M second data indexes in the ordered set, where the M second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0119] Determine M data to be transmitted corresponding to the M second data indexes;

[0120] Send the M data to be transmitted to the second server.

[0121] It can be seen that the server described in the embodiment of the present application can determine the data index corresponding to at least one data to be transmitted, the data index is at least one, and the at least one data index is cached in an ordered set. The ordered set sorts the at least one data index by a score, and according to the preset data sending rule, N first data indexes in the ordered set are determined, N is the preset number of first data indexes, the N first data indexes are the indexes corresponding to the data transmitted in the current data transmission cycle, N is a positive integer greater than or equal to 2, and the N data to be transmitted corresponding to the N first data indexes are determined, and the N data to be transmitted are sent to the second server. In the case of an abnormality in the data transmission in the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting. N is adjusted according to a preset shrinkage ratio, and the adjusted first preset number is M, where M is a positive integer greater than or equal to 1. M second data indexes in the ordered set are determined, and the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle. M data to be transmitted corresponding to the M second data indexes are determined, and the M data to be transmitted are sent to the second server. In this way, the data indexes are sorted through the ordered set. In the event of an abnormality in data transmission, M data to be transmitted are selected from the ordered set according to the adjusted first preset number M and then sent to the second server. In the event of an abnormality in data transmission, the data transmission volume is automatically adjusted, which helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure stable operation of the server.

[0122] In one possible example, in terms of the ordered set sorting the at least one data index by score, the program includes instructions for performing the following steps:

[0123] Determine the timestamp corresponding to each data index;

[0124] Determine the score corresponding to each data index according to the timestamp;

[0125] The at least one data index is sorted according to the size of the score.

[0126] In a possible example, in determining the N first data indexes in the ordered set according to a preset data sending rule, the program includes instructions for performing the following steps:

[0127] Determine the current time and the time period to which the current time belongs, wherein the time period includes an idle time period and a busy time period;

[0128] Determine the transmission cardinality according to the time period to which the current time belongs;

[0129] determining the number of the second servers;

[0130] Determining an influence coefficient of the transmission cardinality according to the number of the second servers;

[0131] A preset number N of the first data indexes is determined according to the influence coefficient and the transmission cardinality.

[0132] In a possible example, if there are multiple second servers, each second server corresponds to at least one ordered set, and each ordered set corresponds to only one second server.

[0133] In a possible example, in terms of caching the N first data indexes again in the ordered set for sorting, the program includes instructions for performing the following steps:

[0134] Determine a second server where an exception occurs, where the second server where the exception occurs is a target server, and there is at least one target server;

[0135] Perform the following steps AB for each target server:

[0136] Step A, determining an ordered set corresponding to a target server currently being processed, wherein the ordered set corresponding to the target server currently being processed is a target ordered set, and the target server currently being processed corresponds to at least one target ordered set;

[0137] Step B: Cache the N first data indexes into each target ordered set for sorting.

[0138] In a possible example, the program further includes instructions for executing the following steps:

[0139] When no abnormality occurs in the data transmission of the current data transmission cycle, N is adjusted according to the preset expansion ratio, and the adjusted second preset number is P, where P is a positive integer greater than N;

[0140] Determine P second data indexes in the ordered set, where the P second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0141] Determine P data to be transmitted corresponding to the P second data indexes;

[0142] Send the P data to be transmitted to the second server.

[0143] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should be easily aware that, in combination with the units and algorithm steps of each example described in the embodiment provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0144] The embodiment of the present application can divide the functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0145] See also Figure 3 , Figure 3 300 is a functional unit composition block diagram of a data transmission capacity reduction and expansion device provided in an embodiment of the present application. The device 300 includes: a determination unit 301, a cache unit 302, a sending unit 303 and an adjustment unit 304, wherein:

[0146] The determining unit 301 is used to determine a data index corresponding to at least one data to be transmitted, and the data index is at least one;

[0147] The cache unit 302 is used to cache at least one data index into an ordered set, and the ordered set sorts the at least one data index by score;

[0148] The determining unit 301 is further configured to determine, according to a preset data sending rule, N first data indexes in the ordered set, where N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission period, and N is a positive integer greater than or equal to 2;

[0149] The determining unit 301 is further configured to determine N data to be transmitted corresponding to the N first data indexes;

[0150] The sending unit 303 is used to send the N to-be-transmitted data to the second server;

[0151] The cache unit 302 is further configured to cache the N first data indexes again into the ordered set for sorting when an abnormality occurs in the data transmission in the current data transmission cycle;

[0152] The adjusting unit 304 is used to adjust N according to a preset shrinkage ratio, and the adjusted first preset number is M, where M is a positive integer greater than or equal to 1;

[0153] The determining unit 301 is further configured to determine M second data indexes in the ordered set, where the M second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0154] The determining unit 301 is further configured to determine M data to be transmitted corresponding to the M second data indexes;

[0155] The sending unit 303 is further configured to send the M data to be transmitted to the second server.

[0156] It can be seen that the data transmission capacity reduction and expansion device described in the embodiment of the present application can determine the data index corresponding to at least one data to be transmitted, the data index is at least one, and the at least one data index is cached in an ordered set. The ordered set sorts the at least one data index by a score, and according to the preset data sending rule, the N first data indexes in the ordered set are determined, N is the preset number of first data indexes, the N first data indexes are the indexes corresponding to the data transmitted in the current data transmission cycle, N is a positive integer greater than or equal to 2, and the N data to be transmitted corresponding to the N first data indexes are determined, and the N data to be transmitted are sent to the second server. In the case of an abnormality in the data transmission in the current data transmission cycle, the N first data indexes are cached again in the ordered set for transmission. Sorting, adjusting N according to a preset shrinkage ratio, the adjusted first preset number is M, M is a positive integer greater than or equal to 1, determining M second data indexes in the ordered set, the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle, determining M data to be transmitted corresponding to the M second data indexes, and sending the M data to be transmitted to the second server; in this way, the data indexes are sorted through the ordered set, and in the event of an abnormality in data transmission, M data to be transmitted are selected from the ordered set according to the adjusted first preset number M and then sent to the second server, and the data transmission volume is automatically adjusted when data transmission is abnormal, which helps to improve the flexibility of server data transmission and the efficiency of data synchronization between servers, and at the same time helps to ensure stable operation of the server.

[0157] In a possible example, in terms of the ordered set sorting the at least one data index by score, the cache unit 302 is specifically configured to:

[0158] Determine the timestamp corresponding to each data index;

[0159] Determine the score corresponding to each data index according to the timestamp;

[0160] The at least one data index is sorted according to the size of the score.

[0161] In a possible example, in determining the N first data indexes in the ordered set according to a preset data sending rule, the determining unit 301 is specifically configured to:

[0162] Determine the current time and the time period to which the current time belongs, wherein the time period includes an idle time period and a busy time period;

[0163] Determine the transmission cardinality according to the time period to which the current time belongs;

[0164] determining the number of the second servers;

[0165] Determining an influence coefficient of the transmission cardinality according to the number of the second servers;

[0166] A preset number N of the first data indexes is determined according to the influence coefficient and the transmission cardinality.

[0167] In a possible example, if there are multiple second servers, each second server corresponds to at least one ordered set, and each ordered set corresponds to only one second server.

[0168] In a possible example, in terms of caching the N first data indexes again into the ordered set for sorting, the cache unit 302 is specifically configured to:

[0169] Determine a second server where an exception occurs, where the second server where the exception occurs is a target server, and there is at least one target server;

[0170] Perform the following steps AB for each target server:

[0171] Step A, determining an ordered set corresponding to a target server currently being processed, wherein the ordered set corresponding to the target server currently being processed is a target ordered set, and the target server currently being processed corresponds to at least one target ordered set;

[0172] Step B: Cache the N first data indexes into each target ordered set for sorting.

[0173] In a possible example, the adjustment unit 304 is further configured to adjust N according to a preset expansion ratio when no abnormality occurs in the data transmission in the current data transmission cycle, and the adjusted second preset number is P, where P is a positive integer greater than N;

[0174] The determining unit 301 is further configured to determine P second data indexes in the ordered set, where the P second data indexes are indexes corresponding to data to be transmitted in the next data transmission period;

[0175] The determining unit 301 is further configured to determine P data to be transmitted corresponding to the P second data indexes;

[0176] The sending unit 303 is further configured to send the P data to be transmitted to the second server.

[0177] It can be understood that the functions of each program module of the data transmission capacity reduction and expansion device of this embodiment can be specifically implemented according to the method in the above method embodiment, and its specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0178] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0179] The present application also provides a computer program product, which includes a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. A computer program product should be understood as a software product that implements its solution mainly through a computer program.

[0180] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0181] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0183] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0186] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0187] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for scaling data transmission, characterized in that: Applied to a first server, the first server is in communication connection with a second server, the first server is used to send data, and the second server is used to receive data, the method includes: Determine a data index corresponding to at least one data to be transmitted, where the data index is at least one; cache the at least one data index into an ordered set, where the ordered set sorts the at least one data index by a score, where there are multiple second servers, each second server corresponds to at least one ordered set, and each ordered set corresponds to only one second server; determine N first data indexes in the ordered set according to a preset data sending rule, where N is a preset number of first data indexes, the N first data indexes are indexes corresponding to data transmitted in the current data transmission cycle, and N is a positive integer greater than or equal to 2; determine N data to be transmitted corresponding to the N first data indexes; and send the N data to be transmitted to the second server; In the event that an exception occurs in the data transmission of the current data transmission cycle, the N first data indexes are cached again in the ordered set for sorting, including: determining the second server where the exception occurs, the second server where the exception occurs is the target server, and there is at least one target server; executing the following steps AB for each target server: step A, determining the ordered set corresponding to the currently processed target server, the ordered set corresponding to the currently processed target server is the target ordered set, and the currently processed target server corresponds to at least one target ordered set; step B, caching the N first data indexes in each target ordered set for sorting; and adjusting N according to a preset shrinkage ratio, the adjusted first preset number is M, M is a positive integer greater than or equal to 1, and M is less than N; determining M second data indexes in the target ordered set, the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle; determining M data to be transmitted corresponding to the M second data indexes; sending the M data to be transmitted to the target server; When no abnormality occurs in the data transmission of the current data transmission cycle, N is adjusted according to the preset expansion ratio, and the adjusted second preset number is P, where P is a positive integer greater than N; P second data indexes in the ordered set are determined, and the P second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle; P data to be transmitted corresponding to the P second data indexes are determined; and the P data to be transmitted are sent to the second server.

2. The method according to claim 1, characterized in that The step of sorting the at least one data index by score comprises: Determine the timestamp corresponding to each data index; Determine the score corresponding to each data index according to the timestamp; The at least one data index is sorted according to the size of the score.

3. The method according to claim 2, characterized in that The determining, according to a preset data sending rule, N first data indexes in the ordered set includes: Determine the current time and the time period to which the current time belongs, wherein the time period includes an idle time period and a busy time period; Determine the transmission cardinality according to the time period to which the current time belongs; determining the number of the second servers; Determining an influence coefficient of the transmission cardinality according to the number of the second servers; According to the influence coefficient and the transmission cardinality, a preset number N=α*B1 of the first data indexes is determined, wherein B1 is the transmission cardinality and α is the influence coefficient.

4. A data transmission capacity reduction and expansion device, characterized in that: Applied to a first server, the first server is connected to a second server for communication, the first server is used to send data, and the second server is used to receive data, the device includes: a determination unit, a cache unit, a sending unit and an adjustment unit, wherein: The determining unit is used to determine a data index corresponding to at least one data to be transmitted, and the data index is at least one; the caching unit is used to cache at least one data index into an ordered set, and the ordered set sorts the at least one data index by a score, and there are multiple second servers, each second server corresponds to at least one ordered set, and each ordered set corresponds to only one second server; the determining unit is also used to determine N first data indexes in the ordered set according to a preset data sending rule, N is a preset number of first data indexes, and the N first data indexes are indexes corresponding to data transmitted in the current data transmission cycle, and N is a positive integer greater than or equal to 2; it is also used to determine N data to be transmitted corresponding to the N first data indexes; the sending unit is used to send the N data to be transmitted to the second server; The cache unit is further used to cache the N first data indexes again in the ordered set for sorting when an exception occurs in the data transmission of the current data transmission cycle; in terms of caching the N first data indexes again in the ordered set for sorting, the cache unit is specifically used to: determine the second server where the exception occurs, the second server where the exception occurs is the target server, and there is at least one target server; perform the following steps AB for each target server: step A, determine the ordered set corresponding to the currently processed target server, the ordered set corresponding to the currently processed target server is the target ordered set, and the currently processed target server corresponds to at least one target ordered set; step B, cache the N first data indexes in each target ordered set for sorting; the adjustment unit is used to adjust N according to a preset shrinkage ratio, and the adjusted first preset number is M, M is a positive integer greater than or equal to 1, and M is less than N; the determination unit is also used to determine M second data indexes in the target ordered set, the M second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle; and is also used to determine M data to be transmitted corresponding to the M second data indexes; the sending unit is also used to send the M data to be transmitted to the target server; The adjustment unit is further used to adjust N according to a preset expansion ratio when no abnormality occurs in the data transmission in the current data transmission cycle, and the second preset number after adjustment is P, where P is a positive integer greater than N; the determination unit is further used to determine P second data indexes in the ordered set, and the P second data indexes are indexes corresponding to the data to be transmitted in the next data transmission cycle; determine P data to be transmitted corresponding to the P second data indexes; and the sending unit is further used to send the P data to be transmitted to the second server.

5. A server comprising a processor, a memory and a computer program or instruction stored in the memory, characterized in that: The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Citation Information

Patent Citations

  • Data transmission system, method and device

    CN110300188A

  • Automatic expansion method based on micro-service

    CN112448848A

  • Data transmission method and system, electronic equipment and storage medium

    CN112486697A

  • Intelligent scaling in microservice-based distributed systems

    US20220043699A1