Data transmission method and device, medium and equipment

Abstract

The invention relates to a data transmission method, which is applied to a scheduling platform and comprises the following steps: acquiring parameters of each piece of data; formulating a transmission scheme of each piece of data according to the parameters of the data, wherein the transmission scheme comprises transmission starting time and customized transmission rate; and based on the transmission scheme of each data, generating a data transmission task, and issuing the data transmission task to an access POP point of the transmission network, so that the access POP point processes each data according to the data transmission task. By making a personalized transmission scheme, a transmission starting moment and a transmission rate can be intelligently selected according to characteristics of different data, intelligent scheduling of a data transmission process can be realized, idle bandwidth of the Internet can be fully utilized, congestion in a peak period can be avoided, and low-cost and efficient data transmission can be realized.

Description

Technical Field

[0001] This application relates to the field of computer networks, and in particular to a method, apparatus, medium and device for data transmission. Background Technology

[0003] To achieve coordinated allocation of computing power needs and resources, low-cost and efficient transmission of massive amounts of data is a crucial issue. The core problem that needs to be solved is how to intelligently select transmission time, mode, and path based on the characteristics of various types of data. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this application provides a method, apparatus, medium and device for data transmission.

[0005] According to a first aspect of this application, a data transmission method is provided, applied to a scheduling platform, comprising:

[0006] Obtain the parameters for each data item;

[0007] Based on the parameters of the data, a transmission scheme is formulated for each data item, and the transmission scheme includes the start transmission time and the customized transmission rate.

[0008] Based on the transmission schemes for each data item, a data transmission task is generated and sent to the access point of the transmission network so that the access point processes each data item according to the data transmission task.

[0009] In some embodiments of this application, based on the foregoing scheme, the data parameters include latency requirements, data packet size, or the amount of new data per second, and the step of formulating a transmission scheme for each data based on the data parameters includes:

[0010] When the latency requirement of the data is less than a preset threshold, the data is marked as real-time transmission data, and a customized transmission rate of the data is set according to the latency requirement and the data packet size or the amount of data added per second.

[0011] When the latency requirement of the data is greater than or equal to a preset threshold, the data is marked as delayed transmission data, and the start time of the delayed transmission data and the customized transmission rate are determined according to the latency requirement and the data packet size or the amount of data added per second.

[0012] In some embodiments of this application, based on the foregoing scheme, the data transmission method further includes:

[0013] Based on real-time or historical bandwidth data, determine whether the available bandwidth meets the requirements of the data transmission scheme;

[0014] If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth meets the requirements of the data transmission scheme.

[0015] In some embodiments of this application, based on the aforementioned scheme, when the idle bandwidth does not meet the adjusted data transmission scheme, an alarm message is sent.

[0016] In some embodiments of this application, based on the foregoing scheme, generating a data transmission task based on the data transmission scheme includes:

[0017] A sending task is generated for the real-time transmitted data, the delayed transmitted data that has reached the start transmission time, and the data being transmitted. The sending task includes the customized transmission rate.

[0018] A transfer task is generated for delayed transmission data that has not reached the start transmission time.

[0019] According to another aspect of this application, a data transmission method is provided, applied to an access point (POP) in a transmission network, comprising:

[0020] Receive data transmission tasks sent by the scheduling platform, wherein the data transmission tasks include sending tasks or transferring tasks;

[0021] Data is sent according to the sending task, or data is processed according to the transfer task.

[0022] In some embodiments of this application, based on the foregoing scheme, the sending task includes the customized transmission rate, and the sending of data according to the sending task includes:

[0023] Obtain the current idle bandwidth; when the idle bandwidth is greater than a preset bandwidth value, send data using a second transmission rate, which is greater than the customized rate; when the idle bandwidth is less than or equal to the preset bandwidth value, send data using a customized transmission rate.

[0024] When the actual transmission rate is less than the customized transmission rate, a new transmission path is selected.

[0025] In some embodiments of this application, based on the foregoing scheme, processing the data to be sent according to the transfer task includes:

[0026] Transfer the data to a storage platform or the storage location of the record data source, and mark the data as "pending transfer".

[0027] According to another aspect of this application, a data transmission apparatus is provided, applied to a scheduling platform, comprising:

[0028] The parameter acquisition module is used to acquire the parameters of each data item.

[0029] A transmission scheme formulation module is used to formulate a transmission scheme for each piece of data based on the parameters of the data. The transmission scheme includes a start transmission time and a customized transmission rate.

[0030] The task distribution module is used to generate data transmission tasks based on the transmission schemes of the various data, and distribute the data transmission tasks to the access POP points of the transmission network, so that the access POP points can process the various data according to the data transmission tasks.

[0031] In some embodiments of this application, based on the foregoing scheme, the parameters of the data include latency requirements, data packet size, or the amount of new data added per second;

[0032] The transmission scheme formulation module is also used to mark the data as real-time transmission data when the latency requirement of the data is less than a preset threshold, and to set a customized transmission rate of the data according to the latency requirement and the data packet size or the amount of data added per second.

[0033] When the latency requirement of the data is greater than or equal to a preset threshold, the data is marked as delayed transmission data, and the start time of the delayed transmission data and the customized transmission rate are determined according to the latency requirement and the data packet size or the amount of data added per second.

[0034] In some embodiments of this application, based on the foregoing scheme, the transmission scheme formulation module is further configured to determine whether the idle bandwidth meets the requirements of the data transmission scheme based on real-time bandwidth or historical bandwidth data.

[0035] If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth meets the requirements of the data transmission scheme.

[0036] In some embodiments of this application, based on the foregoing scheme, the transmission scheme formulation module is further configured to send an alarm message when the idle bandwidth does not meet the adjusted data transmission scheme.

[0037] In some embodiments of this application, based on the foregoing scheme, the task distribution module is further configured to generate a sending task for the real-time transmitted data, the delayed transmitted data that has reached the start transmission time, and the data being transmitted, wherein the sending task includes the customized transmission rate;

[0038] A transfer task is generated for delayed transmission data that has not reached the start transmission time.

[0039] According to another aspect of this application, a data transmission apparatus is provided, applied to an access point (POP) in a transmission network, comprising:

[0040] The task receiving module is used to receive data transmission tasks sent by the scheduling platform, wherein the data transmission tasks include sending tasks or transferring tasks.

[0041] The processing module is used to send data according to the sending task, or to process data according to the transfer task.

[0042] In some embodiments of this application, based on the foregoing scheme, the processing module is further configured to obtain the current idle bandwidth; when the idle bandwidth is greater than a preset bandwidth value, data is sent using a second transmission rate, the second transmission rate being greater than the customized rate; when the idle bandwidth is less than or equal to the preset bandwidth value, data is sent using a customized transmission rate.

[0043] When the actual transmission rate is less than the customized transmission rate, a new transmission path is selected.

[0044] In some embodiments of this application, based on the foregoing scheme, the processing module is further configured to transfer the data to the storage location of the storage platform or the record data source according to the transfer task, and mark the data as "pending transfer".

[0045] According to another aspect of this application, a computer-readable storage medium is provided that stores a computer program thereon, wherein the computer program, when executed, implements the steps of a method for data transmission.

[0046] According to another aspect of this application, a computer device is provided, including a processor, a memory, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a method for data transmission.

[0047] This application provides a data transmission method that, through the formulation of personalized transmission schemes, can intelligently select the start time and transmission rate based on the characteristics of different data. This enables intelligent scheduling of the data transmission process, fully utilizing idle internet bandwidth and avoiding congestion during peak hours, thereby achieving low-cost and efficient data transmission. It significantly improves the efficiency and economy of data transmission. It optimizes resource allocation and transmission costs, enabling the rational allocation of computing power needs in the east and computing resources in the west, improving the overall resource utilization rate of society and reducing total energy consumption. This provides strong support for the sustainable development of modern industry and manufacturing.

[0048] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0049] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0050] Figure 1 This is a schematic diagram of a data transmission system according to an exemplary embodiment.

[0051] Figure 2 This is a flowchart illustrating a data transmission method according to an exemplary embodiment.

[0052] Figure 3 This is a flowchart illustrating a data transmission method according to an exemplary embodiment.

[0053] Figure 4 This is a block diagram illustrating a data transmission apparatus according to an exemplary embodiment.

[0054] Figure 5 This is a block diagram illustrating a data transmission apparatus according to an exemplary embodiment.

[0055] Figure 6 This is a block diagram illustrating a computer device for data transmission according to an exemplary embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0058] The energy advantages of the western region can be leveraged to generate inexpensive computing power for data production, which can then be used in the eastern region. This involves setting up the data source in the west, utilizing its cheap computing power to generate the data, and then setting up the data application cluster in the east, transmitting the data from the data source to the data application cluster for use in the east. This resolves the conflict between energy consumption and computing power demands in the eastern region.

[0059] This requires transmitting the massive amounts of data generated in the western region to the eastern region, so as to achieve a reasonable joint allocation of the needs of the east and the computing resources of the west, thereby improving the utilization rate of resources across society and reducing energy consumption.

[0060] Figure 1This is a schematic diagram of a data transmission system according to an exemplary embodiment. (Reference) Figure 1 The data transmission system includes: data source, transmission network, transmission scheduling platform, and data application cluster.

[0061] Data sources are used to generate data. For example, in the application scenario of "East Data West Computing," the data source is set up in the western region, making full use of the natural advantages of the western region in energy, renewable resources, etc., which bring advantages in computing power and cost to produce data for use in other regions.

[0062] Data application clusters are used to store data and make it available to people. Clearly, given the dense population in the eastern region, setting up data application clusters there can reduce the cost of data access and improve data acquisition efficiency.

[0063] The transmission network, consisting of multiple interconnected POPs, is used to transmit massive amounts of data generated in the western region to the data application cluster in the east, enabling the joint allocation of computing power demand and computing resources.

[0064] Clearly, achieving coordinated allocation of computing power needs and resources necessitates low-cost, efficient transmission of massive amounts of data. The core issue that needs to be addressed is how to intelligently select transmission time, mode, and path based on the characteristics of various types of data.

[0065] To solve the above problems, fiber optic cables can be laid to achieve fast data transmission. However, laying fiber optic cables is costly, time-consuming, and inflexible in terms of capacity expansion. Furthermore, if line backup is implemented, twice the amount of fiber optic cable needs to be laid, which also involves high costs and long construction periods.

[0066] This application provides a data transmission method. By setting up a transmission scheduling platform, the platform can communicate with all Points of Presence (POPs) in the transmission network, collect the real-time bandwidth of all POPs, and use this data as a basis for calculating idle bandwidth. The transmission platform can also obtain parameters of the data to be transmitted, formulate a data transmission scheme based on the data parameters and idle bandwidth, and control the data transmission of POPs to improve the data transmission efficiency of the existing network, thereby achieving low-cost and efficient transmission of massive amounts of data.

[0067] Figure 2 This is a flowchart illustrating a data transmission method according to an exemplary embodiment. (Reference) Figure 2 The data transmission methods are applied to the scheduling platform, including:

[0068] Step S21: Obtain the parameters for each data point.

[0069] Different types of data have different characteristics. For example, news data, real-time communication data, live broadcast data, transaction data, and data frequently accessed within a certain time period require timely transmission, thus exhibiting low transmission latency. Conversely, archived data, old files, and other data with low access frequency do not require timely transmission, resulting in high transmission latency. Therefore, based on latency requirements, data with latency requirements below a preset threshold can be classified as "hot data," while data with latency requirements greater than or equal to the preset threshold can be classified as "cold data."

[0070] In practical applications, the parameters of each data point can be configured manually. For example, based on the characteristics of the data, it can be manually labeled as hot or cold data; or, a data latency can be set for each data point. Alternatively, the parameters of each data point can be automatically generated by intelligent analysis. For example, based on historical data transmission, the characteristics of the current data can be intelligently determined, and the current data can be automatically labeled as hot or cold data; or, a data latency requirement can be set for each data point.

[0071] Data parameters include, but are not limited to, data latency requirements, packet size or number of packets per second, data transmission encryption method, and destination IP address for data transmission.

[0072] In this context, latency refers to the time required for a message or packet to travel from one end of a network to the other. Latency requirement can be understood as the time range within which the data is sent to the receiving end. The shorter the latency requirement, the earlier the data needs to be sent to the receiving end, and it should be sent at a higher rate. The longer the latency requirement, the more likely the data can be sent later, as long as it reaches the receiving end within the time range corresponding to the latency requirement. Therefore, it can be sent at a lower rate at a certain time.

[0073] Step S22: Develop a transmission scheme for each data item based on the data parameters. The transmission scheme includes the start time of transmission and the customized transmission rate.

[0074] Once the scheduling platform obtains the parameters of the data to be transmitted, it can formulate a transmission plan for each piece of data based on these parameters.

[0075] If the latency requirement for certain data is very short, below a preset threshold, when formulating the transmission plan, the start transmission time is set to the current time, and the transmission rate is customized to ensure that the data can be transmitted to the receiving end in a timely manner.

[0076] If the latency requirement for certain data is relatively long, exceeding a preset threshold, the transmission plan should set the start time for transmission to a future point in time and customize the transmission rate accordingly. This ensures that the data can be transmitted to the receiving end within the required latency period. Furthermore, the future point in time can be chosen during off-peak network hours, such as 9 PM to 7 AM, when network usage is low and there is ample idle bandwidth. A higher transmission rate can then be used to transmit data, making efficient use of idle bandwidth and reducing bandwidth costs.

[0077] A higher transmission rate allows data to be transmitted faster, reducing data transmission time. However, the transmission rate is limited by the available bandwidth of the line or node, and is also affected by whether other data is being transmitted at the same time and the total amount of data that needs to be transmitted at that time. Therefore, when customizing the transmission rate, the customized transmission rate is not the actual transmission rate, nor is a higher rate always better. In this application, the customized transmission rate can be understood as the minimum transmission rate that ensures the data is successfully transmitted to the receiving end within the required delay time after transmission begins.

[0078] Step S23: Based on the transmission scheme of each data, generate a data transmission task and send the data transmission task to the access POP point of the transmission network so that the access POP point can process each data according to the data transmission task.

[0079] The scheduling platform generates data transmission tasks based on the transmission schemes for each data item and distributes these tasks to the access points (POPs) of the corresponding transmission networks. Data transmission tasks can be for immediate data transmission (real-time transmission) or for transmission of data scheduled to begin at a future time (timed transmission). Upon receiving a data transmission task, the access point (POP) processes the incoming data according to its instructions. If the data transmission task indicates real-time transmission, it forwards the data immediately; if it indicates timed transmission, it saves the data and forwards it at the time specified in the task. The data transmission task also includes a customized transmission rate, allowing the access point (POP) to determine the actual transmission rate of the data based on this customized rate.

[0080] The scheduling platform categorizes data into real-time and scheduled transmission based on its parameters, formulates transmission plans, determines the transmission time and rate for each data type, and generates data transmission tasks for the access points (POPs) of the transmission network. The POPs, following these tasks, send different data at different times, ensuring that data with low latency requirements is transmitted in real-time, while data with high latency requirements is sent at a later time. This guarantees that all data is delivered to the receiving end within the required latency timeframe, avoids network congestion caused by sending multiple data points simultaneously, and improves data transmission efficiency. Furthermore, the platform can schedule future transmissions during off-peak network periods to fully utilize idle internet bandwidth, achieving low-cost and high-efficiency data transmission.

[0081] By developing personalized transmission schemes, the start time and transmission rate can be intelligently selected based on the characteristics of different data. This enables intelligent scheduling of the data transmission process, fully utilizing idle internet bandwidth and avoiding congestion during peak hours, thus achieving low-cost and efficient data transmission. This significantly improves the efficiency and economy of data transmission. It optimizes resource allocation and transmission costs, enabling the rational allocation of computing power needs in the east and resources in the west, improving overall resource utilization and reducing total energy consumption. This provides strong support for the sustainable development of modern industry and manufacturing.

[0082] In an exemplary embodiment, the data parameters include latency requirements, packet size, or the amount of data added per second. In step S22, formulating a transmission scheme for each data item based on the data parameters includes:

[0083] When the latency requirement of the data is less than the preset threshold, the data is marked as real-time transmission data, and the customized transmission rate of the data is set according to the latency requirement and the size of the data packet or the amount of data added per second.

[0084] When the data latency requirement is greater than or equal to a preset threshold, the data is marked as delayed transmission data, and the start time of delayed transmission data and the customized transmission rate are determined according to the latency requirement and the data packet size or the amount of new data per second.

[0085] For example, if the preset latency requirement is 5 seconds, and the latency requirement for a certain data is 4 seconds, data that is less than the preset threshold of 5 seconds is marked as real-time transmission data. When the data is a fixed-size data packet (500KB), to transmit the packet to the receiving end within 4 seconds, a custom transmission rate can be set to 500KB / 4 = 125KB. When the data is a video stream, with 500KB of new data added per second, a custom transmission rate of 500KB can be set to ensure timely transmission of the newly added data, preventing the video stream from becoming too late and affecting the viewing experience at the receiving end. During data transmission, the transmission rate can be adjusted based on the current available bandwidth. As long as the real-time transmission rate is greater than or equal to the custom rate, the latency requirement for the data can be guaranteed.

[0086] For example, if the preset threshold for latency is 5 seconds, and the latency requirement for certain data is 8 hours, exceeding the preset threshold, the data is marked as delayed transmission data. If the current time is 13:00:00, the size of this delayed transmission data is 5MB. Furthermore, based on historical statistics, the available bandwidth around 21:00 is 5-10MB, allowing us to determine a customized transmission rate of 0.5MB for this delayed data, with the transmission start time being any time before 22:59:50. This ensures the data is transmitted before 21:00:00. In practical applications, the start transmission time can be set to 22:59:00, and the actual transmission rate can be determined to be 1MB based on the available bandwidth at that time, completing the data transmission as quickly as possible.

[0087] In an exemplary embodiment, in step S23, generating a data transmission task based on the transmission schemes of each data includes:

[0088] Sending tasks are generated for real-time transmitted data, delayed transmitted data that has reached the start time of transmission, and data that is being transmitted. The sending tasks include customized transmission rates.

[0089] Generate a transfer task for delayed transmission data that has not yet reached the start time of transmission.

[0090] After determining the customized transmission rate for each data based on the transmission scheme for each data, a sending task is generated for real-time transmitted data, delayed transmitted data that has reached the start time of transmission, and data that is being transmitted. The sending task includes the customized transmission rate for each data and instructs the access POP point to send data in real time according to the customized transmission rate for each data.

[0091] For delayed transmission data that has not yet reached its start transmission time, no data transmission is required at the current time. The scheduling platform generates a transfer task for this data, instructing the access POP point to transfer the incoming data as delayed transmission data. When the start transmission time for this data arrives, the scheduling platform generates a transmission task, instructing the access POP point to retrieve the data from the storage platform and perform data transmission.

[0092] The scheduling platform formulates transmission plans for various data types and generates and issues data transmission tasks based on these plans. It instructs access points (POPs) to transmit or transfer data in real time, enabling precise control of data transmission, effective management of data transfer and storage, and optimization of data transmission accuracy and timeliness. Data with different latency requirements is transmitted at different times, especially during off-peak network periods, which fully utilizes idle internet bandwidth to achieve low-cost, high-efficiency data transmission.

[0093] In one exemplary embodiment, the data transmission method further includes, after formulating a transmission scheme for each data item:

[0094] Based on real-time or historical bandwidth data, determine whether the available bandwidth meets the requirements of the data transmission scheme.

[0095] If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth meets the data transmission scheme requirements.

[0096] The scheduling platform can collect real-time bandwidth data from all POPs and calculate available bandwidth accordingly. It can also estimate available bandwidth at a future time based on historical bandwidth data from all POPs at different historical moments.

[0097] Idle bandwidth is the minimum available bandwidth among all POPs in the transmission path at a given time. When data needs to be transmitted through multiple POPs, the scheduling platform simultaneously calculates the bandwidth of all POPs along the path. This allows it to select a transmission path with higher available bandwidth for data transmission, maximizing bandwidth utilization and improving data transmission speed and stability. Furthermore, it can dynamically select the optimal transmission path based on bandwidth changes and network conditions, ensuring data transmission efficiency and reliability.

[0098] After formulating the transmission schemes for each data item, the scheduling platform also needs to determine whether the available bandwidth meets the requirements of the data transmission scheme based on real-time bandwidth or historical bandwidth data.

[0099] Assuming the preset latency threshold is 5 seconds, and data A has a latency requirement of 8 hours, exceeding the preset threshold, the data is marked as delayed transmission data. If the current time is 13:00:00, the size of this delayed transmission data is 5MB. Furthermore, based on historical statistics, the idle bandwidth around 21:00 is 5-10MB, so the customized transmission rate for this delayed data can be determined to be 0.5MB, with transmission starting at 22:59:00.

[0100] For example, after customizing the data transmission plan above, the scheduling platform queries the existing plans and finds that at 22:59:00, multiple data points need to be transmitted simultaneously, and the sum of the customized transmission rates for each data point exceeds the available bandwidth around 21:00. Clearly, if multiple data points are transmitted simultaneously according to the existing transmission plans, the available bandwidth will be exhausted, preventing timely transmission of each data point; that is, the available bandwidth does not meet the requirements of the data transmission plan. In this case, the transmission time of data A can be adjusted. For example, the start time of data A's transmission can be adjusted to 22:50:00, with a customized transmission rate of 0.1M. By reducing the customized transmission rate and adjusting the start time, the situation of exhausting the available bandwidth by transmitting multiple data points at the same time can be avoided.

[0101] For example, after customizing the data transmission scheme, the scheduling platform queries the scheme and finds that the available bandwidth at 22:59:00 meets the requirements. A transmission task is then sent to the access point (POP). However, at 22:59, a transmission request for data B is received. Data B's latency requirement is less than a preset threshold, meaning it needs to be transmitted in real-time. The sum of the amount of new data B per second and the customized transmission rate of data A exceeds the available bandwidth at that moment, meaning the available bandwidth does not meet the data transmission scheme's requirements. In this case, the customized transmission rate of data A can be reduced to allow the available bandwidth to support the normal transmission of both data A and data B; alternatively, the transmission of data A can be interrupted to allow the available bandwidth to support the normal transmission of data B. After data B is transmitted, data A can then be transmitted.

[0102] Based on the latency requirements of different data, the start time of each data transmission is reasonably allocated, and the transmission rate of each data is customized according to the amount of idle bandwidth, so as to make reasonable use of bandwidth resources and reduce bandwidth costs.

[0103] In one exemplary embodiment, an alarm message is sent when the idle bandwidth does not meet the adjusted data transmission scheme.

[0104] If the available bandwidth still cannot meet the adjusted data transmission scheme, the scheduling platform sends an alarm message, at which point manual intervention can be implemented for timely adjustments. For example, it can determine whether the latency requirements for delayed data transmission are reasonable, whether real-time data transmission can be modified to delayed data transmission, and whether the existing network bandwidth meets the requirements. By adjusting the data latency or increasing the bandwidth capacity of each POP point in the existing network, the available bandwidth can be made to meet the requirements of the data transmission scheme, thereby improving data transmission efficiency.

[0105] Figure 3 This is a flowchart illustrating a data transmission method according to an exemplary embodiment. (Reference) Figure 3 Data transmission methods are applied to access points (POPs) in transmission networks, including:

[0106] Step S32: Receive data transmission task sent by the scheduling platform. The data transmission task includes a sending task or a transfer task.

[0107] Data is sent according to the sending task, or data is processed according to the transfer task.

[0108] The scheduling platform formulates a transmission plan for each data point based on its parameters. Data with latency requirements less than a preset threshold is marked as real-time transmission data; data with latency requirements greater than or equal to the preset threshold is marked as delayed transmission data.

[0109] For real-time data transmission, delayed data transmission that has reached its start time, and data currently being transmitted, the scheduling platform generates transmission tasks. Each transmission task includes a customized transmission rate for each data point. Access points (POPs) then transmit each data point according to the customized transmission rate specified in the transmission task.

[0110] For delayed data transmission, the scheduling platform generates a transfer task and sends it to the access POP point. The access POP point then transfers the corresponding data according to the transfer task.

[0111] The access point (POP) can send or transfer data with different latency requirements according to the data transmission task, thereby effectively managing data transfer and storage and precisely controlling data transmission.

[0112] In one exemplary embodiment, the sending task includes a customized transmission rate, and sending data according to the sending task includes:

[0113] Obtain the current available bandwidth. If the available bandwidth is greater than the preset bandwidth value, send data using the second transmission rate, which is greater than the customized rate. If the available bandwidth is less than or equal to the preset bandwidth value, send data using the customized transmission rate.

[0114] The scheduling platform can collect real-time bandwidth data from all Points of Presence (POPs) and calculate available bandwidth accordingly. Accessing POPs can obtain the available bandwidth for their current data transmission path from the scheduling platform. When the available bandwidth is greater than a preset bandwidth value, data is sent at a second transmission rate greater than the customized rate, improving data transmission efficiency, optimizing data transmission timeliness, fully utilizing available bandwidth, and increasing bandwidth resource utilization. When the available bandwidth is less than or equal to the preset bandwidth value, data is sent at the customized transmission rate to ensure the normal transmission of other data and to ensure that the data is transmitted in a timely and accurate manner.

[0115] When the actual transmission rate is lower than the customized transmission rate, a new transmission path is selected. If the actual transmission rate is lower than the customized transmission rate, it indicates that the current path is congested or faulty, and cannot guarantee normal data transmission. Since the customized transmission rate is the minimum transmission rate required to ensure successful data transmission to the receiver within the specified latency period, an actual transmission rate lower than the customized rate will prevent data from being sent to the receiver within the required latency period. The access point (POP) selects a new transmission path and sends data at a higher transmission rate to ensure timely delivery to the receiver. This method can serve as a disaster recovery mode, allowing for timely switching of the transmission path when a selected path fails.

[0116] In one exemplary embodiment, processing the data to be sent according to the transfer task includes:

[0117] The data is transferred to a storage platform or the storage location of the data source is recorded, and the data is marked as "pending transmission". After receiving the transfer task from the scheduling platform, the access POP point transfers the received data to the storage platform according to the instructions of the transfer task, or records the storage location of the data source on the storage platform and marks the data as "pending transmission". When the start time for data transmission is reached, the access POP point will receive a transmission task for the data from the scheduling platform. According to the transmission task, the access POP point retrieves the transferred data from the storage platform and transmits the data based on the idle bandwidth and the customized transmission rate indicated by the transmission task.

[0118] Through the above embodiments, the data transmission method provided in this application can achieve low-cost, high-efficiency transmission of massive amounts of data based on the existing Internet; intelligent data transmission is performed according to data latency requirements and data packet size or the amount of new data per second, taking into account idle bandwidth, to achieve optimal cost-effectiveness. In practical applications, transmission costs will be reduced by more than 30%; multiple transmission rates can be used to achieve data transmission, making full use of idle bandwidth and improving the utilization rate of bandwidth resources; automatic disaster recovery and intelligent selection of transmission paths are also achieved.

[0119] Figure 4This is a block diagram illustrating a data transmission apparatus according to an exemplary embodiment. (Reference) Figure 4 The data transmission device is used in the scheduling platform and includes a parameter acquisition module 401, a transmission scheme formulation module 402, and a task distribution module 403.

[0120] The parameter acquisition module 401 is configured to acquire parameters for various data.

[0121] The transmission scheme formulation module 402 is configured to formulate a transmission scheme for each piece of data based on the parameters of the data. The transmission scheme includes a start transmission time and a customized transmission rate.

[0122] The task distribution module 403 is configured to generate data transmission tasks based on the transmission scheme of each data, and distribute the data transmission tasks to the access POP points of the transmission network so that the access POP points can process each data according to the data transmission tasks.

[0123] In one exemplary embodiment, the parameters of the data include latency requirements, packet size, or the amount of new data per second;

[0124] The transmission scheme formulation module 402 is also configured to mark data as real-time transmission data when the data latency requirement is less than a preset threshold, and to set a customized data transmission rate based on the latency requirement and data packet size or the amount of data added per second.

[0125] When the latency requirement of the data is greater than or equal to the preset threshold, the data is marked as delayed transmission data, and the start time of delayed transmission data and the customized transmission rate are determined according to the latency requirement and the size of the data packet or the amount of new data per second.

[0126] In an exemplary embodiment, the transmission scheme formulation module 402 is further configured to determine whether the idle bandwidth meets the requirements of the data transmission scheme based on real-time bandwidth or historical bandwidth data.

[0127] If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth can meet the data transmission scheme.

[0128] In one exemplary embodiment, the transmission scheme formulation module 402 is further configured to send an alarm message when the idle bandwidth does not meet the adjusted data transmission scheme.

[0129] In an exemplary embodiment, the task dispatch module 403 is further configured to generate a sending task for the real-time transmitted data, the delayed transmitted data that has reached the start transmission time, and the data being transmitted, the sending task including the customized transmission rate;

[0130] A transfer task is generated for delayed transmission data that has not reached the start transmission time.

[0131] Figure 5 This is a block diagram illustrating a data transmission apparatus according to an exemplary embodiment. (Reference) Figure 5 The data transmission device is applied to the access point (POP) of the transmission network and includes a task receiving module 501 and a processing module 502.

[0132] The task receiving module 501 is configured to receive data transmission tasks sent by the scheduling platform. The data transmission tasks include sending tasks or transferring tasks.

[0133] The processing module 502 is configured to send data according to a sending task or process data according to a transfer task.

[0134] In one exemplary embodiment, the processing module 502 is further configured to acquire the current idle bandwidth, and when the idle bandwidth is greater than a preset bandwidth value, send data using a second transmission rate, which is greater than a customized rate; and when the idle bandwidth is less than or equal to the preset bandwidth value, send data using a customized transmission rate.

[0135] When the actual transmission rate is less than the customized transmission rate, a new transmission path is selected.

[0136] In one exemplary embodiment, the processing module 502 is further configured to transfer data to the storage location of a storage platform or a record data source according to the transfer task, and mark the data as "pending transfer".

[0137] Figure 6 This is a block diagram illustrating a computer device 600 for data transmission according to an exemplary embodiment. For example, the computer device 600 may be provided as a server. (Refer to...) Figure 6 The computer device 600 includes a processor 601, the number of which can be set to one or more as needed. The computer device 600 also includes a memory 602 for storing instructions executable by the processor 601, such as application programs. The number of memories can be set to one or more as needed. The stored application programs can be one or more. The processor 601 is configured to execute instructions to perform the aforementioned data transfer method.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data), including but not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0139] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0143] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0144] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations fall within the scope of the claims of this application and their equivalents, the intent of this application also includes these modifications and variations.

Claims

1. A method for data transmission, characterized in that, Applications in scheduling platforms include: Obtain the parameters for each data item; Based on the parameters of the data, a transmission scheme is formulated for each data item, and the transmission scheme includes the start transmission time and the customized transmission rate. Based on the transmission schemes for each data item, a data transmission task is generated and sent to the access point of the transmission network so that the access point processes each data item according to the data transmission task.

2. The data transmission method as described in claim 1, characterized in that, The parameters of the data include latency requirements, data packet size, or the amount of new data per second. The step of formulating a transmission scheme for each data item based on the data parameters includes: When the latency requirement of the data is less than a preset threshold, the data is marked as real-time transmission data, and a customized transmission rate of the data is set according to the latency requirement and the data packet size or the amount of data added per second. When the latency requirement of the data is greater than or equal to a preset threshold, the data is marked as delayed transmission data, and the start time of the delayed transmission data and the customized transmission rate are determined according to the latency requirement and the data packet size or the amount of data added per second.

3. The data transmission method as described in claim 2, characterized in that, Also includes: Based on real-time or historical bandwidth data, determine whether the available bandwidth meets the requirements of the data transmission scheme; If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth meets the requirements of the data transmission scheme.

4. The data transmission method as described in claim 3, characterized in that, If the available bandwidth does not meet the adjusted data transmission scheme, an alarm message is sent.

5. The data transmission method as described in claim 2, characterized in that, The data transmission scheme based on the various data includes generating a data transmission task as follows: A sending task is generated for the real-time transmitted data, the delayed transmitted data that has reached the start transmission time, and the data being transmitted. The sending task includes the customized transmission rate. A transfer task is generated for delayed transmission data that has not reached the start transmission time.

6. A method for data transmission, characterized in that, Access points (POPs) used in transmission networks include: Receive data transmission tasks sent by the scheduling platform, wherein the data transmission tasks include sending tasks or transferring tasks; Data is sent according to the sending task, or data is processed according to the transfer task.

7. The data transmission method as described in claim 6, characterized in that, The sending task includes the customized transmission rate, and the sending data according to the sending task includes: Obtain the current idle bandwidth; when the idle bandwidth is greater than a preset bandwidth value, send data using a second transmission rate, which is greater than the customized rate; when the idle bandwidth is less than or equal to the preset bandwidth value, send data using a customized transmission rate. When the actual transmission rate is less than the customized transmission rate, a new transmission path is selected.

8. The data transmission method as described in claim 6, characterized in that, The data to be sent is processed according to the aforementioned transfer task, including: Transfer the data to a storage platform or the storage location of the record data source, and mark the data as "pending transfer".

9. A data transmission apparatus, characterized in that, Applications in scheduling platforms include: The parameter acquisition module is used to acquire the parameters of each data item. A transmission scheme formulation module is used to formulate a transmission scheme for each piece of data based on the parameters of the data. The transmission scheme includes a start transmission time and a customized transmission rate. The task distribution module is used to generate data transmission tasks based on the transmission schemes of the various data, and distribute the data transmission tasks to the access POP points of the transmission network, so that the access POP points can process the various data according to the data transmission tasks.

10. The data transmission apparatus as described in claim 9, characterized in that, The parameters of the data include latency requirements, data packet size, or the amount of new data added per second; The transmission scheme formulation module is also used to mark the data as real-time transmission data when the latency requirement of the data is less than a preset threshold, and to set a customized transmission rate of the data according to the latency requirement and the data packet size or the amount of data added per second. When the latency requirement of the data is greater than or equal to a preset threshold, the data is marked as delayed transmission data, and the start time of the delayed transmission data and the customized transmission rate are determined according to the latency requirement and the data packet size or the amount of data added per second.

11. The data transmission apparatus as claimed in claim 10, characterized in that, The transmission scheme formulation module is also used to determine whether the idle bandwidth meets the requirements of the data transmission scheme based on real-time bandwidth or historical bandwidth data. If the available bandwidth does not meet the requirements of the data transmission scheme, adjustments can be made by reducing the customized transmission rate, adjusting the start time of transmission, or interrupting the transmission task, so that the available bandwidth meets the requirements of the data transmission scheme.

12. The data transmission apparatus as claimed in claim 11, characterized in that, The transmission scheme formulation module is also used to send an alarm message when the idle bandwidth does not meet the adjusted data transmission scheme.

13. The data transmission apparatus as claimed in claim 10, characterized in that, The task distribution module is also used to generate sending tasks for the real-time transmitted data, the delayed transmitted data that has reached the start transmission time, and the data being transmitted, and the sending tasks include the customized transmission rate; A transfer task is generated for delayed transmission data that has not reached the start transmission time.

14. A data transmission apparatus, characterized in that, Access points (POPs) used in transmission networks include: The task receiving module is used to receive data transmission tasks sent by the scheduling platform, wherein the data transmission tasks include sending tasks or transferring tasks. The processing module is used to send data according to the sending task, or to process data according to the transfer task.

15. The data transmission apparatus as claimed in claim 14, characterized in that, The processing module is also used to obtain the current idle bandwidth; when the idle bandwidth is greater than the preset bandwidth value, data is sent using the second transmission rate, which is greater than the customized rate; when the idle bandwidth is less than or equal to the preset bandwidth value, data is sent using the customized transmission rate. When the actual transmission rate is less than the customized transmission rate, a new transmission path is selected.

16. The data transmission apparatus as claimed in claim 14, characterized in that, The processing module is also used to transfer the data to the storage location of the storage platform or the record data source according to the transfer task, and mark the data as "pending transfer".

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1-8.

18. A computer device comprising a processor, a memory, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-8.

Images