Data transmission method and device, equipment, storage medium and program product

Abstract

The invention relates to a data transmission method and device, equipment, a storage medium and a program product. The method comprises the following steps: determining a service guarantee parameter of each data source according to a first data volume peak value of each data source; determining a sending strategy corresponding to each data source according to throughput parameters of a message bus and service guarantee parameters of each data source; and sending the sending strategy of each data source to a collector of each data source. By adopting the method, the reliability and the stability of acquiring the data of each data source by the message bus can be ensured.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a data transmission method, apparatus, device, storage medium and program product. Background Art

[0002] In the network and information security operation system, the "detection" link is the predecessor of the "response and disposal" link. Accurate and rapid "detection" of the collected data will enable efficient and rapid "response and disposal" later. The accuracy and speed of detection depend on the stability and reliability of the message bus in collecting data from the data source.

[0003] In the related art, when the message bus collects data from different data sources, each data source usually sends as much data generated by itself as possible to the message bus, so that the message bus can collaborate with other devices to perform subsequent data detection and other processing operations.

[0004] However, the above technology is difficult to ensure the reliability of the message bus in collecting data from various data sources in some cases. Summary of the invention

[0005] The embodiments of the present application provide a data transmission method, apparatus, device, storage medium and program product, which can ensure the reliability and stability of the message bus in collecting data from various data sources.

[0006] In a first aspect, the present application provides a data transmission method, the method comprising:

[0007] Determine the service guarantee parameters for each data source according to the first data volume peak value of each data source;

[0008] Determine the sending strategy for each data source based on the throughput parameters of the message bus and the service guarantee parameters of each data source;

[0009] The sending strategy of each data source is sent to the collector of each data source.

[0010] In one of the embodiments, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0011] In one of the embodiments, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0012] In one of the embodiments, the second data throughput includes a maximum throughput of the message bus.

[0013] In one of the embodiments, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0014] In one of the embodiments, the sending strategy includes the amount of data sent by the data source.

[0015] In one of the embodiments, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0016] In one embodiment, determining the service guarantee parameter of each data source according to the first data volume peak value of each data source includes:

[0017] Determine the importance label for each data source;

[0018] A service guarantee parameter for each data source is determined according to the importance label of each data source and the first data volume peak value of each data source.

[0019] In one embodiment, determining the importance label of each data source includes:

[0020] Each data source is categorized according to the system's goals and an importance label is assigned to each data source.

[0021] In one embodiment, determining the service guarantee parameter of each data source according to the importance label of each data source and the first data volume peak value of each data source includes:

[0022] The service guarantee parameter of each data source is calculated according to the first data volume peak value of each data source and the importance label of each data source as a weight.

[0023] In one embodiment, determining the service guarantee parameter of each data source according to the importance label of each data source and the first data volume peak value of each data source includes:

[0024] Substituting the importance label of each data source and the first data volume peak value of each data source into a preset relationship formula to obtain a service guarantee parameter of each data source;

[0025] The above preset relationship includes the mapping relationship between the importance label variable, the data volume peak variable and the service guarantee parameter variable.

[0026] In one embodiment, the importance tag is used to characterize the tolerance of data delay or loss of the data source.

[0027] In one embodiment, the method further comprises:

[0028] Make a theoretical estimate of the message bus, or perform a stress test on the message bus to determine the throughput parameters of the message bus.

[0029] In one embodiment, the above theoretical estimation of the message bus includes:

[0030] A theoretical estimate of the message bus is made based on its computing power and bandwidth configuration.

[0031] In one embodiment, the method further comprises:

[0032] Divide the set time period into multiple time slices;

[0033] According to the data volume of each data source in each time slice, the sending rate of each data source in each time slice is determined, and the sending rate set corresponding to each data source is obtained;

[0034] A first data volume peak value corresponding to each data source is determined according to each sending rate set.

[0035] In one of the embodiments, the sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0036] In one embodiment, the method further comprises:

[0037] When there are major changes in the configuration of the message bus, the type of data source, or the amount of data, the sending strategy of each data source is adjusted according to the service guarantee parameters of each data source to obtain a new sending strategy for each data source.

[0038] In a second aspect, the present application provides a data transmission method, the method comprising:

[0039] receiving a sending strategy sent by a control unit;

[0040] Transmit data to the message bus according to the sending strategy.

[0041] In one of the embodiments, the sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

[0042] In one of the embodiments, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0043] In one of the embodiments, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0044] In one of the embodiments, the second data throughput includes a maximum throughput of the message bus.

[0045] In one of the embodiments, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0046] In one of the embodiments, the sending strategy includes the amount of data sent by the data source.

[0047] In one of the embodiments, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0048] In a third aspect, the present application provides a data transmission device, the device comprising:

[0049] A first determination module is used to determine a service guarantee parameter for each data source according to a first data volume peak value of each data source;

[0050] The second determination module is used to determine the sending strategy corresponding to each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source;

[0051] The sending module is used to send the sending strategy of each data source to the collector of each data source.

[0052] In a fourth aspect, the present application provides a data transmission device, the device comprising:

[0053] A receiving module, used for receiving a sending strategy sent by a control unit;

[0054] The transmission module is used to transmit data to the message bus according to the sending strategy.

[0055] In a fifth aspect, the present application provides a control unit, comprising: a transmitter, a processor, and a memory, wherein the memory stores a computer program;

[0056] The processor executes a computer program to determine a service guarantee parameter for each data source according to a first data volume peak value of each data source; and determine a sending strategy corresponding to each data source according to a throughput parameter of the message bus and the service guarantee parameter of each data source;

[0057] The above-mentioned sender is used to send the sending strategy of each data source to the collector of each data source.

[0058] In a sixth aspect, the present application provides a collector, comprising: a transmitter and a receiver;

[0059] The above-mentioned receiver is used to receive the sending strategy sent by the control unit;

[0060] The above transmitter is used to transmit data to the message bus according to the sending strategy.

[0061] In a seventh aspect, the present application provides a data transmission system, including: a data source, a control unit, a collector, and a message bus;

[0062] The above data source is used to send the collected data to the collector;

[0063] The control unit is used to execute the steps of the method of the first aspect;

[0064] The collector is used to execute the steps of the method of the second aspect;

[0065] The above message bus is used to receive data transmitted by the collector.

[0066] In an eighth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of the first aspect and / or the second aspect described above.

[0067] In a ninth aspect, the present application provides a chip comprising a programmable logic circuit and / or program instructions, which can execute the steps of the method of the first aspect and / or the second aspect when the chip is running.

[0068] In a tenth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method of the first aspect and / or the second aspect mentioned above.

[0069] The above-mentioned data transmission method, device, equipment, storage medium and program product determine the service guarantee parameters of each data source according to the first data volume peak value of each data source, and determine the corresponding sending strategy of each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source, and then send the sending strategy of each data source to the collector of each data source. In this method, since the sending strategy of each data source can be determined by the service guarantee parameters of each data source and the throughput parameters of the message bus and sent to the collector of each data source, the collector of each data source can send the corresponding data to the message bus according to its own sending strategy, ensuring that the message bus can always receive the data of each data source within a unit time or a certain time period, and realize the reliability and stability of the message bus in collecting data from each data source, thereby ensuring that the subsequent detection link can achieve fast and accurate data detection operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a schematic diagram of a data transmission system in one embodiment;

[0071] Figure 2 This is a schematic diagram showing a framework of a data transmission system in conventional technology;

[0072] Figure 3 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0073] Figure 4 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0074] Figure 5 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0075] Figure 6 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0076] Figure 7 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0077] Figure 8 is a structural block diagram of a data transmission device in one embodiment;

[0078] Fig. 9 is a structural block diagram of a data transmission device in another embodiment;

[0079] Fig.10 is an internal structure diagram of a control unit in one embodiment;

[0080] Fig.11 FIG. 4 is a diagram showing the internal structure of a collector in one embodiment. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0082] Figure 1 A data transmission system framework diagram for data transmission provided in an embodiment of the present application. Figure 1 As shown, the data transmission system includes a data source 102, a control unit 104, a collector 106 and a message bus 108; wherein the data source 102 is used to send the collected data to the collector 106;

[0083] The control unit 104 is used to determine the service guarantee parameter of each data source according to the first data volume peak value of each data source, and determine the sending strategy corresponding to each data source according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and send it to the collector 106 of each data source 102;

[0084] Collector 106, used to send corresponding data to message bus 108 according to respective sending strategies;

[0085] The message bus 108 is used to receive data from each data source 102 transmitted by each collector 106 .

[0086] In addition, the data source 102 and the collector 106 can be transmitted via wired or wireless communication. The transmission protocols used for data transmission between each data source 102 and each collector 106 can be the same or different. The transmission protocols here can include, for example, SNMP protocol, Syslog protocol, etc. SNMP protocol refers to the Simple Network Management Protocol, and Syslog protocol refers to the System Log Protocol.

[0087] Data can be transmitted between the control unit 104 and the collector 106 via wired or wireless communication, and data can be transmitted between the collector 106 and the message bus 108 via wired or wireless communication.

[0088] It should be noted that the number of the above-mentioned data sources 102 and collectors 106 can be multiple, and generally each data source 102 corresponds to one collector 106. When constructing the above-mentioned data transmission system, the corresponding collector 106 can be configured for each data source 102 first, and then the message bus 108 can be configured, and each collector 106 can be written to the message bus 108, that is, each collector 106 can transmit data to the message bus 108, and each collector 106 can transmit the data in the corresponding data source 102 to the corresponding topic / Topic in the message bus 108; at the same time, a corresponding data sending strategy controller can be configured for each collector 106, which is used to count the data throughput peak of the corresponding data source within the historical set time period and control the data sending strategy.

[0089] At present, in the network and information security operation system, the "detection" link is the predecessor of the "response and disposal" link. Generally, the efficiency and speed of network threat disposal are strongly related to the detection efficiency and detection accuracy of the preceding "detection" link. Usually, accurate and rapid "detection" can achieve efficient and rapid "response and disposal", and accurate and rapid "detection" usually depends on the stability and reliability of data source collection. For example, in the existing comprehensive security operation systems such as SOAR (security orchestration, automation and response) and XDR (extended detection and response), there are requirements for the stability of data collection and the priority of each data source, but the actual system implementation often takes this into account. For example, when a certain type of data source increases dramatically, it is easy to lose key data sources within a fixed analysis time period, resulting in the failure of the system's detection function and excessive analysis delays, which ultimately makes it difficult to meet security operation needs.

[0090] In traditional technology, see Figure 2As shown in the example diagram, when the message bus collects data from different data sources, the data sources are usually equal, and each data source will send the data it generates to the message bus as much as possible, so that the message bus and other devices can collaborate to perform subsequent data detection and other processing operations. However, if the computing power of the message bus is insufficient, when a sudden high-intensity attack occurs, it is easy to cause unpredictable transmission delays or storage delays for some data sources, resulting in reduced reliability of data collection; and if the computing power of the message bus is redundantly configured with huge resources, it will face high resource waste and reduce the reliability of data collection. It can be seen that the above technology is difficult to guarantee the reliability of the message bus in collecting data from various data sources in some cases. Among them, Figure 2 The HDFS involved is a distributed file system.

[0091] Based on the above-mentioned traditional technology, an embodiment of the present application provides a data transmission method, in which the collectors of each data source send the corresponding data to the message bus according to their respective sending strategies within a unit time or a set time period, thereby ensuring that the message bus can always receive the data of each data source within a unit time, thereby achieving the reliability and stability of the message bus in collecting data from each data source.

[0092] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0093] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0094] The following embodiments firstly illustrate the data transmission method by taking the control unit as the execution subject.

[0095] In one embodiment, Figure 3 As shown, a data transmission method is provided, which is applied to Figure 1 Taking the control unit in as an example, the method may include the following steps:

[0096] S202: Determine a service guarantee parameter for each data source according to a first data volume peak value of each data source.

[0097] The number of each data source can be set according to actual conditions, for example, it can be n, where n is greater than or equal to 2. The data stored in each data source can generally be different types of data, such as business data, log data, etc. of a computer device / computer system, or different types of log data of a computer device / computer system. Different types of log data can include, for example: availability logs of load balancing / routing devices / servers, IDS / IPS / FW alarm logs, device login logs, etc. IDS refers to Intrusion Detection System, which means in Chinese for Intrusion Detection System, IPS refers to Intrusion Prevention System, which means in Chinese for Intrusion Prevention System, and FW refers to Firewall.

[0098] Here, the first data volume peak value of each data source can be the data volume peak value of each data source within a set longer period of time, or it can be the real-time data volume peak value of each data source, or it can be the data volume peak value of each data source per unit time. Of course, it can also be the data volume peak value in other situations, which is not specifically limited here.

[0099] Specifically, it may be that within a pre-set time period, statistics are collected / obtained for each data source, and here the data volume of the collected / obtained data, or the data transmission rate or efficiency of the collected / obtained data, etc., may be counted, and the first data volume peak value of each data source may be determined based on the statistical results of each data source. Of course, it may also be possible to obtain the first data volume peak value of each data source by collecting statistics on the data collected / obtained by each data source within a unit time; or other methods may be used. In addition, here, the first data volume peak value of each data source may be determined separately based on the statistical results of each data source itself, or the first data volume peak value of each data source may be determined after the statistical results of each data source are combined and processed, or the first data volume peak value of each data source may be determined using the statistical results of some data sources, or other situations may be used, which are not specifically limited here.

[0100] After obtaining the first data volume peak value of each data source, the first data volume peak values ​​can be averaged and the obtained average value can be used as the service guarantee parameter of each data source; or the first data volume peak values ​​can be summed and the proportion of each first data volume peak value in the sum of all first data volume peak values ​​can be calculated, and the obtained proportions are the service guarantee parameters of each data source; or the first data volume peak values ​​can be weighted and summed, and the first data volume peak values ​​can be weighted, and the proportions of each weighted first data volume peak value in the total weighted sum can be calculated, and the obtained proportions are the service guarantee parameters of each data source; or other methods can be used, which are not specifically limited here.

[0101] The service guarantee parameters of the above-mentioned data sources can be service guarantee parameters of each data source within a unit time, service guarantee parameters within a historical set period, or service guarantee parameters at each moment, etc. The service guarantee parameters can include QoS values, and of course other parameters. Here, QoS stands for Quality of Service. The QoS value of each data source within a unit time can represent the service quality that the message bus can provide to each data source within a unit time.

[0102] S204: Determine a sending strategy corresponding to each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source.

[0103] Among them, the throughput parameter of the message bus represents the data receiving capability or data transmission capability of the message bus, etc. The data receiving capability may represent the data receiving capability or data transmission capability of the message bus in a unit time, or may represent the data receiving capability or data transmission capability of the message bus in a certain time period, or may also represent other situations; the receiving capability or transmission capability here may represent the maximum receiving capability or maximum transmission capability of the message bus for data, or may also represent the average receiving capability or average transmission capability of the message bus for data.

[0104] The method for obtaining the throughput parameters of the message bus may be to obtain the throughput parameters of the message bus by counting or testing the data receiving capacity or data transmission capacity of the message bus within a unit time or a certain time period, or other obtaining methods may be used.

[0105] After determining the service guarantee parameters of each data source and the throughput parameters of the message bus as described above, by comparing the throughput parameters of the message bus with each service guarantee parameter, as an optional embodiment, each service guarantee parameter can be multiplied by the throughput parameters of the message bus to obtain the sending strategy of each data source in unit time.

[0106] The sending strategy here can be the strategy for the collector of each data source to send data to the message bus, which indicates the ability of the collector to write data to the message bus. For example, it may include the amount of data written by the collector to the message bus within a unit time or a set period of time, or the transmission efficiency or transmission rate of the data written by the collector to the message bus, or the writing method of the collector to the message bus, or what specific data the collector transmits to the message bus, etc.

[0107] For example, taking the service guarantee parameter of each data source as the QoS value, the throughput parameter of the message bus as the maximum throughput of the message bus in unit time, and the sending strategy including the maximum amount of data sent as an example, here the QoS value of each data source can be multiplied by the maximum throughput of the message bus to obtain the maximum amount of data sent by each data source in unit time.

[0108] In this way, the sending strategy of each data source can be determined more accurately through the service guarantee parameters, thereby improving the accuracy of subsequent message bus data collection.

[0109] S206: Send the sending strategy of each data source to the collector of each data source.

[0110] In this step, each data source corresponds to a collector, and each collector is mainly used to transmit the data in the corresponding data source to the message bus. After obtaining the sending strategy corresponding to each data source, the control unit can send each sending strategy to the collector corresponding to each data source according to the corresponding relationship between each data source and the corresponding collector, so that each collector can obtain the sending strategy that matches its own data source. As an optional embodiment, the sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0111] Generally, after collecting or acquiring data, each data source can send the collected or acquired data to the corresponding collector. The collector can generally include a message cache queue and a data sending strategy controller. Generally, the data collected or acquired by the data source can be first sent to the message cache queue in the collector. Each sending strategy is generally sent by the control unit to the data sending strategy controller in the corresponding collector. After that, each data sending strategy controller can perform strategy control on the data cached in the corresponding message cache queue in each unit time, or in a set period of time, or at each moment according to the received sending strategy, and send the data in each message cache queue after strategy control to the message bus, so that the message bus can collect, acquire or receive data from each data source in each unit time, or in a set period of time, or at each moment.

[0112] For example, for the current high-value SOAR, XDR and low-latency threat detection scenarios, the reliable transmission and caching of different related data sources within the same time analysis window (i.e., within unit time) is guaranteed. For example, multiple data sources such as the "system availability log" data source and the "border firewall alarm log" data source can always be transmitted within unit time. This ensures that comprehensive analysis can be performed through these different types of data sources, and decisions such as starting traffic cleaning can be made quickly and accurately.

[0113] As can be seen from the above description, the method of the embodiment of the present application can ensure that the message bus can always collect data from various data sources within a unit time. In this way, in the case of unpredictable business emergencies, it is possible to avoid the situation where a certain type of security data source is prone to flooding and easily blocking other important data sources, ensuring the reliability and stability of various data sources for collection, and then ensuring that subsequent detection links are carried out accurately and quickly, thereby meeting the system's security operation requirements. For example, in a security system with low-latency analysis requirements, it is possible to ensure that within the same time window, the system has the requirements of correlation analysis of multi-source heterogeneous data sources (such as different data sources), and the reliability and stability of the collection of multi-source heterogeneous data.

[0114] In the above data transmission method, the service guarantee parameters of each data source are determined according to the first data volume peak value of each data source, and the sending strategy corresponding to each data source is determined according to the throughput parameters of the message bus and the service guarantee parameters of each data source, and then the sending strategy of each data source is sent to the collector of each data source. In this method, since the sending strategy of each data source can be determined by the service guarantee parameters of each data source and the throughput parameters of the message bus and sent to the collector of each data source, the collector of each data source can send the corresponding data to the message bus according to its own sending strategy, ensuring that the message bus can always receive the data of each data source within a unit time or a certain time period, realizing the reliability and stability of the message bus in collecting data from each data source, and then ensuring that the subsequent detection link can achieve fast and accurate data detection operations.

[0115] The above embodiments mention the first data volume peak value of each data source, and the following embodiments illustrate the first data volume peak value.

[0116] In another embodiment, based on the above embodiment, the above first data volume peak value includes a historical first data throughput peak value of the data source.

[0117] Among them, the first data volume peak of each data source can include the data throughput peak of the data source within a historical set time period, which can be recorded as the historical first data throughput peak. Of course, it can also include other parameters, such as data packet loss rate, data throughput time, etc.

[0118] The set period refers to a set time period, which may be a set period composed of historical time. Generally, the set period includes at least one unit time, and the unit time here may be, for example, 1ms, 1s, 1min, 1hour, 1day, etc. The data throughput peak value of each data source in the set period may be recorded as a first data throughput peak value, and each first data throughput peak value is used to characterize the maximum data receiving and sending capability of the corresponding data source, or the maximum data transmission capability of the corresponding data source (such as transmission rate, transmission efficiency, etc.).

[0119] The historical first data throughput peak value of each data source may be the maximum data throughput of each data source within a historical set period, that is, the maximum amount of data that can be collected / acquired / transmitted by each data source within the set period.

[0120] In this embodiment, the first data volume peak of each data source includes the historical first data throughput peak of the data source, so that the maximum amount of data that each data source can collect / acquire / transmit within the historical set time period can be clearly defined, so that the sending strategy that matches each data source can be quickly and accurately determined based on the maximum data volume transmitted by each data source, thereby further improving the accuracy and efficiency of data transmission.

[0121] The above embodiments also mention the throughput parameters of the message bus. The following embodiments will illustrate the throughput parameters of the message bus.

[0122] In another embodiment, based on the above embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0123] Among them, the throughput parameters of the message bus mainly include the data throughput of the data bus in unit time. Here, the data throughput of the message bus can be recorded as the second data throughput. The unit time here is the same as the unit time mentioned above, such as 1ms, 1s, 1min, 1 hour, 1 day, etc.

[0124] The second data throughput of the message bus per unit time may be the amount of data successfully transmitted by the message bus per unit time (the amount here may be measured in bits, bytes, packets, etc.), or it may be the maximum data rate or average data rate that the message bus can receive and forward without data loss per unit time.

[0125] In this embodiment, the throughput parameter of the message bus includes the second data throughput of the message bus per unit time. In this way, the data throughput can simply and intuitively represent the reception or transmission of data by the message bus per unit time, which can facilitate the subsequent rapid and accurate adjustment of the amount of data sent by each data source to the message bus, ensuring that the message bus can always receive data from various data sources per unit time, and facilitating the subsequent comprehensive analysis and processing of heterogeneous data sources.

[0126] The following embodiments illustrate the specific meaning of the second data throughput of the message bus.

[0127] In another embodiment, based on the above embodiment, the second data throughput includes a maximum throughput of the message bus.

[0128] In this embodiment, the second data throughput of the message bus includes the maximum throughput of the message bus per unit time. The second data throughput limit value indicates the maximum amount of data that the message bus can actually receive / collect per unit time, and may also be the maximum amount of data that the message bus can theoretically receive / collect per unit time.

[0129] In this embodiment, the second data throughput of the message bus represents the maximum throughput of the message bus in unit time, so that the maximum amount of data that the message bus can receive / transmit in unit time can be clearly defined, so that the amount of data sent by each data source to the message bus can be adjusted in a timely and accurate manner in the future, thereby ensuring the reliability and stability of the message bus in collecting or receiving data from various data sources.

[0130] In another embodiment, based on the above embodiment, the throughput parameter of the message bus may include the maximum throughput of the message bus.

[0131] That is, the throughput parameter of the message bus may include the maximum throughput of the message bus within a unit time or a set period of time. For the explanation of the maximum throughput, please refer to the explanation of the above embodiment, which will not be repeated here.

[0132] In this embodiment, the throughput parameters of the message bus include the maximum throughput of the message bus. This makes it easy for the control unit to quickly calculate a sending strategy that matches the computing power of the message bus based on the maximum throughput of the message bus and the service guarantee parameters of each data source, thereby improving the efficiency and accuracy of determining the sending strategy.

[0133] The above embodiments mention the sending strategies of each data source within a unit time. The following embodiments illustrate the sending strategies.

[0134] In another embodiment, based on the above embodiment, the above sending strategy includes the amount of data sent by the data source.

[0135] Each sending strategy is a strategy sent by the message bus to a corresponding collector. Each collector has a corresponding data source and will send the data in the corresponding data source to the message bus.

[0136] Each sending strategy may include the sending data volume of the corresponding data source, which may be the sending data volume per unit time, the sending data volume within a set period of time, or the sending data volume at each moment.

[0137] The amount of data sent by each data source can be the amount of data sent by the corresponding data source to its own collector, or the amount of data sent by the collector of the corresponding data source to the message bus. The amount of data sent here can be the maximum amount of data of each data source, or the average amount of data, etc.

[0138] In this embodiment, the sending strategy of each data source includes the amount of data sent by the data source. In this way, the amount of data sent can simply and intuitively represent the amount of data sent by each data source or each collector to the message bus, so that the subsequent message bus can always receive data from various data sources, thereby ensuring the reliability and stability of the message bus in collecting data from various data sources.

[0139] The following embodiments illustrate the specific meaning of the amount of data sent by each of the above data sources.

[0140] In another embodiment, based on the above embodiment, the amount of data sent by the above data source includes the maximum amount of data sent by the data source within a unit time.

[0141] The amount of data sent by each data source includes the maximum amount of data sent by the corresponding data source within a unit time, that is, the maximum amount of data that each data source or the collector corresponding to each data source can transmit to the message bus within a unit time.

[0142] It should be noted that each collector can adopt a corresponding sending strategy to transmit the data of the data source cached in the message cache queue to the message bus based on the corresponding maximum amount of data sent; as an optional embodiment, the sending strategy of each data source can include two parts: the first part is the maximum amount of data sent by each data source per unit time, which can be calculated by calculating the product of its service guarantee parameter and the message bus throughput parameter; the second part can be to require each data collector to send at least one log to indicate that there is no log, so as to distinguish it from unpredictable data interruptions.

[0143] In this embodiment, the amount of data sent by each data source includes the maximum amount of data sent by the data source per unit time. In this way, it can be made clear that each data source can send the maximum amount of data to the message bus per unit time, so that the corresponding collectors of each subsequent data source can subsequently transmit the corresponding data to the message bus quickly and accurately based on the maximum amount of data sent, thereby improving the efficiency and accuracy of data transmission to the message bus.

[0144] The above embodiment briefly describes how to determine the sending strategy of each data source based on the first data volume peak of each data source and the throughput parameter of the message bus. The following embodiment describes a possible implementation method of how to determine the sending strategy of each data source.

[0145] The above embodiment mentioned that the corresponding service guarantee parameters can be determined based on the first data volume peak value of each data source. The following embodiment describes a possible implementation method of how to determine the service guarantee parameters of each data source.

[0146] In another embodiment, Figure 4 As shown, another data transmission method is provided. Based on the above embodiment, the above S202 may include the following steps:

[0147] S302, determining the importance label of each data source.

[0148] In this step, as an optional embodiment, each data source can be classified according to the goal of the system, and an importance label can be assigned to each data source. The above-mentioned importance label is used to characterize the tolerance of data delay or loss of the data source. The goal of the system here can be the functional goal of the system, for example, it can be which data sources must have data and cannot be delayed to achieve a certain system functional goal, that is, the tolerance to data loss or delay. Data loss refers to packet loss, frame error, etc. that occur during data transmission. Data delay refers to the time delay for the data source to reach the message bus; the data source's tolerance to data loss or delay can reflect whether the data source can complete the subsequent corresponding business when data is missing or delayed.

[0149] The system's goal of tolerant each data source to data loss can be used to set a corresponding importance label for each data source. Here, the importance label of each data source can be expressed in the form of weights. The importance labels of each data source can be the same or different. For example, the importance labels can include different labels such as: very important, important, and ordinary. The importance label of each data source is expressed in the form of weights. Generally, the sum of the weights of each data source can be 1.

[0150] For example, assume that there are only three categories of importance labels, such as "very important", "important", and "ordinary". These three importance labels correspond to three different weight values, denoted as: a, b, c, where a+b+c=1. For example, a can be 0.5, b can be 0.3, and c can be 0.2.

[0151] S304: Determine a service guarantee parameter for each data source according to the importance label of each data source and the first data volume peak value of each data source.

[0152] In this step, after obtaining the importance label of each data source, the importance label of each data source can be mathematically processed with the corresponding first data volume peak value to obtain the service guarantee parameters of each data source. For example, the proportion of the importance label of each data source in all importance labels can be calculated, and then multiplied by the respective first data volume peak value to obtain the service guarantee parameters of each data source; or the proportion of each first data volume peak value in all first data volume peak values ​​can be calculated, and then multiplied by the respective importance label to obtain the service guarantee parameters of each data source; or the proportion of each first data volume peak value and the weighted importance label in all weighted first data volume peak values ​​can be calculated to obtain the service guarantee parameters of each data source.

[0153] As an optional embodiment, the service guarantee parameter of each data source may be calculated based on the first data volume peak value of each data source and the importance label of each data source as a weight, so that the service guarantee parameter of each data source can be obtained more accurately and quickly.

[0154] In this embodiment, the importance label of each data source is determined, and the respective service guarantee parameters are obtained based on the importance label of each data source combined with the corresponding first data volume peak value. Here, the respective service guarantee parameters are determined by combining the importance label of each data source, so that the final determined service guarantee parameters are more in line with the actual business scenario and can better meet the actual business needs.

[0155] The above embodiment mentioned that the corresponding service guarantee parameters can be determined based on the importance label of each data source. The following embodiment describes a possible implementation method of how to determine the service guarantee parameters of each data source based on the importance label of each data source.

[0156] In another embodiment, another data transmission method is provided. Based on the above embodiment, the above S304 may include the following steps:

[0157] Step A, substitute the importance label of each data source and the first data volume peak value of each data source into a preset relationship formula to obtain the service guarantee parameters of each data source; the above preset relationship formula includes the mapping relationship between the importance label variable, the data volume peak value variable and the service guarantee parameter variable.

[0158] A possible way of presetting the relationship can be shown in the following formula (1):

[0159]

[0160] Where si represents the i-th data source, n is the total number of data sources, generally greater than or equal to 1; QoS _si is the service guarantee parameter variable, which represents the QoS value of the i-th data source; tag _si is an importance label variable, which indicates the importance label of the i-th data source (e.g. tag _si ∈{a,b,c}); V _si is the data volume peak value variable, indicating the first data volume peak value of the i-th data source.

[0161] That is to say, according to the above formula (1), when calculating the service guarantee parameters of each data source, that is, the service quality value, it can be specifically performed by performing a weighted sum operation on each first data volume peak value and the corresponding importance label to obtain the sum data volume peak value, and after multiplying each first data volume peak value and the corresponding importance label, they are divided by the sum data volume peak value to obtain each service guarantee parameter.

[0162] It should be noted that the preset relationship formula for calculating each service guarantee parameter can be the above formula (1) or any modified formula of formula (1).

[0163] In this embodiment, by inputting the importance label of each data source and the first data volume peak into a preset relationship formula, the corresponding service guarantee parameters can be obtained. The preset relationship formula includes a mapping relationship between importance label variables, data volume peak variables and service guarantee parameter variables. In this way, the service guarantee parameters of each data source can be quickly and accurately determined through the preset relationship formula, thereby improving the efficiency and accuracy of the service guarantee parameter calculation.

[0164] The following embodiment describes a possible implementation method of determining the throughput parameter of the message bus mentioned above.

[0165] In another embodiment, another data transmission method is provided. Based on the above embodiment, the above method may further include the following steps:

[0166] Make a theoretical estimate of the message bus, or perform a stress test on the message bus to determine the throughput parameters of the message bus.

[0167] In this step, the message bus can be theoretically estimated based on the existing configuration of the message bus, the maximum throughput of the message bus in a unit time can be determined, and the throughput parameters of the message bus can be obtained. Alternatively, the message bus can be subjected to a stress test (also called a performance test), which can be specifically to continuously apply pressure to the message bus, forcing the message bus to send and receive data under extreme conditions, and to collect statistics on the maximum throughput of the message bus in a unit time under extreme conditions, so as to obtain the throughput parameters of the message bus.

[0168] In this embodiment, the throughput parameters of the message bus are obtained by performing theoretical estimation or stress testing on the message bus, so that the throughput parameters of the message bus can be determined relatively quickly and accurately, thereby improving the efficiency and accuracy of subsequent data transmission.

[0169] Furthermore, the theoretical estimation of the message bus may include: based on the computing capability and bandwidth configuration of the message bus, the theoretical estimation of the message bus.

[0170] The computing capacity refers to the number of cores that are fixed for data processing included in the message bus, such as the number of CPUs and GPUs included in the message bus. CPU stands for Central Processing Unit, and GPU stands for Graphics Processing Unit; or the computing capacity may also include the fixed memory capacity of the message bus, where the memory capacity may include, for example, the storage capacity of a solid-state drive, a disk, a message bus memory, etc. Bandwidth configuration refers to the frequency range that the message bus can process simultaneously, which may be the maximum frequency range, etc.

[0171] In this step, the computing power and bandwidth configuration of the message bus can be determined in advance. Then, under the relevant conditions such as the computing power and bandwidth configuration of the message bus, the maximum throughput of the message bus in unit time can be theoretically estimated according to the relevant theoretical calculation formula to obtain the throughput parameters of the message bus.

[0172] In this embodiment, the throughput parameters of the message bus are obtained by theoretically estimating the message bus under the conditions of computing power and bandwidth configuration. In this way, the throughput parameters of the message bus can be determined relatively quickly and accurately under the bandwidth configuration and computing power, thereby improving the efficiency and accuracy of subsequent data transmission.

[0173] Furthermore, when there are significant changes in the configuration of the message bus, the type of data source, or the amount of data, the sending strategy of each data source is adjusted according to the service guarantee parameters of each data source to obtain a new sending strategy for each data source.

[0174] That is to say, when there are major changes in the computing power and bandwidth configuration of the message bus, or the types of data sources increase or decrease, or a large amount of data suddenly appears, the sending strategy of the existing data source can be recalculated according to the current actual situation to obtain a new sending strategy for each existing data. In this way, the sending strategy can be adjusted for each data source in real time to ensure the flexibility of data sending and better meet the actual data transmission scenario requirements.

[0175] The following embodiment describes a possible implementation method of determining the first data volume peak value of each data source mentioned above.

[0176] In another embodiment, Figure 5 As shown, another data transmission method is provided. Based on the above embodiment, the above method may further include the following steps:

[0177] S402: Divide the set time period into multiple time slices.

[0178] The time slice here can be the smallest time unit, such as unit time, or a time size determined according to actual conditions. The time slice here can generally be considered as a time interval or a duration, such as 30 seconds or 60 seconds, etc. The set duration here can be, for example, 24 hours or a week.

[0179] The number of time slices into which the set duration is divided can be set according to actual conditions. When dividing the time slices, the set duration can be equally divided into multiple time slices, or in some cases, it can be unequally divided into multiple time slices.

[0180] S404, determining the sending rate of each data source in each time slice according to the data volume of each data source in each time slice, and obtaining a sending rate set corresponding to each data source.

[0181] In this step, after obtaining each divided time slice, the amount of data obtained by each data source in each time slice may be counted. For example, the number of log data items obtained may be used as the data amount.

[0182] Taking a data source as an example, the amount of data of the data source in each time slice can be obtained, and then each data amount can be divided by the corresponding time slice. The quotient values ​​obtained are the corresponding sending rates of the data source in each time slice; then the sending rates of the data source are combined or arranged in chronological order to obtain the sending rate set corresponding to the data source.

[0183] The same operation is performed for other data sources, and then the sending rate set corresponding to each data source can be obtained.

[0184] S406: Determine a first data volume peak value corresponding to each data source according to each sending rate set.

[0185] In this step, after obtaining the sending rate set corresponding to each data source, the maximum sending rate or peak sending rate can be obtained from the sending rate set of each data source, and the maximum sending rate or peak sending rate found by each data source is used as its corresponding first data volume peak.

[0186] For example, taking data source s1, its minimum statistical time period or a time slice is t1, t1 can be 30 seconds or 60 seconds. By counting the number of data items n1 included in s1 within the t1 time period, the sending rate within the t1 time period is v1=n1 / t1.

[0187] In a longer historical time period T (i.e., a set period), such as 24 hours or a week, the data transmission rate of the data source s1 in each time slice can be calculated in the same way, and each data transmission rate can constitute a data transmission rate sequence [v1, v2, ..., vm], where m is the sequence number of the last time slice. Then, in the historical time period T, the peak transmission rate (or the maximum data transmission rate) of the data source s1 is V_s1 = argmax{v1, v2, ..., vm}.

[0188] Using the same calculation method, the peak sending rate of each data source in the same historical time period T can be obtained. For example, a peak sending rate sequence is obtained: [V_s1, V_s2, …, V_si, …, V_sn], where V_si is the peak sending rate of the i-th data source in the historical time period T, which can also be called the peak throughput value.

[0189] In this embodiment, the sending rate in the corresponding time slice is determined by the amount of data in multiple time slices divided by each data source in a set time period, and a sending rate set is obtained, and then the first data volume peak value of the data source is determined. In this way, the set time period is divided into multiple time slices for separate parameter statistics, which can improve the efficiency and accuracy of the data volume peak statistics of the data source, thereby ensuring that the subsequent control unit can quickly and accurately determine the corresponding sending strategy based on the data volume peak value of the data source.

[0190] Secondly, the following embodiments illustrate the data transmission method using a collector as the execution subject.

[0191] In another embodiment, Figure 6 As shown, another data transmission method is provided, which is applied to Figure 1 Taking the collector in as an example, the method may include the following steps:

[0192] S502: Receive a sending strategy sent by a control unit.

[0193] In this step, as an optional embodiment, the above-mentioned sending strategy is determined by the control unit according to the throughput parameters of the message bus and the service guarantee parameters of each data source, and the service guarantee parameters of each data source are determined according to the first data volume peak value of each data source.

[0194] Among them, the sending strategy received by each collector is determined by the control unit based on the service guarantee parameters of the corresponding data source and the throughput parameters of the message bus, which can represent the ability of the collector to write data to the message bus within a unit time.

[0195] For the specific meaning of each parameter and the execution process of the specific steps, please refer to the explanation of the corresponding method embodiment of the above-mentioned control unit, which will not be repeated here.

[0196] S504: Transmit data to the message bus according to the sending strategy.

[0197] In this step, for how each collector transmits data to the message bus according to the sending strategy, please refer to the explanation of the corresponding method embodiment of the above control unit, which will not be repeated here.

[0198] In this embodiment, the service guarantee parameters of each data source are determined according to the first data volume peak value of each data source, and the sending strategy corresponding to each data source is determined according to the throughput parameters of the message bus and the service guarantee parameters of each data source, and then the sending strategy of each data source is sent to the collector of each data source. In this method, since the sending strategy of each data source can be determined by the service guarantee parameters of each data source and the throughput parameters of the message bus and sent to the collector of each data source, the collector of each data source can send the corresponding data to the message bus according to its own sending strategy, ensuring that the message bus can always receive the data of each data source within a unit time or a certain time period, realizing the reliability and stability of the message bus in collecting data from each data source, and then ensuring that the subsequent detection link can achieve fast and accurate data detection operations.

[0199] In another embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0200] For the explanation of the first data volume peak value in this embodiment, reference may be made to the explanation in the above control unit side embodiment, which will not be repeated here.

[0201] In another embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0202] For the explanation of the throughput parameters in this embodiment, reference may be made to the explanation in the above control unit side embodiment, which will not be repeated here.

[0203] In another embodiment, the second data throughput includes a maximum throughput of the message bus.

[0204] For the explanation of the maximum throughput in this embodiment, reference may be made to the explanation in the above control unit side embodiment, which will not be repeated here.

[0205] In another embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0206] For the explanation of the maximum throughput in this embodiment, reference may be made to the explanation in the above control unit side embodiment, which will not be repeated here.

[0207] In another embodiment, the sending strategy includes the amount of data sent by the data source.

[0208] For the explanation of the sending strategy in this embodiment, please refer to the explanation in the above control unit side embodiment, which will not be repeated here.

[0209] In another embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0210] For the explanation of the maximum amount of data to be sent in this embodiment, please refer to the explanation in the above-mentioned control unit side embodiment, which will not be repeated here.

[0211] In one embodiment, Figure 7 Provide a detailed flow chart of data transmission. Figure 7 As shown, the method comprises the following steps:

[0212] S1, the control unit performs a theoretical estimation or a stress test on the message bus to determine the maximum throughput of the message bus per unit time;

[0213] S2, dividing the set time period into multiple time slices;

[0214] S3, determining the sending rate of each data source in each time slice according to the amount of data of each data source in each time slice, and obtaining a sending rate set corresponding to each data source;

[0215] S4, determining a historical first data throughput peak value corresponding to each data source according to each sending rate set;

[0216] S5, determining the importance label corresponding to each data source; each importance label is used to characterize the tolerance of data delay or loss of the data source;

[0217] S6, substituting the importance label of each data source and the corresponding first historical data throughput peak value into a preset relational expression to obtain each service quality value; the preset relational expression includes a mapping relationship between an importance label variable, a data throughput peak value variable and a service quality value variable;

[0218] S7, multiplying the service quality value of each data source and the maximum throughput of the message bus to obtain the maximum amount of data sent by each data source per unit time;

[0219] S8, sending the maximum sending data volume of each data source to the corresponding collector respectively;

[0220] S9, each collector transmits the data in the data source to the message bus within a unit time according to the corresponding maximum data sending amount.

[0221] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0222] In one embodiment, Figure 8 As shown, a data transmission device is provided, including: a determination module and a sending module, wherein:

[0223] A first determination module is used to determine a service guarantee parameter for each data source according to a first data volume peak value of each data source;

[0224] The second determination module is used to determine the sending strategy corresponding to each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source;

[0225] The sending module is used to send the sending strategy of each data source to the collector of each data source.

[0226] Optionally, the above sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0227] In another embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0228] In another embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0229] In another embodiment, the second data throughput includes a maximum throughput of the message bus.

[0230] In another embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0231] In another embodiment, the sending strategy includes the amount of data sent by the data source.

[0232] In another embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0233] In another embodiment, another data transmission device is provided. Based on the above embodiment, the first determination module may include:

[0234] A label determination unit, used to determine the importance label of each data source;

[0235] The guarantee parameter determination unit is used to determine the service guarantee parameter of each data source according to the importance label of each data source and the first data volume peak value of each data source.

[0236] Optionally, the label determination unit is specifically configured to classify each data source according to a system objective and assign an importance label to each data source.

[0237] Optionally, the above-mentioned guarantee parameter determination unit is specifically used to calculate the service guarantee parameter of each data source according to the first data volume peak value of each data source and the importance label of each data source as a weight.

[0238] Optionally, the importance label is used to characterize the tolerance of data delay or loss of the data source.

[0239] Optionally, the above-mentioned guarantee parameter determination unit is specifically used to substitute the importance label of each data source and the first data volume peak value of each data source into a preset relationship formula to obtain the service guarantee parameters of each data source; the above-mentioned preset relationship formula includes the mapping relationship between the importance label variable, the data volume peak value variable and the service guarantee parameter variable.

[0240] In another embodiment, another data transmission device is provided. Based on the above embodiment, the above device may further include:

[0241] The test module is used to make theoretical estimates of the message bus, or to perform stress tests on the message bus to determine the throughput parameters of the message bus.

[0242] Optionally, the test module includes an estimation unit, which is used to perform a theoretical estimation on the message bus based on the computing capability and bandwidth configuration of the message bus.

[0243] In another embodiment, another data transmission device is provided. Based on the above embodiment, the above device may further include:

[0244] A division module is used to divide a set time period into multiple time slices;

[0245] A set determination module is used to determine the sending rate of each data source in each time slice according to the data volume of each data source in each time slice, and obtain a sending rate set corresponding to each data source;

[0246] The parameter determination module is used to determine the first data volume peak value corresponding to each data source according to each sending rate set.

[0247] In another embodiment, another data transmission device is provided. Based on the above embodiment, the above device may further include:

[0248] The adjustment module is used to adjust the sending strategy of each data source according to the service guarantee parameters of each data source when there are major changes in the configuration of the message bus, the type of data source or the amount of data, so as to obtain a new sending strategy for each data source.

[0249] In another embodiment, Fig. 9 As shown, another data transmission device is provided. Based on the above embodiment, the device may include: a receiving module and a transmission module, wherein:

[0250] A receiving module, used for receiving a sending strategy sent by a control unit;

[0251] The transmission module is used to transmit data to the message bus according to the sending strategy.

[0252] In another embodiment, the sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

[0253] In another embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0254] In another embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0255] In another embodiment, the second data throughput includes a maximum throughput of the message bus.

[0256] In another embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0257] In another embodiment, the sending strategy includes the amount of data sent by the data source.

[0258] In another embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0259] For the specific definition of the data transmission device, please refer to the definition of the data transmission method above, which will not be repeated here. Each module in the above-mentioned data transmission device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0260] In one embodiment, a control unit is provided. Fig.10 . Fig.10 It is a schematic diagram of the structure of a control unit provided in an embodiment of the present invention. Fig.10 The control unit 700 shown includes: at least one processor 701, a memory 702, at least one network interface 704 and a user interface 703. The various components in the control unit 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 705 is not shown in FIG. Figure 1 In the figure, various buses are labeled as bus system 705. In addition, in the embodiment of the present invention, a transceiver 706 is also included. The transceiver can be multiple components, that is, including a transmitter and a receiver, and is provided as a unit for communicating with various other devices on a transmission medium. Among them, the user interface 703 may include a display, a keyboard or a pointing device (for example, a mouse, a trackball, a touch pad or a touch screen, etc.).

[0261] It can be understood that the memory 702 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 702 of the system and method described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0262] In some implementations, the memory 702 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 7021 and an application 7022 .

[0263] The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 7022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 7022.

[0264] In an embodiment of the present invention, by calling the program or instructions stored in the memory 702, specifically, the program or instructions stored in the application 7022, the processor is used to determine the service guarantee parameters of each data source according to the first data volume peak of each data source; determine the sending strategy corresponding to each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source; the transmitter is used to send the sending strategy of each data source to the collector of each data source.

[0265] Part or all of the methods disclosed in the above embodiments of the present invention may also be applied to the processor 701, or implemented by the processor 701, or implemented by the processor 701 in cooperation with other components (such as a transceiver). The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 701 or by instructions in the form of software. The above processor 701 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and completes the steps of the above method in combination with its hardware.

[0266] It is to be understood that the embodiments described in the embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0267] For software implementation, the technology described in the embodiments of the present invention can be implemented by a module (such as a process, function, etc.) that performs the functions described in the embodiments of the present invention. The software code can be stored in a memory and executed by the processor 701. The memory can be implemented in the processor 701 or outside the processor 701.

[0268] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0269] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0270] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0271] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0272] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0273] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0274] In one embodiment, the processor is further used to determine an importance label for each data source; and determine a service guarantee parameter for each data source based on the importance label for each data source and a first data volume peak value for each data source.

[0275] In one embodiment, the processor is specifically configured to classify each data source according to a system goal and assign an importance label to each data source.

[0276] In one embodiment, the processor is specifically configured to calculate a service guarantee parameter for each data source according to a first data volume peak value of each data source and an importance label of each data source as a weight.

[0277] In one embodiment, the processor is specifically used to substitute the importance label of each data source and the first data volume peak value of each data source into a preset relationship formula to obtain the service guarantee parameter of each data source; the above preset relationship formula includes a mapping relationship between the importance label variable, the data volume peak value variable and the service guarantee parameter variable.

[0278] In one embodiment, the importance tag is used to characterize the tolerance of data delay or loss of the data source.

[0279] In one embodiment, the processor is further used to perform a theoretical estimation on the message bus, or perform a stress test on the message bus to determine a throughput parameter of the message bus.

[0280] In one embodiment, the processor is specifically configured to perform a theoretical estimation of the message bus based on the computing capability and bandwidth configuration of the message bus.

[0281] In one embodiment, the processor is further used to divide the set time period into multiple time slices; determine the sending rate of each data source in each time slice according to the amount of data of each data source in each time slice, and obtain the sending rate set corresponding to each data source; determine the first data volume peak value corresponding to each data source according to each sending rate set.

[0282] In one embodiment, the above sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0283] In one embodiment, the processor is further used to adjust the sending strategy of each data source according to the service guarantee parameters of each data source when there are significant changes in the configuration of the message bus, the type of data source or the amount of data, so as to obtain a new sending strategy for each data source.

[0284] In one embodiment, a collector is provided. Fig.11 . Fig.11 It is a schematic diagram of the structure of a collector provided by an embodiment of the present invention. Fig.11 The collector 800 shown includes: at least one processor 801, a memory 802, at least one network interface 804 and a user interface 803. The various components in the collector 800 are coupled together through a bus system 805. It can be understood that the bus system 805 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 805 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 805 is not shown in FIG. Figure 1In the figure, various buses are labeled as bus system 805. In addition, in the embodiment of the present invention, a transceiver 806 is also included. The transceiver can be multiple components, that is, it includes a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium. Among them, the user interface 803 may include a display, a keyboard or a pointing device (for example, a mouse, a trackball, a touch pad or a touch screen, etc.).

[0285] It can be understood that the memory 802 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 802 of the system and method described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0286] In some implementations, the memory 802 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 8021 and an application program 8022 .

[0287] The operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program 8022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application program 8022.

[0288] In an embodiment of the present invention, by calling the program or instructions stored in the memory 802, specifically, the program or instructions stored in the application 8022, the receiver is used to receive the sending strategy sent by the control unit; the processor is used to transmit data to the message bus according to the sending strategy.

[0289] Part or all of the methods disclosed in the above embodiments of the present invention may also be applied to the processor 801, or implemented by the processor 801, or implemented by the processor 801 in cooperation with other components (such as a transceiver). The processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 801 or by instructions in the form of software. The above processor 801 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 802, and the processor 801 reads the information in the memory 802 and completes the steps of the above method in combination with its hardware.

[0290] It is to be understood that the embodiments described in the embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0291] For software implementation, the technology described in the embodiments of the present invention can be implemented by a module (such as a process, function, etc.) that performs the functions described in the embodiments of the present invention. The software code can be stored in a memory and executed by the processor 801. The memory can be implemented in the processor 801 or outside the processor 801.

[0292] In one embodiment, the sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

[0293] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0294] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0295] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0296] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0297] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0298] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0299] An embodiment of the present application also provides a data transmission system, including: a data source, a control unit, a collector and a message bus; the above-mentioned data source is used to send the collected data to the collector; the above-mentioned control unit is used to execute the steps of the method on the control unit side; the above-mentioned collector is used to execute the steps of the method on the collector side; the above-mentioned message bus is used to receive data transmitted by the collector.

[0300] An embodiment of the present application also provides a chip, which includes a programmable logic circuit and / or program instructions, and when the chip is running, it can execute the steps of the above-mentioned data transmission method.

[0301] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0302] Determine the service guarantee parameters of each data source based on the first data volume peak of each data source; determine the sending strategy corresponding to each data source based on the throughput parameters of the message bus and the service guarantee parameters of each data source; and send the sending strategy of each data source to the collector of each data source.

[0303] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0304] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0305] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0306] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0307] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0308] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0309] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0310] Determine the importance label of each data source; and determine the service guarantee parameter of each data source based on the importance label of each data source and the first data volume peak value of each data source.

[0311] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0312] Each data source is categorized according to the system's goals and an importance label is assigned to each data source.

[0313] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0314] The service guarantee parameter of each data source is calculated according to the first data volume peak value of each data source and the importance label of each data source as a weight.

[0315] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0316] Substitute the importance label of each data source and the first data volume peak value of each data source into the preset relationship formula to obtain the service guarantee parameter of each data source; the above preset relationship formula includes the mapping relationship between the importance label variable, the data volume peak value variable and the service guarantee parameter variable.

[0317] In one embodiment, the importance tag is used to characterize the tolerance of data delay or loss of the data source.

[0318] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0319] Make a theoretical estimate of the message bus, or perform a stress test on the message bus to determine the throughput parameters of the message bus.

[0320] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0321] A theoretical estimate of the message bus is made based on its computing power and bandwidth configuration.

[0322] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0323] The set time period is divided into multiple time slices; according to the amount of data of each data source in each time slice, the sending rate of each data source in each time slice is determined to obtain the sending rate set corresponding to each data source; according to each sending rate set, the first data volume peak value corresponding to each data source is determined.

[0324] In one embodiment, the above sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0325] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0326] When there are major changes in the configuration of the message bus, the type of data source, or the amount of data, the sending strategy of each data source is adjusted according to the service guarantee parameters of each data source to obtain a new sending strategy for each data source.

[0327] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0328] The sending strategy sent by the receiving control unit is transmitted to the message bus according to the sending strategy.

[0329] In one embodiment, the sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

[0330] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0331] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0332] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0333] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0334] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0335] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0336] The present application also provides a computer program product including instructions, which, when executed on a computer, causes the computer to perform the following steps:

[0337] Determine the service guarantee parameters of each data source based on the first data volume peak of each data source; determine the sending strategy corresponding to each data source based on the throughput parameters of the message bus and the service guarantee parameters of each data source; and send the sending strategy of each data source to the collector of each data source.

[0338] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0339] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0340] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0341] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0342] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0343] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0344] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0345] Determine the importance label of each data source; and determine the service guarantee parameter of each data source based on the importance label of each data source and the first data volume peak value of each data source.

[0346] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0347] Each data source is categorized according to the system's goals and an importance label is assigned to each data source.

[0348] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0349] The service guarantee parameter of each data source is calculated according to the first data volume peak value of each data source and the importance label of each data source as a weight.

[0350] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0351] Substitute the importance label of each data source and the first data volume peak value of each data source into the preset relationship formula to obtain the service guarantee parameter of each data source; the above preset relationship formula includes the mapping relationship between the importance label variable, the data volume peak value variable and the service guarantee parameter variable.

[0352] In one embodiment, the importance tag is used to characterize the tolerance of data delay or loss of the data source.

[0353] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0354] Make a theoretical estimate of the message bus, or perform a stress test on the message bus to determine the throughput parameters of the message bus.

[0355] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0356] A theoretical estimate of the message bus is made based on its computing power and bandwidth configuration.

[0357] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0358] The set time period is divided into multiple time slices; according to the amount of data of each data source in each time slice, the sending rate of each data source in each time slice is determined to obtain the sending rate set corresponding to each data source; according to each sending rate set, the first data volume peak value corresponding to each data source is determined.

[0359] In one embodiment, the above sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

[0360] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0361] When there are major changes in the configuration of the message bus, the type of data source, or the amount of data, the sending strategy of each data source is adjusted according to the service guarantee parameters of each data source to obtain a new sending strategy for each data source.

[0362] The present application also provides a computer program product including instructions, which, when executed on a computer, causes the computer to perform the following steps:

[0363] The sending strategy sent by the receiving control unit is transmitted to the message bus according to the sending strategy.

[0364] In one embodiment, the sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

[0365] In one embodiment, the first data volume peak value includes a historical first data throughput peak value of the data source.

[0366] In one embodiment, the throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

[0367] In one embodiment, the second data throughput includes a maximum throughput of the message bus.

[0368] In one embodiment, the throughput parameter of the message bus includes the maximum throughput of the message bus.

[0369] In one embodiment, the sending strategy includes the amount of data sent by the data source.

[0370] In one embodiment, the amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

[0371] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0372] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0373] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data transmission method, characterized in that: The method comprises: Determining a service guarantee parameter for each data source according to a first data volume peak value for each data source; Determine a sending strategy corresponding to each data source according to the throughput parameters of the message bus and the service guarantee parameters of each data source; The sending strategy of each data source is sent to the collector of each data source.

2. The method according to claim 1, characterized in that: The first data volume peak value includes a historical first data throughput peak value of the data source.

3. The method according to claim 1, characterized in that: The throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

4. The method according to claim 3, characterized in that The second data throughput comprises a maximum throughput of the message bus.

5. The method according to claim 1, characterized in that The throughput parameter of the message bus includes the maximum throughput of the message bus.

6. The method according to claim 1, characterized in that The sending strategy includes the sending data volume of the data source.

7. The method according to claim 6, characterized in that The amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

8. The method according to claim 1, characterized in that The determining of the service guarantee parameter of each data source according to the first data volume peak value of each data source includes: determining a significance label for each of said data sources; A service guarantee parameter for each of the data sources is determined according to the importance label of each of the data sources and the first data volume peak value of each of the data sources.

9. The method according to claim 8, characterized in that Determining the importance label of each data source includes: Each of the data sources is categorized according to the goal of the system, and an importance label is assigned to each of the data sources.

10. The method according to claim 8, characterized in that Determining the service guarantee parameter of each data source according to the importance label of each data source and the first data volume peak value of each data source includes: The service guarantee parameter of each data source is calculated according to the first data volume peak value of each data source and the importance label of each data source as a weight.

11. The method according to claim 8, characterized in that Determining the service guarantee parameter of each data source according to the importance label of each data source and the first data volume peak value of each data source includes: Substituting the importance label of each data source and the first data volume peak value of each data source into a preset relationship formula to obtain a service guarantee parameter of each data source; The preset relationship includes a mapping relationship between an importance label variable, a data volume peak value variable, and a service guarantee parameter variable.

12. The method according to claim 8, characterized in that The importance tag is used to characterize the tolerance of data delay or loss of the data source.

13. The method according to claim 1, characterized in that The method further comprises: A theoretical estimate is made on the message bus, or a stress test is performed on the message bus to determine a throughput parameter of the message bus.

14. The method according to claim 13, characterized in that The theoretical estimation of the message bus includes: A theoretical estimation of the message bus is performed based on the computing power and bandwidth configuration of the message bus.

15. The method according to claim 1, characterized in that The method further comprises: Divide the set time period into multiple time slices; Determine the sending rate of each data source in each time slice according to the amount of data of each data source in each time slice, and obtain a sending rate set corresponding to each data source; A first data volume peak value corresponding to each of the data sources is determined according to each of the sending rate sets.

16. The method according to claim 1, characterized in that The sending strategy is used to instruct the collector to transmit data to the message bus according to the sending strategy.

17. The method according to claim 1, characterized in that The method further comprises: When there are significant changes in the configuration of the message bus, the type of data source or the amount of data, the sending strategy of each data source is adjusted according to the service guarantee parameters of each data source to obtain a new sending strategy for each data source.

18. A data transmission method, characterized in that: The method comprises: receiving a sending strategy sent by a control unit; The data is transmitted to the message bus according to the sending strategy.

19. The method according to claim 18, characterized in that The sending strategy is determined by the control unit according to the throughput parameter of the message bus and the service guarantee parameter of each data source, and the service guarantee parameter of each data source is determined according to the first data volume peak value of each data source.

20. The method according to claim 19, characterized in that The first data volume peak value includes a historical first data throughput peak value of the data source.

21. The method according to claim 19, characterized in that The throughput parameter of the message bus includes a second data throughput of the message bus per unit time.

22. The method according to claim 21, characterized in that The second data throughput comprises a maximum throughput of the message bus.

23. The method according to claim 19, characterized in that The throughput parameter of the message bus includes the maximum throughput of the message bus.

24. The method according to any one of claims 19 to 23, characterized in that: The sending strategy includes the sending data volume of the data source.

25. The method according to claim 24, characterized in that The amount of data sent by the data source includes the maximum amount of data sent by the data source within a unit time.

26. A data transmission device, characterized in that: The device comprises: A first determination module is used to determine a service guarantee parameter of each data source according to a first data volume peak value of each data source; A second determination module is used to determine a sending strategy corresponding to each data source according to a throughput parameter of the message bus and a service guarantee parameter of each data source; The sending module is used to send the sending strategy of each data source to the collector of each data source.

27. A data transmission device, characterized in that: The device comprises: A receiving module, used for receiving a sending strategy sent by a control unit; The transmission module is used to transmit data to the message bus according to the sending strategy.

28. A control unit, characterized in that: include: A transmitter, a processor and a memory, wherein the memory stores a computer program; The processor executes the computer program to determine the service guarantee parameter of each data source according to the first data volume peak value of each data source; determine the sending strategy corresponding to each data source according to the throughput parameter of the message bus and the service guarantee parameter of each data source; The transmitter is used to send the sending strategy of each data source to the collector of each data source.

29. A collector, characterized in that: include: Transmitters and receivers; The receiver is used to receive the sending strategy sent by the control unit; The transmitter is used to transmit data to the message bus according to the sending strategy.

30. A data transmission system, characterized in that: include: Data source, control unit, collector and message bus; The data source is used to send the collected data to the collector; The control unit is used to perform the steps of the method according to any one of claims 1 to 17; The collector is used to perform the steps of any one of the methods of claims 18-25; The message bus is used to receive data transmitted by the collector.

31. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 25 are implemented.

32. A chip, characterized in that: The chip comprises a programmable logic circuit and / or program instructions, and when the chip is run, the steps of the method according to any one of claims 1 to 25 are implemented.

33. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 25 are implemented.

Images