A method and device for automatically adjusting data processing time window

Abstract

The present application discloses a method for automatically adjusting a data processing time window, including obtaining the amount of data to be processed; judging whether the current data processing time window needs to be adjusted according to the amount of data to be processed and the standard data processing time window, wherein the standard data processing time window is a data processing time window set in advance for reference; if adjustment is required, a new data processing time window is determined according to a data processing time window record table set in advance, and the new data processing time window is used as the current data processing time window, wherein the data processing time window record table includes a corresponding relationship between the data processing time window and a single window processing capacity value. By applying the technical solution disclosed in the present application, the data processing time window can be dynamically adjusted according to the amount of data, which can not only cope with the processing of ordinary data, but also take corresponding strategies in time when the peak data arrives to ensure the smoothness of data processing.

Description

Technical Field

[0001] The present application relates to the field of big data processing technology, and in particular to a method and device for automatically adjusting a data processing time window. Background Art

[0002] With the development of e-commerce, people are becoming more and more accustomed to buying goods and conducting transactions online, and even shopping festivals have emerged to guide people to consume online. During these shopping concentrated time periods, thousands of clients will submit shopping orders from all directions almost at the same time. The data generated by these orders will impact the back-end system and big data computing platform on the network side like a flood, causing a data peak.

[0003] How to deal with such data spikes has become a more difficult problem in big data processing. With the rapid development of big data, business scenarios are becoming more and more complex. The offline batch processing framework MapReduce can no longer meet business needs. A large number of scenarios require real-time data processing results for analysis and decision-making. Real-time computing is usually based on a distributed parallel computing framework. A single machine is far from achieving the effect of real-time processing for a large amount of data in a short period of time. However, even under the distributed parallel computing framework, in the face of data spikes, existing technologies usually add servers or expand the capacity of existing servers to improve the backend system's ability to process big data. However, this method mainly increases hardware facilities, resulting in increased hardware and maintenance costs. Summary of the invention

[0004] The present application provides a method for automatically adjusting a data processing time window, which can cope with the processing of peak data without increasing hardware costs.

[0005] The method for automatically adjusting the data processing time window in this application includes:

[0006] Get the amount of data to be processed;

[0007] Determining whether the current data processing time window needs to be adjusted according to the amount of data to be processed and the standard data processing time window, wherein the standard data processing time window is a data processing time window set in advance for reference;

[0008] If adjustment is required, a new data processing time window is determined according to a pre-set data processing time window record table, and the new data processing time window is used as the current data processing time window. The data processing time window record table includes the correspondence between the data processing time window and the single window processing capacity value.

[0009] Furthermore, the step of obtaining the amount of data to be processed includes:

[0010] Get the current position of the data in the data queue;

[0011] Get the last position of the data in the recorded data queue;

[0012] The amount of data to be processed is determined according to the current position and the previous position.

[0013] Furthermore, the step of determining whether the current data processing time window needs to be adjusted according to the amount of data to be processed and the standard data processing time window includes:

[0014] According to the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, it is determined whether there is a backlog in the amount of data to be processed. If there is no backlog, the standard data processing time window is used as a new data processing time window; otherwise,

[0015] The ratio of the amount of data to be processed to the data processing capacity value corresponding to the standard data processing time window is calculated, and it is determined whether the current data processing time window needs to be adjusted according to the ratio result.

[0016] Furthermore, the data processing time window record table includes a standard data processing time window, an ultra-high data processing time window and a peak data processing time window, the ultra-high data processing time window is larger than the standard data processing time window, and the peak data processing time window is larger than the ultra-high data processing time window;

[0017] The corresponding relationship between the data processing time window and the single window processing capacity value includes:

[0018] Standard data processing time window, and the processing capacity value corresponding to a single standard data processing time window;

[0019] Ultra-high data processing time window, and the processing capacity value corresponding to a single ultra-high data processing time window;

[0020] The peak data processing time window, and the processing capacity value corresponding to a single peak data processing time window.

[0021] Furthermore, the step of determining a new data processing time window according to a pre-set data processing time window record table includes:

[0022] If the ratio result is greater than the preset first ratio threshold and less than the preset second ratio threshold, determining that the new data processing time window is an ultra-high data processing time window;

[0023] If the ratio result is greater than or equal to the preset second ratio threshold, the new data processing time window is determined to be the peak data processing time window.

[0024] The present application also provides a device for automatically adjusting the data processing time window, which can cope with the peak data without increasing the hardware cost. The device includes:

[0025] A data volume acquisition unit, used to acquire the volume of data to be processed;

[0026] a determination unit, for determining whether the current data processing time window needs to be adjusted according to the amount of data to be processed and the standard data processing time window, wherein the standard data processing time window is a data processing time window set in advance for reference;

[0027] The window adjustment unit determines a new data processing time window according to a pre-set data processing time window record table, and uses the new data processing time window as the current data processing time window. The data processing time window record table includes a correspondence between data processing time windows and single window processing capacity values.

[0028] Furthermore, the determination unit includes:

[0029] The first determining subunit is used to determine whether there is a backlog in the amount of data to be processed according to the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, and if there is no backlog, use the standard data processing time window as a new data processing time window; otherwise, trigger the second determining subunit;

[0030] The second determination subunit is used to calculate the ratio of the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, and determine whether the current data processing time window needs to be adjusted based on the ratio result, and trigger the window adjustment unit.

[0031] Furthermore, the window adjustment unit comprises:

[0032] A storage subunit, used for saving a data processing time window record table; the data processing time window record table includes a standard data processing time window, an ultra-high data processing time window and a peak data processing time window, the ultra-high data processing time window is larger than the standard data processing time window, and the peak data processing time window is larger than the ultra-high data processing time window; the correspondence between the data processing time windows and the processing capacity values ​​of individual windows includes: a standard data processing time window, and a processing capacity value corresponding to a single standard data processing time window; an ultra-high data processing time window, and a processing capacity value corresponding to a single ultra-high data processing time window; a peak data processing time window, and a processing capacity value corresponding to a single peak data processing time window;

[0033] The new window determination subunit is used to determine, when determining a new data processing time window, if the ratio result is greater than the preset first ratio threshold and less than the preset second ratio threshold, determine that the new data processing time window is an ultra-high data processing time window; if the ratio result is greater than or equal to the preset second ratio threshold, determine that the new data processing time window is a peak data processing time window.

[0034] The present application also provides a computer-readable storage medium, which stores instructions, characterized in that when the instructions are executed by a processor, the processor executes the steps in the method of automatically adjusting the data processing time window as described above.

[0035] The present application also provides a server, including an input interface and an output interface. The server also includes the computer-readable storage medium as described above, and a processor capable of executing instructions in the computer-readable storage medium.

[0036] It can be seen from the above technical solutions that the present application applies the technical solutions disclosed in the present application, which can dynamically adjust the data processing time window according to the data volume. It can not only cope with the processing of ordinary data, but also take corresponding strategies in time when peak data arrives to ensure smooth data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the first embodiment of the method of the present application.

[0038] Figure 2 It is a flow chart of the second embodiment of the method of the present application.

[0039] Figure 3 It is a flow chart of the first embodiment of the device of the present application.

[0040] Figure 4 It is a flow chart of the second embodiment of the device of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0042] The present application can automatically adjust the data processing capacity according to the amount of data to cope with possible data peaks. The time when the data peak occurs is unpredictable, so dynamic processing capabilities are required. In an embodiment of the present application, this dynamic processing capability is reflected in the automatic adjustment of the data time window. The so-called data time window refers to the time period in which the processor processes the data, and the data is processed in batches according to this time period. If the amount of data is greater than the amount that the data time window can accommodate, it may cause memory overflow, and if the amount of data is much smaller than the amount that the set data time window can accommodate, it will cause waste.

[0043] In practical applications, Spark Streaming is a distributed big data real-time computing framework provided by Spark, which can provide dynamic, high-throughput, and fault-tolerant streaming data processing. For such streaming data, the first embodiment of the present application method provides a method for automatically adjusting the data processing time window. Practical application Figure 1 is a flow chart of this method. Figure 1 As shown, the method includes:

[0044] Step S01: Obtain the amount of data to be processed.

[0045] In actual applications, Spark Streaming can obtain data from a variety of data queues, such as Kafka, Flume, Kinesis, Twitter, and TCP sockets, so it can also obtain the amount of data to be processed from these queues.

[0046] Step S02: judging whether the current data processing time window needs to be adjusted according to the amount of data to be processed and the standard data processing time window, wherein the standard data processing time window is a data processing time window set in advance for reference.

[0047] In data processing, since the change in data volume cannot be reliably predicted, in order to fully utilize its processing performance and not waste resources, a standard data processing time window can be set in advance. The processor processes the data in batches according to this standard data processing time window. The change in data volume can also refer to this standard data processing time window. For example, if the data volume can be completed within the standard data processing time window, it means that there is no backlog of data. If the data volume cannot be completed within the standard data processing time window, it means that there is a backlog of data.

[0048] Step S03: If adjustment is required, a new data processing time window is determined according to a pre-set data processing time window record table, and the new data processing time window is used as the current data processing time window. The data processing time window record table includes the correspondence between the data processing time window and the single window processing capacity value.

[0049] In actual applications, different data processing time windows correspond to different processing capabilities. In other words, the larger the data processing time window, the more data can be processed and the stronger its capability. In this step, the data processing time window and the corresponding processing capability value are recorded in advance. When adjustment is needed, the record table can be searched to adjust the appropriate data processing time window.

[0050] In order to better illustrate the embodiment scheme of the present application, the following is a detailed description using method embodiment 2. In method embodiment 2, a data processing time window record table is first set for subsequent use. The data processing time window record table can be set according to experience, or it can be determined through testing using experimental data. If the present method embodiment is to be determined through experiments, different data processing time windows can be set in advance. For example, it can be divided into three categories: standard data processing time window, ultra-high data processing time window, and peak data processing time window, and then different processing capacity values ​​are matched with them, as shown in Table 1:

[0051]

[0052] Table 1

[0053] That is to say, when the amount of data to be processed is known, the required capability can be determined according to the data processing time window record table shown in Table 1, thereby determining the corresponding time window. Among them, the standard data processing time window can be used as the system default time window. According to experience, it can be set to 10 seconds, which can usually process 100,000 data records, of which 1 data record is 1K bytes; the ultra-high data processing time window is the time window in which the system needs to process a large backlog but has not yet reached the peak stage. According to experience, it can be set to 20 seconds, 30 seconds or 60 seconds and other different times, which can process 200,000, 300,000 or 600,000 different data records, of which 1 data record is 1K bytes; the peak data processing time window is the time window in which the system needs to process the top peak data. According to experience, it can be set to 90 seconds, which can process 900,000 data records, of which 1 data record is 1K bytes. In actual applications, these data processing time windows can be further refined and divided into more levels. In this way, the data processing time window record table can be set to the form shown in Table 2:

[0054]

[0055] Table 2

[0056] Among them, the 10-second data processing time window can be used as the standard data processing time window, the 90-second data processing time window can be used as the peak data processing time window, and the others can be used as the exceeding data processing time windows of each level. The number of records and data size can also be used to reflect the capacity value corresponding to a single time window, and the corresponding processing time can be recorded accordingly.

[0057] In the experiment, the data in Kafka can be used as the data source that the processor needs to process, and the data is written to Kafka at a rate of 10,000 data records per second, and the average size of a data record is calculated to be 1K bytes. After writing the data, the processor can perform some operations to process the data in Kafka, such as performing a map operation, to test its processing capacity and processing time. In actual applications, the processor can also perform other types of operations, such as reduce or join operations.

[0058] Regardless of which detection method is performed in advance or based on experience, a data processing time window record table can be set, and the table at least includes the corresponding relationship between the data processing time window and the single window processing capacity value.

[0059] Figure 2 : is a flow chart of the method for automatically adjusting the data processing time window in the second embodiment. Figure 2 As shown, the method includes:

[0060] Step L201: Get the current position of the data in the data queue.

[0061] Step L202: Get the last position of the data in the recorded data queue.

[0062] Step L203: Determine the amount of data to be processed based on the current position and the previous position.

[0063] Here, steps L201 to L203 can obtain the amount of data to be processed, that is, the specific implementation of step S01. Still taking the above-mentioned Kafka data as an example, in actual applications, the data to be transmitted to the processor for processing is stored in a queue in advance. After each batch of data is transmitted from Kafka to the processor, it will be marked with a site value, which can represent the starting position of the data in Kafka, recorded as offset1. Similarly, if new data is input into Kafka, it will also be marked with a site value, which can represent the end position of the data in Kafka, recorded as offset2. In this way, the size of the data in Kafka can be calculated according to the positions of offset1 and offset2, that is, the amount of data to be processed by the processor. Among them, offset2 is the current position of the data described in step L201, and offset1 is the last position of the data described in step L202. In actual applications, the amount of data to be processed can be recorded in the configuration file, and the current time can be recorded. For example, the record: time point: 2018 / 5 / 6 11:07:00, the amount of data is 200,000.

[0064] Step L204: Based on the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, determine whether there is a backlog in the amount of data to be processed. If there is no backlog, execute step L205; otherwise, execute step L206.

[0065] In actual applications, if the amount of data to be processed is greater than the data processing capacity value corresponding to the standard data processing time window, it means that there is a backlog of data, otherwise there is no backlog. For example, if the current amount of data to be processed is 200,000 data records, and the standard data processing time window is 10 seconds, the corresponding capacity is 100,000 records, which means that there is a data backlog.

[0066] Step L205: Use the standard data processing time window as the new data processing time window and exit this process.

[0067] In actual applications, if there is no data backlog, no matter what type of data processing time window it was before, it can be restored to the standard data processing time window, and the processor no longer runs at high performance to save resources.

[0068] Step L206: Calculate the ratio of the amount of data to be processed to the data processing capacity value corresponding to the standard data processing time window.

[0069] Step L207: Determine whether the current data processing time window needs to be adjusted based on the ratio result. If adjustment is required, execute step L208; otherwise, determine that adjustment is not required and exit this process.

[0070] If there is a data backlog, steps L206 and L207 of this embodiment are a method for determining whether the current data processing time window needs to be adjusted. Although this embodiment uses the set standard data processing time window as a reference standard for whether there is a backlog, in order not to switch the data processing time window frequently, the amount of data to be processed is not directly compared with the current data processing time window, but the ratio of the amount of data to be processed to the data processing capacity value corresponding to the standard data processing time window is used as the standard for whether to adjust.

[0071] Step L208: Determine a new data processing time window according to the pre-set data processing time window record table, and use the new data processing time window as the current data processing time window. Specifically:

[0072] 1) If the ratio result is greater than the preset first ratio threshold and less than the preset second ratio threshold, the new data processing time window is determined to be an ultra-high data processing time window.

[0073] 2) If the ratio result is greater than or equal to the preset second ratio threshold, the new data processing time window is determined to be the peak data processing time window.

[0074] In practical applications, a first ratio threshold and a second ratio threshold can be set as conditions for adjusting to an ultra-high data processing time window or a peak data processing time window. When the ratio meets the first ratio threshold, it is set to an ultra-high data processing time window. Of course, if the ultra-high data processing time window has multiple levels, the accurate level can be further found according to the amount of data. When the ratio meets the second ratio threshold, it is set to a peak data processing time window.

[0075] Take Table 2 as an example, and assume that the first ratio threshold is N1=3, and the second ratio threshold is N2=10. If the current amount of data to be processed is 200,000 data records, and the standard data processing time window is 10 seconds, step L204 is used to determine that there is currently a data backlog. However, since the ratio does not exceed the first ratio threshold N1, no adjustment is currently required, and data can still be processed according to the standard data processing time window of 10 seconds.

[0076] If the current amount of data to be processed is 600,000 data records, and the standard data processing time window is 10 seconds, step L204 is used to determine that there is currently a data backlog. And because the ratio has exceeded the first ratio threshold N1 and has not exceeded the second ratio threshold N2, it is determined that the current data processing time window needs to be adjusted to an ultra-high data processing time window. According to the corresponding relationship in Table 2, 60 seconds can be selected as the new data processing time window.

[0077] If the amount of data that needs to be processed is very large, reaching 1.2 million data records, and the standard data processing time window is 10 seconds, step L204 is used to determine that there is a data backlog. Moreover, the ratio has exceeded the second ratio threshold N2, and it is determined that the current data processing time window needs to be adjusted to the peak data processing time window. According to the corresponding relationship in Table 2, 90 seconds can be selected as the new data processing time window.

[0078] In actual applications, the processing performance of the processor is limited. If the data processing time window is continuously increased according to the data volume, it will not only fail to speed up the data processing, but will cause memory overflow and reduce the processing speed because the data volume is much larger than its processing performance. Here, if the peak data processing time window is assumed to be 90 seconds, even if the amount of data to be processed is greater than 900,000 data records, the data processing time window will not be further expanded to ensure that memory overflow does not occur.

[0079] By applying the solution of this embodiment, the amount of data to be processed is obtained in advance, and a data processing time window record table is set, and the data processing time window is dynamically adjusted according to the amount of data. Therefore, when the data is seriously backlogged, or when peak data is generated, the data processing time window can be automatically adjusted to the peak data processing time window to cope with the big data situation. When there is no backlog of data, it can be automatically restored to the standard data processing time window to save resources.

[0080] With respect to the above-mentioned method embodiments, the present application also provides a device for automatically adjusting a data processing time window. Figure 3 is a structural diagram of the first embodiment of the device of the present application, such as Figure 3 As shown, the device includes: a data volume acquisition unit 301, a determination unit 302, and a window adjustment unit 303. Among them,

[0081] The data volume acquisition unit 301 is used to acquire the data volume to be processed. The determination unit 302 determines whether the current data processing time window needs to be adjusted according to the data volume to be processed and the standard data processing time window, wherein the standard data processing time window is a data processing time window set in advance for reference. The window adjustment unit 303 determines a new data processing time window according to a data processing time window record table set in advance, and uses the new data processing time window as the current data processing time window, wherein the data processing time window record table includes the corresponding relationship between the data processing time window and the single window processing capacity value.

[0082] That is, the data volume acquisition unit 301 acquires the data volume to be processed from the data queue, and the determination unit 302 determines whether the current data processing time window needs to be adjusted. If adjustment is required, the window adjustment unit 303 is triggered. The window adjustment unit 303 determines a new data processing time window according to the pre-set data processing time window record table.

[0083] Figure 4 This is a structural diagram of the second embodiment of the device of the present application. Figure 4 As shown, the device includes a data volume acquisition unit 301, a determination unit 302, and a window adjustment unit 303. The determination unit 302 includes:

[0084] The first determining subunit 3021 is used to determine whether there is a backlog in the amount of data to be processed based on the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window. If there is no backlog, the standard data processing time window is used as the new data processing time window; otherwise, the second determining subunit 3022 is triggered.

[0085] The second determination subunit 3022 is used to calculate the ratio of the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, and determine whether the current data processing time window needs to be adjusted based on the ratio result, and trigger the window adjustment unit 303.

[0086] The window adjustment unit 303 includes:

[0087] The storage subunit 3031 is used to save a data processing time window record table; the data processing time window record table includes a standard data processing time window, an ultra-high data processing time window and a peak data processing time window, the ultra-high data processing time window is larger than the standard data processing time window, and the peak data processing time window is larger than the ultra-high data processing time window; the correspondence between the data processing time windows and the processing capacity values ​​of a single window includes: a standard data processing time window, and a processing capacity value corresponding to a single standard data processing time window; an ultra-high data processing time window, and a processing capacity value corresponding to a single ultra-high data processing time window; a peak data processing time window, and a processing capacity value corresponding to a single peak data processing time window.

[0088] The new window determination subunit 3032 is used to, when determining a new data processing time window, determine that the new data processing time window is an ultra-high data processing time window if the ratio result is greater than the preset first ratio threshold and less than the preset second ratio threshold; if the ratio result is greater than or equal to the preset second ratio threshold, determine that the new data processing time window is a peak data processing time window.

[0089] The data processing time window record table stored in the storage subunit 3031 may be as shown in Table 1 or Table 2. That is, the data volume acquisition unit 301 acquires the amount of data to be processed from the data queue, and the first determination subunit 3021 determines whether there is a backlog in the amount of data to be processed according to the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window. If there is no backlog, the standard data processing time window is used as a new data processing time window; otherwise, the second determination subunit 3022 is triggered. The second determination subunit 3022 calculates the ratio of the amount of data to be processed to the data processing capacity value corresponding to the standard data processing time window, and determines whether the current data processing time window needs to be adjusted according to the ratio result, and triggers the window adjustment unit 303. The new window determination subunit 3032 in the window adjustment unit 303 queries the data processing time window record table saved by the storage subunit 3031. If the ratio result is greater than the preset first ratio threshold and less than the preset second ratio threshold, the new data processing time window is determined to be an ultra-high data processing time window; if the ratio result is greater than or equal to the preset second ratio threshold, the new data processing time window is determined to be a peak data processing time window.

[0090] The present application also provides a computer-readable storage medium, which may be a storage medium such as a ROM, RAM, EPROM, SIM card, or optical disk, for storing instructions. When the instructions are executed by a processor, the processor executes the steps of the method for automatically adjusting the data processing time window in the above embodiments.

[0091] The present application also provides a server, including an input interface and an output interface, and also includes the above-mentioned computer-readable storage medium, and a processor capable of executing instructions in the computer-readable storage medium.

[0092] By applying the solution of this embodiment, the amount of data to be processed is obtained in advance, and a data processing time window record table is set, and the data processing time window is dynamically adjusted according to the amount of data. Therefore, when the data is seriously backlogged, or when peak data is generated, the data processing time window can be automatically adjusted to the peak data processing time window to cope with the big data situation. When there is no backlog of data, it can be automatically restored to the standard data processing time window to save resources.

[0093] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for automatically adjusting a data processing time window, characterized in that: The method includes: Get the amount of data to be processed; According to the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, determine whether there is a backlog in the amount of data to be processed; if there is no backlog, use the standard data processing time window as a new data processing time window; otherwise, calculate the ratio of the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, and determine whether the current data processing time window needs to be adjusted according to the ratio result; the standard data processing time window is a data processing time window set in advance for reference; If adjustment is required, a new data processing time window is determined according to a pre-set data processing time window record table, and the new data processing time window is used as the current data processing time window, wherein the data processing time window record table includes a corresponding relationship between data processing time windows and single window processing capacity values; The data processing time window record table includes an ultra-high data processing time window, and determining a new data processing time window according to a pre-set data processing time window record table includes: if the ratio result is greater than a preset first ratio threshold and less than a preset second ratio threshold, then determining that the new data processing time window is an ultra-high data processing time window.

2. The method according to claim 1, characterized in that: The step of obtaining the amount of data to be processed comprises: Get the current position of the data in the data queue; Get the last position of the data in the recorded data queue; The amount of data to be processed is determined according to the current position and the previous position.

3. The method according to claim 1, characterized in that The data processing time window record table also includes a standard data processing time window and a peak data processing time window, the super-high data processing time window is larger than the standard data processing time window, and the peak data processing time window is larger than the super-high data processing time window; The corresponding relationship between the data processing time window and the single window processing capacity value includes: Standard data processing time window, and the processing capacity value corresponding to a single standard data processing time window; Ultra-high data processing time window, and the processing capacity value corresponding to a single ultra-high data processing time window; The peak data processing time window, and the processing capacity value corresponding to a single peak data processing time window.

4. The method according to claim 3, characterized in that The step of determining a new data processing time window according to a pre-set data processing time window record table comprises: If the ratio result is greater than or equal to the preset second ratio threshold, the new data processing time window is determined to be the peak data processing time window.

5. A device for automatically adjusting a data processing time window, characterized in that: The device includes: A data volume acquisition unit, used to acquire the volume of data to be processed; a determination unit, judging whether there is a backlog in the amount of data to be processed according to the amount of data to be processed and the data processing capacity value corresponding to the standard data processing time window, and if there is no backlog, taking the standard data processing time window as a new data processing time window; otherwise, calculating the ratio of the amount of data to be processed to the data processing capacity value corresponding to the standard data processing time window, and determining whether it is necessary to adjust the current data processing time window according to the ratio result, wherein the standard data processing time window is a data processing time window set in advance for reference; A window adjustment unit determines a new data processing time window according to a pre-set data processing time window record table, and uses the new data processing time window as the current data processing time window, wherein the data processing time window record table includes a correspondence between data processing time windows and single window processing capacity values; the data processing time window record table includes an ultra-high data processing time window, and the new data processing time window is determined according to the pre-set data processing time window record table, including: if the ratio result is greater than a preset first ratio threshold and less than a preset second ratio threshold, then the new data processing time window is determined to be an ultra-high data processing time window.

6. The device according to claim 5, characterized in that The window adjustment unit comprises: A storage subunit, used for saving a data processing time window record table; the data processing time window record table includes a standard data processing time window and a peak data processing time window, the ultra-high data processing time window is larger than the standard data processing time window, and the peak data processing time window is larger than the ultra-high data processing time window; the correspondence between the data processing time windows and the processing capacity values ​​of the individual windows includes: a standard data processing time window, and a processing capacity value corresponding to a single standard data processing time window; an ultra-high data processing time window, and a processing capacity value corresponding to a single ultra-high data processing time window; a peak data processing time window, and a processing capacity value corresponding to a single peak data processing time window; The new window determination subunit is used to determine, when determining a new data processing time window, that the new data processing time window is a peak data processing time window if the ratio result is greater than or equal to the preset second ratio threshold.

7. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by a processor, the processor is caused to perform the steps in the method for automatically adjusting a data processing time window according to any one of claims 1 to 4.

8. A server, comprising an input interface and an output interface, characterized in that: The server also includes the computer-readable storage medium of claim 7, and a processor capable of executing instructions in the computer-readable storage medium.

Images