Data off-peak uploading method and device and medium
By pulling the data upload amount of the device in the message queue and calculating the upload time offset, and adjusting the data upload time, the problem of data upload uniformity and efficiency in the case of random or discontinuous device numbers is solved, and the balanced distribution of data between devices and efficient utilization of resources is achieved.
Patent Information
- Application Number
- CN202311380702.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-23
- Publication Date
- 2025-04-25
AI Technical Summary
The prior art is difficult to ensure uniformity and efficiency of data upload when the device numbers are random or discontinuous, especially in scenarios where users require data to be reported at designated time points.
By pulling data based on preset interval time in the message queue, determining the amount of data uploaded by the device, and calculating its upload time offset according to the working time rules of the target device, and adjusting the data upload time to achieve data staggered upload.
It realizes the balanced distribution of data between devices, reduces peak and trough fluctuations, reduces data latency, and efficiently utilizes server and network resources.
Smart Images

Figure CN119892735A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a method, device and medium for staggered data uploading. Background Art
[0002] In recent years, the Internet of Things (IoT) has been widely used in the fields of industry, agriculture, environment, transportation, etc., effectively promoting the intelligent development of these fields and improving the efficiency and benefits of the industry. It integrates technologies such as equipment, sensors, networks and cloud computing to realize the interconnection of all things, bringing great changes to our work and life.
[0003] However, as the number of devices and sensors increases and their application scope expands, how to upload, process and analyze the massive data information generated in a timely and efficient manner has also become an important concern for practitioners in this field. For example, in the daily operation and inspection of power transmission lines, online monitoring has gradually replaced the traditional manual inspection method, in which monitoring equipment is responsible for the images and videos of the line channels, and various sensors are responsible for the collection and uploading of temperature, micro-meteorological, ice coverage and other data. This faces the challenges of wide equipment coverage, large number, and high data generation rate. How to balance the upload pressure, reduce peak congestion, and make full use of network and server resources have become the main technical concerns to ensure the timeliness and efficiency of data.
[0004] "CN110719230B A method, device and computer-readable storage medium for adjusting off-peak intervals" proposes a method for adjusting off-peak intervals applied to terminal devices: determine the reporting period of the service, receive the service processing parameters sent by the server, and determine the time window offset for reporting the service according to the service reporting period and the service processing parameters, thereby dynamically adjusting the off-peak interval of the terminal from the perspective of collaboration between the terminal device and the server to improve the reporting efficiency. However, this solution relies on the continuity of the terminal numbering and is not suitable for situations where the device numbering is random or discontinuous. In this case, the uniformity of the offset cannot be guaranteed. In addition, this solution is also not suitable for scenarios where users require data to be reported at specified time points. In this scenario, the solution only evenly disperses the devices whose reporting data time points can be adjusted, and cannot achieve overall uniform distribution.
[0005] "CN115834619A A method and system for staggered uploading of IoT data" proposes a method for staggered uploading of IoT data: according to the set of all terminal devices, the set of terminal devices that need to adjust the working time period, and the adjusted working time period, the ideal number of devices and the distribution statistics corresponding to the devices that do not need to be adjusted are obtained, and then the adjustment is iterated and cyclically adjusted until the start working time of all terminal devices that need to adjust the working time period is adjusted, so as to avoid peak congestion caused by uploading device data to the server. However, this scheme needs to iterate and cycle continuously to obtain the minimum and smaller values of each round, and calculate and adjust accordingly. This multi-round operation increases the computing burden and reduces the adjustment efficiency; and the scheme is based only on the number of devices, and does not consider the factor of the actual amount of data uploaded by the device. For example, one device may have 5 channels to upload 8 data, while another device has only 1 channel to upload 1 data. Although the number of devices is the same, the amount of data is different. Summary of the invention
[0006] In order to solve the above problems, the present application proposes a method, a device and a medium, wherein the method includes:
[0007] Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; determine the target device whose data upload time needs to be adjusted, and the working time rule corresponding to the target device to be adjusted, and determine the upload time offset of the target device; based on the upload time offset, adjust the data upload time of the target device.
[0008] In one example, determining the amount of data uploaded by the device within the previous interval time period specifically includes: aggregating the data upload time to obtain a time point set and a first data volume result set; querying the cache through the time point set to obtain a second data volume result set; merging the first data volume result set and the second data volume result set to update the second data volume result set; storing the updated second data volume result set in the cache, and refreshing the update time corresponding to each time point in the second data volume result set.
[0009] In one example, the merging of the first data volume result set and the second data volume result set to update the second data volume result set specifically includes: selecting any time node in the first data volume result set, and traversing the any time node in the second data volume result set; if the any time node exists in the second data volume result set, obtaining the first data volume result and the second data volume result corresponding to the any time node in the first data volume result set and the second data volume result set respectively; adding the first data volume result and the second data volume result to replace the second data volume result; if the any time node does not exist in the second data volume result set, filling the any time node and the corresponding first data volume result into the second data volume result set.
[0010] In one example, determining the upload time offset of the target device specifically includes: determining the target device and a third data volume result set of the amount of data that the target device should upload, and the amount of data that should be uploaded corresponding to the target device; based on the working time rule, obtaining a set of working time points of the target device, and obtaining a first historical working time point and a second historical working time point that are closest to the current adjustment time in the set of working time points; based on the first historical working time point and the second historical working time point, determining a target historical data set and a target historical working time point; based on the target historical data set and the target historical working time point, determining the upload time offset.
[0011] In one example, determining a target historical data set and a target historical working time point based on the first historical working time point and the second historical working time point specifically includes: determining a historical data set corresponding to the first historical working interval by querying a cache; determining whether the historical data set is available based on an update time of each time point in the first historical working interval; if not available, selecting historical working time points forward in the working time point set until the historical data set is available; using the historical data set as a target historical data set, and determining a target historical working time point corresponding to the target historical data set.
[0012] In one example, determining the upload time offset based on the target historical data set and the target historical working time points specifically includes: determining the target upload average of the upload data volume per minute corresponding to all devices based on the upload data volume corresponding to each time point in the target historical data set, and the upload data volume and working interval corresponding to the target device; obtaining a difference data set of time points and upload differences in the target historical data set that are lower than the target upload average; determining the total difference of the difference data set based on the difference data set; determining the upload time offset based on the difference data set and the total difference.
[0013] In one example, determining the upload time offset based on the difference data set and the total difference specifically includes: determining the offset and ideal data volume corresponding to each time point in the difference data set of the target device based on the difference data set, the total difference, the target historical working time point, and the amount of data to be uploaded; determining the offset data set of the target device based on the offset and the ideal data volume; determining the upload time offset result set corresponding to the target device based on the offset data set and the third data volume result set.
[0014] In one example, adjusting the data upload time of the target device based on the upload time offset specifically includes: calculating the actual working rule result set of the target device based on the upload time offset result set and the working time rules to be adjusted; and sending the actual working rule result set to the target device based on a preset sending protocol to adjust the data upload time corresponding to the target device.
[0015] The present application also provides a data staggered upload device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: based on a preset interval time, pull data from a message queue, and determine the amount of data uploaded by the device in the previous interval time period; determine a target device whose data upload time needs to be adjusted, and the working time rule corresponding to the target device to be adjusted, and determine the upload time offset of the target device; based on the upload time offset, adjust the data upload time of the target device.
[0016] The present application also provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: based on a preset interval time, pull data from a message queue to determine the amount of data uploaded by the device in the previous interval time period; determine a target device for which data upload time needs to be adjusted, and a working time rule to be adjusted corresponding to the target device, and determine an upload time offset for the target device; and adjust the data upload time for the target device based on the upload time offset.
[0017] The method proposed in this application can bring the following beneficial effects: starting from the perspective of the amount of real-time data uploaded by all devices, according to the amount of data uploaded by the device and the working time rules, the upload time offset is calculated in real time. Compared with the scheme of calculating by the number of devices and multiple rounds, the offset distribution is more accurate, the amount of data uploaded at each time point is more balanced, and the calculation rounds are significantly reduced, thereby achieving the purpose of overall balanced upload, reducing peak and valley fluctuations, reducing data delays, and efficiently utilizing server and network resources. At the same time, the present invention is also applicable to scenarios such as random or discontinuous device numbers and users requiring data to be reported at specified time points. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0019] Figure 1 A flowchart of a method for staggered data uploading in an embodiment of the present application is shown;
[0020] Figure 2 This is a flow chart of real-time statistics of the uploaded data volume in an embodiment of the present application;
[0021] Figure 3 This is a schematic diagram of the overall process of calculating the upload time offset in an embodiment of the present application;
[0022] Figure 4 This is a schematic diagram of a process for calculating the upload time offset of a target device in an embodiment of the present application;
[0023] Figure 5 This is a structural diagram of a data peak-shifting upload device in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0025] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0026] Figure 1 A flowchart of a data staggered upload method provided for one or more embodiments of this specification. The method can be applied to data corresponding to different business fields, such as data in the Internet financial business field, data in the e-commerce business field, data in the instant messaging business field, data in the game business field, data in the official business field, etc. The process can be executed by a computing device in the corresponding field (such as a risk control server or smart mobile terminal corresponding to the payment business, etc.), and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.
[0027] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.
[0028] It should be noted that the server can be a single device or a system composed of multiple devices, that is, a distributed server, and this application does not make any specific limitations on this.
[0029] like Figure 1 As shown, the embodiment of the present application provides a method for staggered data uploading, including:
[0030] S101: based on a preset interval time, pull data from a message queue to determine the amount of data uploaded by the device in the previous interval time period.
[0031] The device uploads data to the server, which in this embodiment is a message queue uploaded to the server (Note: other upload or storage methods are also possible, and the relevant content is not within the scope of discussion of the present invention and will not be repeated). Pull a batch of data from the message queue at regular intervals (Note: the data pulled this time will not be repeated with the data pulled previously, this is a mechanism of the message queue, and the relevant content is not within the scope of discussion of the present invention and will not be repeated) to determine the amount of data uploaded by the device in the previous interval time period.
[0032] like Figure 2As shown in the figure, when determining the amount of data uploaded by the device, it is necessary to aggregate the data upload time to obtain the time point set T <time>And the first data volume result set M o <time,dataCount(int)> ; Wherein M0 is in the form of a key-value pair, mapping the corresponding relationship between the time point time and the data upload amount dataCount, time is a time point in the format of yyyy-MM-dd HH:mm, and dataCount is the amount of data uploaded by all devices at the corresponding time point; query the cache EachMinuteCountCache through the time point set T (Note: it can also be other methods such as a database, and the relevant content is not within the scope of discussion of the present invention and will not be repeated), and obtain the second data amount result set M1<time,dataCount(int)> ; Merge the first data volume result set M0 and the second data volume result set M1 to update the second data volume result set M1; store the updated second data volume result set M1 in the cache EachMinuteCountCache, and refresh the update time corresponding to each time point in the second data volume result set M1.
[0033] Furthermore, when merging the first data volume result set M0 and the second data volume result set M1 to update the second data volume result set M1, it is necessary to select any time node time0 in the first data volume result set M0, and search whether there is any time node time0 in the second data volume result set M1. If there is any time node time0 in the second data volume result set M1, obtain the first data volume result dataCount0 and the second data volume result dataCount1 corresponding to any time node time0 in the first data volume result set M0 and the second data volume result set M1 respectively; add the first data volume result dataCount0 and the second data volume result dataCount1 to replace the second data volume result dataCount1; if there is no time node time0 in the second data volume result set M1, fill any time node time0 and the corresponding first data volume result dataCount0 into the second data volume result set M1.
[0034] S102: Determine a target device whose data upload time needs to be adjusted, and a working time rule corresponding to the target device that needs to be adjusted, and determine an upload time offset of the target device.
[0035] Based on the amount of data uploaded by the device, combined with the device that needs to adjust the data upload time and the working time rule to be adjusted, the upload time offset of each device can be calculated.
[0036] like Figure 3 As shown, in one embodiment, when calculating the upload time offset, it is necessary to determine the target device and the third data volume result set D of the data volume that the target device should upload.<deviceId,DUC(int)> , and the corresponding amount of data to be uploaded by the target device. Where D is in the form of a key-value pair, mapping the correspondence between the device's unique number deviceId and its amount of data to be uploaded DUC. It should be noted that in this embodiment, the amount of data to be uploaded is determined based on the device channel-related information. For example, if there are a total of 5 preset positions under 2 channels of a certain device, then the amount of data to be uploaded by the device is 5 (Note: it can also be determined in other ways, and the relevant content is not within the scope of discussion of this invention and will not be repeated). Calculate the amount of data to be uploaded DUC total It can be calculated by the following formula:
[0037]
[0038] Where n is the number of devices that need to adjust the data upload time. Then, based on the working time rules, namely the working start time T1, the working end time T2 and the working interval I, the working time point set S of the target device is obtained. <time>, where time is a time point in the format of HH:mm. And get the first historical working time point time closest to the current adjustment time nowTime in the working time point set S. n And the second historical working time point time n-1 In this embodiment, if T1 = 07:00, T2 = 19:00, I = 10, then S <time>contains time points such as 11:00, 11:10, and 11:20. If nowTime = 11:12, then time n =11:10, time n-1 =11:00.
[0039] In one embodiment, when determining the target historical data set and the target historical working time point, it is necessary to query the first historical working interval [time n-1 , time n ) historical data set M n-1 <time,dataCount(int)> , and based on the update time of each time point in the first historical working interval, determine whether the dataset is available. If not, then take [time n-2 , time n-1 ) interval data set M n-2 And determine whether it is available... and so on, until the data set M j Available, the corresponding working time point is the target historical working time point time j , at this time, the historical data set M j As the target historical dataset.
[0040] In one embodiment, after determining the target historical data set and the target historical working time point, it is also necessary to j The amount of uploaded data dataCount corresponding to each time point time, and the amount of data that should be uploaded DUC corresponding to the target device total And the working interval I, the target upload average DC of the data upload volume per minute corresponding to all devices is determined by the following formula: avg :
[0041]
[0042] Then obtain the target historical dataset M j The dataCount is lower than the target upload average DC avg The time point time and the difference dataset V of the uploaded difference<time,DV(double)> , where V is a key-value pair, mapping the corresponding relationship between the time point time and its difference DV, DV = DC avg -dataCount, and sort them from large to small according to the difference value. Then, based on the difference data set V, the total difference value DV of the difference data set is determined by the following formula total ,Right now:
[0043]
[0044] Where n is dataCount lower than DCav g The number of time points time.
[0045] In one embodiment, after obtaining the difference data set and the total difference, it is also necessary to calculate the difference data set V and the total difference DV based on the difference data set V and the total difference DV. total 、Target historical working time point time j , Amount of data to be uploaded DUC total , determine the offset offset (int) and the ideal data volume idealCount (int) corresponding to each time point of the target device in the difference data set; based on the offset and the ideal data volume, determine the offset data set O of the target device<offset(int),idealCount(int)> ; Wherein O is in the form of a key-value pair, mapping the corresponding relationship between the offset offset (in minutes) corresponding to the time point time and the ideal data volume idealCount. And based on the offset data set and the third data volume result set, determine the upload time offset result set corresponding to the target device. It should be noted that the offset data set O is also an ordered data set, and its order is consistent with the difference data set V.
[0046] The offset can be calculated using the following formula:
[0047] offset=time-time j
[0048] The ideal data volume can be calculated using the following formula:
[0049]
[0050] like Figure 4 As shown, the offset data set O of the previous step and the third data volume result set D of the device and the data volume to be uploaded are iterated to calculate the device upload time offset result set N<deviceId,offset(int)> , where N is a key-value pair, mapping the correspondence between the device unique number deviceId and its upload time offset offset. Specifically: first take the i-th offset offset in the offset data set O i And its corresponding ideal data volume idealCount i , where the initial value of i is 1; then take the jth device deviceId in the third data volume result set D j The amount of data to be uploaded DUC j , where the initial value of j is 1; if idealCount i >0, then the device deviceId j The upload time offset is set to offset i ,Will <deviceId j , offset i >Put it into result set N, and idealCount i -=DUC j 、DUC total -=DUC j 、j+=1;if idealCount i ≤0, then i+=1; if DUC total > 0, then continue to cycle through steps I and II until DUC total =0, calculation and setting are completed, and the loop is exited.
[0051] S103: Adjusting the data upload time of the target device based on the upload time offset.
[0052] After obtaining the upload time offset, the data upload time of the target device can be adjusted according to the upload time offset.
[0053] In one embodiment, when adjusting the data upload time of the target device by the upload time offset, the actual working rule result set of the target device can be calculated based on the upload time offset result set and the working time rule to be adjusted; based on the preset sending protocol, the actual working rule result set is sent to the target device to adjust the data upload time corresponding to the target device. Specifically, according to the device upload time offset result set N obtained in the previous step, and the working time rule to be adjusted, that is, the working start time T1, the working end time T2 and the working interval I, the actual working rule result set R of the device is calculated.<deviceId,Rule> , where R is a key-value pair, mapping the correspondence between the device unique number deviceId and its actual working rule Rule. Rule includes the actual start time T 1a , Actual end time of work T 2a The calculation formula for the working interval I, the actual work start time and the actual work end time is:
[0054] T 1a =T1+offset
[0055] T 2a =T2+offset
[0056] After obtaining the actual working rule result set R of the device, the actual working rules are sent to the device in combination with the sending protocol (Note: different protocols have different sending methods, and the relevant content is not within the scope of discussion of this invention and will not be repeated), thereby completing the dispersion of the device data upload time.
[0057] like Figure 5 As shown, the embodiment of the present application also provides a data peak-shifting uploading device, including:
[0058] at least one processor; and,
[0059] a memory communicatively connected to the at least one processor; wherein,
[0060] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:
[0061] Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; determine the target device whose data upload time needs to be adjusted, and the working time rule corresponding to the target device to be adjusted, and determine the upload time offset of the target device; based on the upload time offset, adjust the data upload time of the target device.
[0062] The embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:
[0063] Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; determine the target device whose data upload time needs to be adjusted, and the working time rule corresponding to the target device to be adjusted, and determine the upload time offset of the target device; based on the upload time offset, adjust the data upload time of the target device.
[0064] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0065] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0066] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0067] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0070] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0071] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0072] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0073] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.
[0074] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.< / time> < / time> < / time>
Claims
1. A data peak-shifting uploading method, characterized in that: include: Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; Determine a target device whose data upload time needs to be adjusted, and a working time rule corresponding to the target device that needs to be adjusted, and determine an upload time offset of the target device; Based on the upload time offset, the data upload time of the target device is adjusted.
2. The method according to claim 1, characterized in that: Determining the amount of data uploaded by the device in the previous interval time period specifically includes: By aggregating the data upload time, a time point set and a first data volume result set are obtained; Query the cache through the time point set to obtain a second data volume result set; Merging the first data volume result set and the second data volume result set to update the second data volume result set; The updated second data volume result set is stored in the cache, and the update time corresponding to each time point in the second data volume result set is refreshed.
3. The method according to claim 2, characterized in that The merging the first data volume result set and the second data volume result set to update the second data volume result set specifically includes: Select any time node in the first data volume result set, and traverse the any time node in the second data volume result set; If any time node exists in the second data volume result set, obtaining the first data volume result and the second data volume result corresponding to the any time node in the first data volume result set and the second data volume result set respectively; Adding the first data volume result and the second data volume result, and replacing the second data volume result; If any of the time nodes does not exist in the second data volume result set, then any of the time nodes and the corresponding first data volume result are added to the second data volume result set.
4. The method according to claim 1, characterized in that The determining the upload time offset of the target device specifically includes: Determine the target device and a third data volume result set of the amount of data that the target device should upload, and the amount of data that should be uploaded corresponding to the target device; Based on the working time rule, obtain a set of working time points of the target device, and obtain a first historical working time point and a second historical working time point closest to the current adjustment time in the set of working time points; Determine a target historical data set and a target historical working time point based on the first historical working time point and the second historical working time point; The upload time offset is determined based on the target historical data set and the target historical working time point.
5. The method according to claim 4, characterized in that The determining a target historical data set and a target historical working time point based on the first historical working time point and the second historical working time point specifically includes: Determine the historical data set corresponding to the first historical working interval by querying the cache; Determining whether the historical data set is available based on the update time of each time point within the first historical working interval; If it is not available, selecting a historical working time point in the working time point set forward until the historical data set is available; The historical data set is used as a target historical data set, and a target historical working time point corresponding to the target historical data set is determined.
6. The method according to claim 5, characterized in that The determining the upload time offset based on the target historical data set and the target historical working time point specifically includes: Based on the upload data volume corresponding to each time point in the target historical data set, the upload data volume corresponding to the target device and the working interval, determine the target upload average value of the upload data volume per minute corresponding to all devices; Obtain a difference data set of time points and upload differences that are lower than the target upload average value in the target historical data set; Based on the difference data set, determining a total difference value of the difference data set; The upload time offset is determined based on the difference data set and the total difference.
7. The method according to claim 6, characterized in that The determining the upload time offset based on the difference data set and the total difference specifically includes: Based on the difference data set, the total difference, the target historical working time point, and the amount of data to be uploaded, determining the offset and the ideal data amount corresponding to each time point in the difference data set of the target device; Determining an offset data set of the target device based on the offset and the ideal data amount; Based on the offset data set and the third data volume result set, an upload time offset result set corresponding to the target device is determined.
8. The method according to claim 7, characterized in that The adjusting the data upload time of the target device based on the upload time offset specifically includes: Based on the upload time offset result set and the working time rule to be adjusted, calculate and obtain the actual working rule result set of the target device; Based on a preset delivery protocol, the actual work rule result set is delivered to the target device to adjust the data upload time corresponding to the target device.
9. A data peak-shifting upload device, characterized in that: include: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute: Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; Determine a target device whose data upload time needs to be adjusted, and a working time rule corresponding to the target device that needs to be adjusted, and determine an upload time offset of the target device; Based on the upload time offset, the data upload time of the target device is adjusted.
10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Based on the preset interval time, pull data from the message queue to determine the amount of data uploaded by the device in the previous interval time period; Determine a target device whose data upload time needs to be adjusted, and a working time rule corresponding to the target device that needs to be adjusted, and determine an upload time offset of the target device; Based on the upload time offset, the data upload time of the target device is adjusted.
Citation Information
Patent Citations
A method, apparatus and computer-readable storage medium for adjusting peak shift intervals
CN110719230B
Internet of things data off-peak uploading method and system
CN115834619A