Data collection method, system and computer readable storage medium

Abstract

The present invention relates to the field of communication technology, and discloses a data acquisition method, system, and computer-readable storage medium. The data acquisition method comprises: for each acquisition device in the acquisition task within the current acquisition period, determining the priority of each data item of each acquisition device according to the historical acquisition success rate of each data item; then, determining the highest priority acquisition device according to the average historical acquisition success rate of each acquisition device; finally, collecting the highest priority data item for the highest priority acquisition device. Thus, the present invention arranges priorities according to the historical acquisition success rate of different data items of the same device, and then arranges priorities according to the average historical acquisition success rate of different devices, and preferentially collects the highest priority data items of the highest priority device, thereby greatly improving the acquisition efficiency when facing a large number of devices or high-frequency acquisition demand scenarios.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a data acquisition method, system and computer-readable storage medium. Background Art

[0002] Currently, the primary communication method between data collection terminals and equipment in low-voltage substations is carrier communication. Applications within the terminals communicate with local communication modules to implement data collection. Typically, a marketing or distribution master station issues data files and collection tasks to the terminal, which then calculates a specific collection plan based on the files and tasks. Most terminal standards categorize services into different applications based on their functionality.

[0003] Typically, the collection equipment in a distribution substation or low-voltage substation is fixed for a certain period of time. When calculating a specific collection plan, collection business applications sort the data items collected by the collection equipment in order of task numbers based on the execution frequency and task runtime of the collection tasks.

[0004] Some data collection devices in the substation area may experience data collection failures due to differences in communication quality, software versions, and other factors. This phenomenon becomes more pronounced when there are a large number of data collection devices in the substation area or when high-frequency data collection is required. When data collection applications encounter an unresponsive data collection device during concurrent reading, a maximum timeout period must be maintained. This extended waiting time reduces data collection efficiency, resulting in the inability to complete data collection tasks within a single period. The aforementioned data collection methods cannot distinguish the success rates of different data collection devices and do not provide automated means to screen out devices with low success rates. Summary of the Invention

[0005] The purpose of the present invention is to provide a data collection method, system and computer-readable storage medium, which prioritize different data items according to the historical collection success rate of the same device, and then prioritize them according to the average historical collection success rate of different devices, and give priority to collecting the highest priority data items of the highest priority device, thereby greatly improving the collection efficiency when facing a large number of devices or high-frequency collection demand scenarios.

[0006] In order to achieve the above-mentioned purpose, the first aspect of the present invention provides a data collection method, which includes: for each of the multiple collection devices in the collection task within the current collection period, prioritizing each data item according to the historical collection success rate of each data item of each collection device; determining the highest priority collection device according to the average historical collection success rate of each collection device; and collecting the highest priority data items for the highest priority collection device.

[0007] Preferably, the data collection method further includes: obtaining updated data items of the highest priority collection device, wherein the updated data items do not include the highest priority data items; re-determining the highest priority collection device based on the updated average historical collection success rate of the highest priority collection device and the average historical collection success rate of other collection devices in the collection tasks within the current collection period; and collecting the highest priority data items for the re-determined highest priority collection device.

[0008] Preferably, the historical collection success rate of each data item of each collection device is determined by: according to the response time C of the jth data item of the kth collection device collected for the ith time in the previous collection period kji And the following formula is used to determine the historical acquisition success rate X of the jth data item of the kth acquisition device: kj ,

[0009]

[0010] Where b is the number of collections corresponding to the j-th data item of the k-th collection device in the previous collection period; d is the success rate calculation interval.

[0011] Preferably, the average historical acquisition success rate of each acquisition device is determined by the following method: according to the first historical acquisition success rate X of the j-th data item of the k-th device kj And the following formula is used to determine the average historical collection success rate X of the n data items of the k-th device: k ,

[0012]

[0013] Preferably, the data collection method further includes: determining the period collection success rate in each of multiple collection time periods under multiple preset maximum concurrency numbers, wherein the multiple collection time periods are multiple consecutive collection time periods before the current collection time period; determining the maximum period collection success rate when the difference between the period collection success rates in any two of the multiple collection time periods is less than or equal to a preset value; and determining the preset maximum concurrency number corresponding to the maximum period collection success rate as the target maximum concurrency number for communication between the collection terminal and the multiple collection devices.

[0014] Preferably, determining the period collection success rate in each of the multiple collection time periods under the multiple preset maximum concurrency numbers includes: determining the ratio of the number of data items successfully collected by all collection devices in each of the collection time periods under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices as the period collection success rate in each of the collection time periods under the multiple preset maximum concurrency numbers.

[0015] Preferably, the data collection method also includes: judging whether the collection task in the current collection period is the same as the collection task in the previous collection period; if the collection task in the current collection period is the same as the collection task in the previous collection period, determining the shortest completion time based on the response time of each data item of each collection device in the collection task in the previous collection period; and if the total duration of the collection task in the current collection period is greater than a preset multiple of the shortest completion time, stopping the statistical operation of the historical collection success rate.

[0016] Through the above technical solution, the present invention creatively determines the priority of each data item of each of the multiple collection devices in the collection task within the current collection period based on the historical collection success rate of each data item of each collection device; then, determines the highest priority collection device based on the average historical collection success rate of each collection device; and finally, collects the highest priority data item for the highest priority collection device. Thus, the present invention prioritizes different data items based on the historical collection success rate of the same device, then prioritizes them based on the average historical collection success rate of different devices, and prioritizes the collection of the highest priority data items of the highest priority device, thereby greatly improving collection efficiency when facing scenarios with a large number of devices or high-frequency collection requirements.

[0017] A second aspect of the present invention provides a data acquisition system, which includes: a first determination device for determining the priority of each data item of each acquisition device among multiple acquisition devices in the acquisition task within the current acquisition period according to the historical acquisition success rate of each data item of each acquisition device; a second determination device for determining the highest priority acquisition device according to the average historical acquisition success rate of each acquisition device; and an acquisition terminal for collecting the highest priority data items for the highest priority acquisition device.

[0018] Preferably, the data acquisition system further includes: a deleting device for deleting the highest priority data item of the highest priority acquisition device from the acquisition task to obtain an updated acquisition task within the current acquisition period; and a third determining device for re-determining the highest priority acquisition device based on the average historical acquisition success rate of each acquisition device in the updated acquisition task, and the acquisition terminal is further used to collect the highest priority data item for the re-determined highest priority acquisition device.

[0019] Preferably, the data acquisition system further comprises: a first success rate acquisition device for determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by the response time C of the jth data item of the kth acquisition device in the ith acquisition in the previous acquisition period ... kji And the following formula is used to determine the historical acquisition success rate X of the jth data item of the kth acquisition device: kj ,

[0020]

[0021] Where b is the number of collections corresponding to the j-th data item of the k-th collection device in the previous collection period; d is the success rate calculation interval.

[0022] Preferably, the data acquisition system further comprises: a second success rate acquisition device for determining the average historical acquisition success rate of each acquisition device by: determining the average historical acquisition success rate of each acquisition device according to the first historical acquisition success rate X of the j-th data item of the k-th device kj And the following formula is used to determine the average historical collection success rate X of the n data items of the k-th device: k ,

[0023]

[0024] Preferably, the data acquisition system also includes: a third success rate acquisition device, used to determine the period acquisition success rate of each collection time period within multiple collection time periods under multiple preset maximum concurrency numbers, wherein the multiple collection time periods are multiple consecutive collection time periods before the current collection time period; a fourth success rate acquisition device, used to determine the maximum period acquisition success rate when the difference between the period acquisition success rates within any two of the multiple collection time periods is less than or equal to a preset value; and a maximum concurrency number acquisition device, used to determine the preset maximum concurrency number corresponding to the maximum period acquisition success rate as the target maximum concurrency number for communication between the collection terminal and the multiple collection devices.

[0025] Preferably, the third success rate acquisition device is used to determine the period collection success rate in each of the multiple collection time periods under the multiple preset maximum concurrency numbers, including: determining the ratio of the number of data items successfully collected by all collection devices in each of the collection time periods under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices as the period collection success rate in each of the collection time periods under the multiple preset maximum concurrency numbers.

[0026] Preferably, the data acquisition system also includes: a judgment device for judging whether the acquisition task in the current acquisition period is the same as the acquisition task in the previous acquisition period; a shortest time acquisition device for determining the shortest completion time based on the response time of each data item of each acquisition device in the acquisition task in the previous acquisition period when the acquisition task in the current acquisition period is the same as the acquisition task in the previous acquisition period; and a control device for stopping the statistical operation of the historical acquisition success rate when the total duration of the acquisition task in the current acquisition period is greater than a preset multiple of the shortest completion time.

[0027] For specific details and benefits of the data acquisition system provided by the embodiment of the present invention, please refer to the above description of the data acquisition method, which will not be repeated here.

[0028] A third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data acquisition method is implemented.

[0029] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0031] Figure 1 is a flow chart of a data collection method provided by an embodiment of the present invention; and

[0032] Figure 2 It is a structural diagram of a data acquisition system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0034] Figure 1 This is a flow chart of a data acquisition method provided by an embodiment of the present invention. Figure 1As shown, the data collection method may include: step S101, for each of the multiple collection devices in the collection task within the current collection period, prioritizing each data item according to the historical collection success rate of each data item of each collection device; step S102, determining the highest priority collection device according to the average historical collection success rate of each collection device; and step S103, collecting the highest priority data item for the highest priority collection device.

[0035] The specific processes of the above steps S101-S103 are explained and illustrated below.

[0036] Step S101 : for each of the multiple collection devices in the collection task within the current collection period, priority is sorted for each data item according to the historical collection success rate of each data item of each collection device.

[0037] In various embodiments of the present application, each collection period can be reasonably set in advance, for example, every 15 minutes is a collection period.

[0038] The collection device may be a smart meter; the data items may be daily freeze, minute freeze, load data, voltage, current, alarm or event records, etc.

[0039] Before executing step S101 , the data collection method further includes: determining a historical collection success rate of each data item of each collection device in the following manner.

[0040] According to the response time C of the jth data item of the kth acquisition device during the i-th acquisition in the previous acquisition period kji And the following formula is used to determine the historical acquisition success rate X of the jth data item of the kth acquisition device: kj ,

[0041]

[0042] Where b is the number of collections corresponding to the j-th data item of the k-th collection device in the previous collection period; d is the success rate calculation interval.

[0043] Typically, embedded data acquisition devices have limited computing resources. Therefore, when calculating the historical data acquisition success rate, the existing method of verifying the distribution type by collecting response time distribution characteristics and then fitting a probability curve is not used. In the above embodiment, because the same device may not support some data items due to differences in software versions, it is necessary to consider different data items when calculating the historical data acquisition success rate. The historical data acquisition success rate is calculated using response time.

[0044] The response time in the above formula is explained below. If the time when the collection terminal sends the data frame corresponding to the jth data item to the kth collection device is t1, and the time when the collection terminal receives the response data frame of the jth data item is t2, then the response time C of the jth data item of the kth collection device is kj =t2-t1 (i.e., the time between the sending of the reading completion frame and the receiving of the completion response frame for the data item).

[0045] Specifically, when the success rate calculation interval is 5, if the number of collections corresponding to the jth data item of the kth collection device in the previous collection period is 2, then If the number of collections corresponding to the jth data item of the kth collection device in the previous collection period is 8, then Usually, the computing resources of embedded acquisition devices are limited, so the method of verifying the distribution type by collecting the response time distribution characteristics and then fitting the probability curve is not used when calculating the acquisition success rate.

[0046] The collection task is divided into multiple collection units based on the maximum supported response frame length. Each collection unit contains different data items from the same collection device. For example, a collection task may include: Collection Unit 1, Collection Unit 2, and Collection Unit 3. Collection Unit 1 includes {daily frozen data, load data, voltage, current} from collection device 1; Collection Unit 2 includes {daily frozen data, load data, voltage, current} from collection device 2; and Collection Unit 3 includes {daily frozen data, load data, voltage} from collection device 3.

[0047] According to the first data item of the collection device 1 collected in the last collection period, the daily frozen response time C 11i The above formula (1) can be used to determine the historical acquisition success rate of the first data item of acquisition device 1 - daily freezing; according to the response time C of the second data item of acquisition device 1 - load data collected for the i-th time in the previous acquisition period 12i And the above formula (1) can determine the historical collection success rate of the second data item - load data of the collection device 1; according to the response time C of the third data item - voltage of the collection device 1 collected for the i-th time in the previous collection period 13i The above formula (1) can be used to determine the historical acquisition success rate of the third data item - voltage of acquisition device 1; according to the response time C of the fourth data item - current of acquisition device 1 collected for the i-th time in the previous acquisition period, 14i Using the above formula (1), we can determine the historical acquisition success rate of the fourth data item, current, for acquisition device 1. Similarly, we can determine the historical acquisition success rate of each data item for acquisition devices 2 and 3.

[0048] The corresponding historical collection success rate is calculated with the collection unit as the smallest unit, the queue elements are prioritized according to the historical collection success rate, and each data item of each collection device is collected in descending order according to the collection success rate.

[0049] According to the historical acquisition success rate of each data item of each acquisition device determined above, each data item of each acquisition device is prioritized through step S101: the higher the historical acquisition success rate, the higher the priority. If the historical acquisition success rates of each data item of each acquisition device are sorted in descending order as follows: daily freeze of acquisition device 1, load data of acquisition device 1, daily freeze of acquisition device 2, load data of acquisition device 2, voltage of acquisition device 3, current of acquisition device 1, voltage of acquisition device 1, current of acquisition device 2, daily freeze of acquisition device 3, load data of acquisition device 3, voltage of acquisition device 2, then the priority of each data item is: {daily freeze (11), load data (10), voltage (5), current (6)} of acquisition device 1; acquisition unit 2 includes {daily freeze (9), load data (8), voltage (1), current (4)} of acquisition device 2; acquisition unit 3 includes {daily freeze (3), load data (2), voltage (7)} of acquisition device 3.

[0050] That is, within each collection unit, data items are sorted from high to low priority according to the historical collection success rate, while different collection units do not distinguish between the front and back positions.

[0051] Step S102: determining the highest priority collection device according to the average historical collection success rate of each collection device.

[0052] First, multiple collection devices (e.g., collection device 1, collection device 2, and collection device 3) can be prioritized based on the average historical collection success rate of each collection device: the higher the average historical collection success rate, the higher the priority. If the average historical collection success rates of the collection devices are sorted in descending order as collection device 2, collection device 1, and collection device 3, then the priorities from high to low are collection device 2, collection device 1, and collection device 3, and thus collection device 2 can be determined to have the highest priority.

[0053] Of course, the acquisition device with the highest average historical acquisition success rate (for example, acquisition device 2 ) may also be determined based on the average historical acquisition success rates of the various acquisition devices. Then, the acquisition device with the highest priority is acquisition device 2 .

[0054] In one embodiment, the data collection method further includes: determining an average historical collection success rate of each collection device in the following manner.

[0055] According to the first historical collection success rate X of the j-th data item of the k-th device kj And the following formula is used to determine the average historical collection success rate X of the n data items of the k-th device: k ,

[0056]

[0057] Due to the provisions of the meter communication protocol, multiple data items of the same device are usually read simultaneously in one frame of data. The success rate of collecting n data items of the same device in the same frame can be determined by the above formula.

[0058] That is, once the historical acquisition success rate of each data item for each acquisition device is obtained, the historical acquisition success rates of the individual data items in each acquisition device can be averaged to obtain the average historical acquisition success rate for each acquisition device. If multiple acquisition devices are prioritized according to their average historical acquisition success rates, multiple acquisition units (corresponding to the multiple acquisition devices) can be inserted into a list structure according to their priorities to form a data request queue.

[0059] Step S103: collecting the data item with the highest priority from the collection device with the highest priority.

[0060] In the prior art, if the collection process is executed in the order of the collection tasks, when the collection business application encounters a situation where the collection device does not respond during concurrent reading, it needs to wait for a maximum timeout period. Therefore, a longer waiting time will reduce the collection efficiency and thus lead to the inability to complete the collection task within a collection period. In this embodiment, if the priority from high to low in the data request queue obtained by step S102 is: collection device 2, collection device 1, collection device 3. Therefore, it can be determined that the collection device with the highest priority is collection device 2. Moreover, according to the priority of each data item of each collection device obtained in step S101, it can be determined that the data item with the highest priority of collection device 2 is daily freeze (9). Therefore, the daily freeze of collection device 2 can be collected first. In this collection process, even if the collection device does not respond during concurrent reading, there is no need to wait for a long time.

[0061] In the following embodiments, in addition to the highest-priority data items of the highest-priority acquisition device, the average historical acquisition success rate of the highest-priority acquisition device may change, so that the priority of the highest-priority acquisition device is no longer the highest priority: compared with other acquisition devices, the average historical acquisition success rate of the highest-priority acquisition device is lower than the average historical acquisition success rates of other acquisition devices. Based on this, the priorities of multiple acquisition devices need to be re-ordered to update the highest-priority acquisition device.

[0062] The data collection method may also include: obtaining updated data items of the highest priority collection device, wherein the updated data items do not include the highest priority data items; re-determining the highest priority collection device based on the updated average historical collection success rate of the highest priority collection device and the average historical collection success rate of other collection devices in the collection tasks within the current collection period; and collecting the highest priority data items for the re-determined highest priority collection device.

[0063] Specifically, the updated data items of the acquisition device with the highest priority are obtained as follows: acquisition unit 2 includes {load data (8), voltage (1), current (4)} of acquisition device 2; then, the average historical acquisition success rate of acquisition device 2 is updated according to the above formula for determining the average historical acquisition success rate, and the updated average historical acquisition success rate of acquisition device 2 is compared with the average historical acquisition success rates of other acquisition devices (for example, acquisition device 1 and acquisition device 3). If the updated average historical acquisition success rates are still acquisition device 2, acquisition device 1, and acquisition device 3 in descending order, then the priorities from high to low are still acquisition device 2, acquisition device 1, and acquisition device 3, thereby re-determining that the acquisition device with the highest priority is still acquisition device 2. Moreover, according to the priority of each data item of each acquisition device obtained in step S101, it can be determined that the data item with the highest priority of acquisition device 2 is load data (8). Therefore, the load data of acquisition device 2 can be collected first.

[0064] When a new collection device is inserted into the data request queue, the device priority will be re-sorted.

[0065] After each collection period, the collection success rates of different data items on each device are sorted and inserted into a historical collection success rate list. This list provides an automated export method to facilitate troubleshooting of devices with low collection success rates. To facilitate troubleshooting, the historical collection success rate list uses the collection success rate of a data item as the smallest unit. For example, if the collection success rate of other data items on the same device is high, but the collection success rate of a data item is 0, it can be determined that the device is likely not capable of reading that data item.

[0066] Considering that the maximum concurrency (the maximum concurrency refers to the maximum number of concurrent data requests allowed when the acquisition terminal performs data reading during carrier communication) has a significant impact on the acquisition efficiency, in this embodiment, the maximum concurrency can be tested for the reading efficiency to obtain the maximum concurrency with the highest efficiency.

[0067] In one embodiment, the data collection method may further include: determining the period collection success rate in each of multiple collection time periods under multiple preset maximum concurrency numbers, wherein the multiple collection time periods are multiple consecutive collection time periods before the current collection time period; determining the maximum period collection success rate when the difference between the period collection success rates in any two of the multiple collection time periods is less than or equal to a preset value; and determining the preset maximum concurrency number corresponding to the maximum period collection success rate as the target maximum concurrency number for communication between the collection terminal and the multiple collection devices.

[0068] Among them, determining the period collection success rate in each of the multiple collection time periods under the multiple preset maximum concurrency numbers includes: determining the ratio of the number of data items successfully collected by all collection devices in each of the collection time periods under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices as the period collection success rate in each of the collection time periods under the multiple preset maximum concurrency numbers.

[0069] For example, the multiple preset maximum concurrency numbers may be three typical values. Three typical values ​​of the maximum concurrency number are selected for testing, and the typical values ​​are selected based on historical application experience and obtained through the configuration file. In order to minimize the impact of other factors on the copying efficiency, it is necessary to test based on historical data with relatively stable collection success rates. First, in the case of each of the three typical values, the period collection success rate in each of the three collection periods before the current collection period is determined; then, when it is determined that the maximum difference in the period collection success rate between any two collection periods in the first three collection periods does not exceed 5, determine which typical value has the largest period collection success rate, and determine this typical value as the target maximum concurrency number. Otherwise, continue to wait for the test conditions and determine any one of the three typical values ​​as the target maximum concurrency number.

[0070] This embodiment proposes a testing method to select the maximum concurrency number that results in a high acquisition success rate in real-world scenarios. By testing to determine the maximum concurrency number that provides the highest acquisition success rate, acquisition efficiency is further improved. Because the maximum concurrency test is performed during a period of relatively stable acquisition success rates, it minimizes the impact of other factors on the acquisition success rate, thereby ensuring the effectiveness of the maximum concurrency test.

[0071] In one embodiment, the data collection method may further include: determining whether the collection task in the current collection period is the same as the collection task in the previous collection period; if the collection task in the current collection period is the same as the collection task in the previous collection period, determining the shortest completion time based on the response time of each data item of each collection device in the collection task in the previous collection period; and if the total duration of the collection task in the current collection period is greater than a preset multiple of the shortest completion time, stopping the statistical operation of the historical collection success rate.

[0072] Specifically, if the collection task in the current collection period includes collection unit 1, collection unit 2 and collection unit 3, wherein collection unit 1 includes {daily freeze, load data, voltage, current} of collection device 1; collection unit 2 includes {daily freeze, load data, voltage, current} of collection device 2; collection unit 3 includes {daily freeze, load data, voltage} of collection device 3, then determine whether the collection task in the previous collection period also includes the above three collection units. If the collection task in the previous collection period also includes the above three collection units, then add the response time of all data items of the three collection devices in the collection task one by one to obtain the shortest completion time of the task; then compare the total duration of the collection task in the current collection period with the shortest completion time. If the total duration is greater than 2 times the shortest completion time, it indicates that the collection success rate in each period under the substation is high, and the success rate statistics function can be turned off by configuration to save computing resources. Since the present application can be applied to scenarios with more devices or high-frequency collection requirements, in most cases, the collection tasks in multiple consecutive collection periods are consistent. If the collection task in the previous collection period does not include the above three collection units, the success rate statistics function is kept open. The statistics function is automatically turned on for the first period starting at midnight every day. Of course, in other embodiments, a reasonable preset multiple can be set according to actual needs.

[0073] Existing collection-related business applications do not consider the historical collection success rate distribution characteristics of the collection devices when arranging the collection data items for different devices, making it impossible to specifically adjust the collection plan and thus improve collection efficiency. The above-mentioned embodiments can be executed by the collection terminal, and their purpose is to: by calculating the distribution characteristics of the historical collection success rate, arrange the collection order of the corresponding data items of specific devices in each collection period according to the rule of prioritizing the collection of devices with high historical collection success rates, thereby improving the collection efficiency of the collection task. Because traditional collection methods lack automated means to collect statistics for devices with low collection success rates, it is necessary to provide a list of historical collection success rates for each time period to facilitate export and troubleshooting.

[0074] In summary, the present invention creatively determines the priority of each data item of each of the multiple collection devices in the collection task within the current collection period based on the historical collection success rate of each data item of each collection device; then, determines the highest priority collection device based on the average historical collection success rate of each collection device; and finally, collects the highest priority data item for the highest priority collection device. Thus, the present invention prioritizes different data items of the same device based on their historical collection success rates, then prioritizes them based on the average historical collection success rates of different devices, and prioritizes the collection of the highest priority data items of the highest priority device, thereby greatly improving collection efficiency when faced with scenarios with a large number of devices or high-frequency collection requirements.

[0075] Figure 2 FIG. 1 is a structural diagram of a data acquisition system provided by an embodiment of the present invention. Figure 2 As shown, the data acquisition system may include: a first determination device 10, for determining the priority of each data item of each acquisition device among multiple acquisition devices in the acquisition task within the current acquisition period according to the historical acquisition success rate of each data item of each acquisition device; a second determination device 20, for determining the highest priority acquisition device according to the average historical acquisition success rate of each acquisition device; and an acquisition terminal 30, for collecting the highest priority data items for the highest priority acquisition device.

[0076] Preferably, the data acquisition system further includes: a deleting device for deleting the highest priority data item of the highest priority acquisition device from the acquisition task to obtain an updated acquisition task within the current acquisition period; and a third determining device for re-determining the highest priority acquisition device based on the average historical acquisition success rate of each acquisition device in the updated acquisition task, and the acquisition terminal is further used to collect the highest priority data item for the re-determined highest priority acquisition device.

[0077] Preferably, the data acquisition system further comprises: a first success rate acquisition device for determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by: determining the historical acquisition success rate of each data item of each acquisition device by the response time C of the jth data item of the kth acquisition device in the ith acquisition in the previous acquisition period ... kji And the following formula is used to determine the historical acquisition success rate X of the jth data item of the kth acquisition device: kj ,

[0078]

[0079] Where b is the number of collections corresponding to the j-th data item of the k-th collection device in the previous collection period; d is the success rate calculation interval.

[0080] Preferably, the data acquisition system further comprises: a second success rate acquisition device for determining the average historical acquisition success rate of each acquisition device by: determining the average historical acquisition success rate of each acquisition device according to the first historical acquisition success rate X of the j-th data item of the k-th device kj And the following formula is used to determine the average historical collection success rate X of the n data items of the k-th device: k ,

[0081]

[0082] Preferably, the data acquisition system also includes: a third success rate acquisition device, used to determine the period acquisition success rate of each collection time period within multiple collection time periods under multiple preset maximum concurrency numbers, wherein the multiple collection time periods are multiple consecutive collection time periods before the current collection time period; a fourth success rate acquisition device, used to determine the maximum period acquisition success rate when the difference between the period acquisition success rates within any two of the multiple collection time periods is less than or equal to a preset value; and a maximum concurrency number acquisition device, used to determine the preset maximum concurrency number corresponding to the maximum period acquisition success rate as the target maximum concurrency number for communication between the collection terminal and the multiple collection devices.

[0083] Preferably, the third success rate acquisition device is used to determine the period collection success rate in each of the multiple collection time periods under the multiple preset maximum concurrency numbers, including: determining the ratio of the number of data items successfully collected by all collection devices in each of the collection time periods under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices as the period collection success rate in each of the collection time periods under the multiple preset maximum concurrency numbers.

[0084] Preferably, the data acquisition system also includes: a judgment device for judging whether the acquisition task in the current acquisition period is the same as the acquisition task in the previous acquisition period; a shortest time acquisition device for determining the shortest completion time based on the response time of each data item of each acquisition device in the acquisition task in the previous acquisition period when the acquisition task in the current acquisition period is the same as the acquisition task in the previous acquisition period; and a control device for stopping the statistical operation of the historical acquisition success rate when the total duration of the acquisition task in the current acquisition period is greater than a preset multiple of the shortest completion time.

[0085] For specific details and benefits of the data acquisition system provided by the embodiment of the present invention, please refer to the above description of the data acquisition method, which will not be repeated here.

[0086] An embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the data acquisition method is implemented.

[0087] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the scope of protection of the present invention.

[0088] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0089] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a single-chip microcomputer, chip or processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0090] In addition, the various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A data collection method, characterized in that: The data collection method comprises: For each of the multiple collection devices in the collection task within the current collection period, determine the priority of each data item according to the historical collection success rate of each data item of each collection device; Determining a highest priority collection device based on an average historical collection success rate of each collection device, wherein the average historical collection success rate of each collection device is calculated based on the historical collection success rates of all data items of the collection device; and collecting the data item with the highest priority on the collection device with the highest priority, The historical collection success rate of each data item of each collection device is determined by: According to the i-th collection in the last collection period k The first acquisition device j Response time for each data item And the following formula, determine the k The first acquisition device j The historical collection success rate of data items , ， in, b The first k The first acquisition device j The number of collections corresponding to each data item; d Calculate the success rate interval.

2. The data collection method according to claim 1, characterized in that: The data collection method further includes: Obtaining updated data items of the highest priority acquisition device, wherein the updated data items do not include the highest priority data item; Re-determining the highest priority collection device based on the updated average historical collection success rate of the highest priority collection device and the average historical collection success rates of other collection devices in the collection tasks within the current collection period; and The data item with the highest priority is collected by the collection device with the re-determined highest priority.

3. The data collection method according to any one of claims 1 to 2, characterized in that: The average historical acquisition success rate for each acquisition device is determined by: According to the said k The first device j The first historical collection success rate of the data item And the following formula, determine the k devices n The average historical collection success rate of data items , 。 4. The data collection method according to any one of claims 1 to 2, characterized in that: The data collection method further includes: Determining a period collection success rate in each of a plurality of collection periods under a plurality of preset maximum concurrency numbers, wherein the plurality of collection periods are a plurality of consecutive collection periods before a current collection period; If the difference between the period acquisition success rates in any two of the plurality of acquisition time periods is less than or equal to a preset value, determining a maximum period acquisition success rate; and The preset maximum concurrent number corresponding to the maximum time period acquisition success rate is determined as the target maximum concurrent number for communication between the acquisition terminal and the plurality of acquisition devices.

5. The data collection method according to claim 4, characterized in that: Determining the period collection success rate in each of the multiple collection periods under the multiple preset maximum concurrency numbers includes: The ratio of the number of data items successfully collected by all collection devices in each collection period under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices is determined as the period collection success rate in each collection period under the multiple preset maximum concurrency numbers.

6. The data collection method according to any one of claims 1-2, characterized in that: The data collection method further includes: Determining whether the collection task in the current collection period is the same as the collection task in the previous collection period; If the collection task in the current collection period is the same as the collection task in the previous collection period, determining the shortest completion time based on the response time of each data item of each collection device in the collection task in the previous collection period; and When the total duration of the collection tasks in the current collection period is greater than a preset multiple of the shortest completion time, the statistical operation of the historical collection success rate is stopped.

7. A data acquisition system, characterized in that: The data acquisition system includes: The first determining means is configured to determine, for each of the plurality of collection devices in the collection task within the current collection period, the priority of each data item according to a historical collection success rate of each data item of each collection device; a second determining means for determining a collection device with the highest priority according to an average historical collection success rate of each collection device, wherein the average historical collection success rate of each collection device is calculated based on the historical collection success rates of all data items of the collection device; and a collection terminal, configured to collect the data item of the highest priority from the collection device of the highest priority; The data collection system further includes: a first success rate acquisition device, configured to determine a historical collection success rate of each data item of each collection device by: According to the i-th collection in the last collection period k The first acquisition device j Response time for each data item And the following formula, determine the k The first acquisition device j The historical collection success rate of data items , ， in, b The first k The first acquisition device j The number of collections corresponding to each data item; d Calculate the success rate interval.

8. The data acquisition system according to claim 7, characterized in that: The data acquisition system also includes: a deleting device for deleting the highest-priority data item of the highest-priority collection device from the collection task to obtain an updated collection task within the current collection period; and The third determining means is configured to re-determine the highest priority acquisition device based on the average historical acquisition success rate of each acquisition device in the updated acquisition task, The collection terminal is further configured to collect the data item with the highest priority from the collection device with the re-determined highest priority.

9. The data acquisition system according to any one of claims 7-8, characterized in that: The data acquisition system further includes: a second success rate acquisition device, configured to determine the average historical acquisition success rate of each acquisition device by: According to the said k The first device j The first historical collection success rate of the data item And the following formula, determine the k devices n The average historical collection success rate of data items , 。 10. The data acquisition system according to any one of claims 7-8, characterized in that: The data acquisition system also includes: A third success rate acquisition device is used to determine the period acquisition success rate of each collection period in a plurality of collection periods under a plurality of preset maximum concurrency numbers, wherein the plurality of collection periods are a plurality of consecutive collection periods before the current collection period; a fourth success rate obtaining means for determining a maximum time period acquisition success rate when a difference between time period acquisition success rates in any two of the plurality of acquisition time periods is less than or equal to a preset value; and The maximum concurrent number obtaining device is used to determine the preset maximum concurrent number corresponding to the maximum time period acquisition success rate as the target maximum concurrent number for communication between the acquisition terminal and the multiple acquisition devices.

11. The data acquisition system according to claim 10, characterized in that: The third success rate acquisition device is used to determine the period acquisition success rate of each acquisition period in multiple acquisition periods under multiple preset maximum concurrency numbers, including: The ratio of the number of data items successfully collected by all collection devices in each collection period under the multiple preset maximum concurrency numbers to the total number of data items of all collection devices is determined as the period collection success rate in each collection period under the multiple preset maximum concurrency numbers.

12. The data acquisition system according to any one of claims 7-8, characterized in that: The data acquisition system also includes: A judging device, configured to judge whether the collection task in the current collection period is the same as the collection task in the previous collection period; a shortest time obtaining means for determining the shortest completion time based on the response time of each data item of each collection device in the collection task in the previous collection period when the collection task in the current collection period is the same as the collection task in the previous collection period; and The control device is used to stop the statistical operation of the historical collection success rate when the total duration of the collection task in the current collection period is greater than a preset multiple of the shortest completion time.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the data acquisition method according to any one of claims 1 to 6.

Images