Method and system for pre-positioning supplementary collection interval in auxiliary collection

Abstract

The invention discloses a method for pre-positioning a supplementary collection interval through auxiliary collection, and the method mainly comprises the following steps: obtaining a collection device file and strategy configuration information, and carrying out the initialization of a to-be-collected list; marking the successfully acquired data to an acquisition list in real time; receiving and analyzing a pre-complementary collection instruction sent by the collection preposition; and traversing the to-be-collected list to obtain data marked as uncollected data and pushing the data to the collection preposition. According to the method, the to-be-collected list is maintained in real time, the to-be-collected pre-collection instruction is responded in time, and the to-be-collected data is traversed and pushed, so that the complementary collection interval is rapidly and accurately positioned, the collection front is assisted to reduce the resource consumption of positioning the complementary collection interval, the problems of repeated data collection and collection leakage are reduced, and the data collection efficiency is improved. And the working efficiency and the intelligent working level are improved for acquisition preposition.

Description

Technical Field

[0001] The invention discloses an auxiliary collection pre-positioning supplementary collection interval method and system, and relates to the technical field of electric energy metering. Background Art

[0002] With the widespread application and development of big data technology, data analysis and processing capabilities have been greatly improved. The requirements of various businesses on data volume and type have become increasingly higher, while the collection pressure faced by the underlying collection system has been increasing.

[0003] The collection front-end is the core component of the collection system. Its main task is to collect data into the system after communicating with the collection equipment. When faced with massive amounts of data, the challenge faced by the collection front-end is how to continuously ensure the high efficiency and high quality of the collection operation. From the perspective of supplementary collection alone, at present, each new round of supplementary collection tasks carried out by the collection front-end requires a long time and more resources to associate and filter archives and data tables when locating the supplementary collection interval, and there are also some duplicate collection and missed collection results. Summary of the invention

[0004] In view of the defects in the above-mentioned background technology, the present invention provides a method and system for auxiliary acquisition of pre-positioning supplementary acquisition interval, which can realize rapid and accurate positioning of supplementary acquisition interval and improve the level of intelligent work.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: a method for auxiliary acquisition of pre-positioning supplementary acquisition interval, comprising the following steps:

[0006] The master station system obtains the collection equipment files and policy configuration information, initializes and generates a full collection list, and marks the successfully collected data to the list in real time; after receiving and parsing the pre-supplementary collection instructions sent by the collection front-end, it traverses the list to obtain the data marked as not collected and pushes it to the collection front-end;

[0007] The collection front receives the supplementary collection range of the data marked as not collected, and sends the supplementary collection instruction to the collection device terminal;

[0008] The acquisition device terminal receives the supplementary acquisition instruction and returns the supplementary acquisition data within the supplementary acquisition range to the main station system.

[0009] Furthermore, the acquisition of the collection equipment files and policy configuration information and the initialization of the generation of the full collection list specifically include the following steps:

[0010] Batch load acquisition device archives and policy configuration information,

[0011] Then, according to the different classifications of collection equipment, taking the collection equipment as the main body and combining the collection frequency of each data item, initialize and generate a full collection list of corresponding frequency;

[0012] Furthermore, the initialization data items in the list to be adopted are marked as "not adopted" and "not collected" types respectively.

[0013] Furthermore, the real-time marking of successfully collected data to the list to be purchased specifically includes the following steps: through the stream processing capability, the collection mark of the data item in the list to be purchased is updated and maintained in real time, changing it from "not collected" to "collected".

[0014] Furthermore, the receiving and parsing of the pre-supplementary sampling instruction sent by the acquisition front-end specifically includes the following steps:

[0015] The main station system communicates with the collection front end through the message queue, subscribes to obtain the pre-supplementary collection instructions in Json format and parses them;

[0016] The contents of instruction parsing include: acquisition device type, region, acquisition date, and data items.

[0017] Furthermore, the traversal of the list to be collected to obtain the data marked as "not collected" and push it to the collection front-end specifically includes:

[0018] By judging whether the collection mark of the data item in the collection list is "not collected", the collection device set of "not collected" is encapsulated into Json format data and pushed to the message queue, and then pushed to the collection front end.

[0019] Furthermore, the acquisition device archive and policy configuration information includes: archive information and data item information:

[0020] The archive information includes: terminal unique identification, terminal type, meter unique identification, and region;

[0021] The data item information includes: electric energy indication, demand, and quadrant reactive power.

[0022] An auxiliary acquisition front-end positioning supplementary acquisition interval system, comprising: a main station system, acquisition front-end and acquisition equipment terminal;

[0023] Main station system: used to obtain the collection equipment files and policy configuration information, initialize the generation of the full collection list, and mark the successfully collected data to the list in real time, receive and parse the pre-collection instructions sent by the collection front-end, traverse the list to obtain the data marked as not collected and push it to the collection front-end;

[0024] The pre-collection stage is used to receive the supplementary collection range of the data marked as uncollected and send the supplementary collection instruction to the collection device terminal;

[0025] The acquisition device terminal is used to receive supplementary acquisition instructions and return the data in the supplementary acquisition range to the main station system.

[0026] Beneficial effect: The present invention realizes fast and accurate positioning of the supplementary collection interval by maintaining the list of items to be collected in real time, responding to the pre-collection and pre-supplementary collection instructions in a timely manner, and traversing and pushing the data to be supplemented, thereby assisting the pre-collection to reduce the resource consumption of locating the supplementary collection interval, as well as reducing the problems of repeated data collection and missed collection, thereby improving the work efficiency and intelligent work level of the main station pre-collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the implementation of the method of the present invention;

[0028] Figure 2 It is a schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0029] The implementation of the technical solution is further described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0030] like Figure 1~2 An embodiment shown in the figure provides a method for assisting in the acquisition of a pre-positioned supplementary acquisition interval, comprising the following steps:

[0031] Obtain the collection equipment files and policy configuration information, and initialize the generation of a full collection list;

[0032] Through the Spark program, the province's terminal files, meter files, meter parameter information, data item information, protocol information, meter collection plan, etc. are batch loaded for associated operations, and the daily frozen list of meters collected in the province is initialized and saved in the HBase database. The daily frozen data items include daily frozen forward / reverse / reactive electric energy (total and various rates), daily frozen first / second / third / fourth quadrant reactive indications (total and various rates), daily frozen maximum forward / reverse active demand and occurrence time of the month (total and various rates), etc.

[0033] In the electricity consumption information collection system, one terminal will be connected to multiple meters. At the same time, the collection front-end is to generate and issue collection tasks based on the terminal as the basic unit. Taking into account the performance and usability of the list to be collected, the data structure of the daily frozen list of electricity meters collected in the whole province is finally designed and generated as shown in Table 1.

[0034] Table 1 Data structure for initializing the list of items to be collected for daily freezing of electric meters in the province

[0035] meaning name Example Data unique identifier RowKey 000c20210715300*****588 Statistical date Stat_date 2021-07-15 Terminal unique identifier Tmnl_id 300*****588 Area Code Org_no 3**01 Terminal Type Tmnl_type_code *5 Electricity meter collection Meter_ids 15****7517,15****7518,15****7519 Electric energy indication (forward / reverse active / reactive) Day_energy 0000,0000,0000 Demand (forward / reverse active / reactive) Day_demand 00NN,00NN,00NN Quadrant reactive power (one / two / three / four) Day_read 0000,NNNN,NNNN

[0036] Supplementary explanation for the information in Table 1:

[0037] 1) Data item collection mark code value meaning 0: not collected, 1: collected, N: not collected;

[0038] 2) Meter_ids, Day_energy, Day_demand and Day_read must be arranged in the same order by default;

[0039] 3) Taking Day_energy as an example, Day_energy: 0000,0000,0000 means that the forward active power, reverse active power, forward reactive power and reverse reactive power collection marks corresponding to the three meters 15****7517, 15****7518 and 15****7519 in Meter_ids are all in the uncollected state;

[0040] 4) RowKey design: four-digit salt field + statistical date + terminal unique identifier.

[0041] Mark successfully collected data to the collection list in real time:

[0042] Subscribe to the Kafka message queue of collected data through the Flink program, and make judgments on the collected data of the daily frozen topic in real time. After the conditions for successful collection are met, update the list of daily frozen electricity meters collected in the province, and change the collection mark of the corresponding successful electricity meter under the corresponding terminal to collected.

[0043] The acquisition front-end generates and sends pre-supplementary acquisition instructions to the master station system, and the master station system receives and parses the pre-supplementary acquisition instructions sent by the acquisition front-end:

[0044] By subscribing to the data of the pre-supplementary procurement instruction topic in the Kafka message queue, the pre-supplementary procurement instructions from the collection front are received and parsed.

[0045] The contents of instruction parsing include: requestId: the unique identifier of this supplementary collection instruction, orgNo: collection area, date: collection date, dataType: collection equipment type, dataItem: data item.

[0046] Traverse the list of items to be collected to obtain the data marked as not collected and push it to the collection frontend:

[0047] After the command is parsed, according to dataType, it is determined that the list of daily frozen items of electricity meters collected in the province needs to be accessed. According to orgNo, date and dataItem, the scope of this supplementary purchase can be determined, and the set of meters marked as uncollected in the corresponding data items can be obtained. The data is encapsulated into Json format and pushed to the Kafka message queue, so that the collection front-end consumption can generate and issue the corresponding supplementary purchase task. The collection front-end receives the supplementary purchase scope of the data marked as uncollected, and issues the supplementary purchase instruction to the collection device terminal;

[0048] The terminal of the acquisition device receives the supplementary acquisition instruction and returns the data in the supplementary acquisition range to the main station system, thereby realizing the complete acquisition of the data of the main station system. From the above content, it can be seen that the returned result is a three-layer structure of Json format data, and the three layers from the outside to the inside are terminal, meter, and data item.

[0049] The present invention realizes fast and accurate positioning of the supplementary collection interval by maintaining the list of items to be collected in real time, responding to the pre-collection and pre-supplementary collection instructions in a timely manner, and traversing and pushing the data to be supplemented, thereby assisting the pre-collection to reduce the resource consumption of locating the supplementary collection interval, as well as reducing the problems of repeated data collection and missed collection, thereby improving the work efficiency and intelligent work level of the main station pre-collection.

[0050] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for auxiliary acquisition of pre-positioning and supplementary acquisition interval, characterized in that: The following steps are involved: The master station system obtains the collection equipment files and policy configuration information, initializes the generation of the full collection list, and marks the successfully collected data to the list in real time; After receiving and parsing the pre-supplementary procurement instructions sent by the collection front-end, traverse the procurement list to obtain the data marked as uncollected and push it to the collection front-end; The collection front receives the supplementary collection range of the data marked as not collected, and sends the supplementary collection instruction to the collection device terminal; The acquisition device terminal receives the supplementary acquisition instruction and returns the supplementary acquisition data within the supplementary acquisition range to the main station system.

2. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 1 is characterized in that: The steps of obtaining the collection equipment files and policy configuration information and initializing the generation of the full collection list specifically include the following steps: Batch load acquisition device archives and policy configuration information, Then, according to the different classifications of collection equipment, with the collection equipment as the main body and combined with the collection frequency of each data item, initialize and generate a full collection list with the corresponding frequency.

3. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 2 is characterized in that: The initialization data items in the list to be adopted are marked as "not adopted" and "not collected" types respectively.

4. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 3 is characterized in that: The real-time marking of successfully collected data to the list to be purchased specifically includes the following steps: through the stream processing capability, the collection mark of the data item in the list to be purchased is updated and maintained in real time, changing it from "not collected" to "collected".

5. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 1 is characterized in that: The receiving and parsing of the pre-supplementary acquisition instruction sent by the acquisition front-end specifically comprises the following steps: The main station system communicates with the collection front end through the message queue, subscribes to obtain the pre-supplementary collection instructions in Json format and parses them; The contents of instruction parsing include: acquisition device type, region, acquisition date, and data items.

6. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 1, characterized in that: The traversal of the list to be collected to obtain the data marked as "not collected" and push it to the collection front-end specifically includes: By judging whether the collection mark of the data item in the collection list is "not collected", the collection equipment set of "not collected" is encapsulated into Json format data and pushed to the message queue, and then pushed to the collection front.

7. The method for auxiliary acquisition pre-positioning and supplementary acquisition interval according to claim 2 is characterized in that: The acquisition equipment archive and policy configuration information includes: archive information and data item information: The archive information includes: terminal unique identification, terminal type, meter unique identification, and region; The data item information includes: electric energy indication, demand, and quadrant reactive power.

8. An auxiliary collection pre-positioning supplementary collection interval system, characterized in that: include: Main station system, acquisition front-end and acquisition equipment terminal; Main station system: used to obtain the collection equipment files and policy configuration information, initialize and generate the full collection list, mark the successfully collected data to the collection list in real time, receive and parse the pre-collection instruction sent by the collection front-end, traverse the collection list to obtain the data marked as not collected and push it to the collection front-end; The pre-collection stage is used to receive the supplementary collection range of the data marked as uncollected and send the supplementary collection instruction to the collection device terminal; The acquisition device terminal is used to receive supplementary acquisition instructions and return the data in the supplementary acquisition range to the main station system.

Images