Data processing method, apparatus, and computer program product
By parsing and splitting the data packets sent by the plant side in the centralized control system, ensuring that only the first data record is retained, the timing conflict problem in the data packets is resolved, and the stability and data integrity of the centralized control system are maintained.
Patent Information
- Application Number
- CN202610314036.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-26
- Estimated Expiration
- 2046-03-16
AI Technical Summary
In a centralized control system, when remote signaling signals and event sequence records generated by plant-side equipment at near-simultaneous intervals are packaged and sent, it may cause a conflict between two time-series data records with the same information object address and timestamp in the data packet, resulting in data loss, disrupting the integrity of the time-series logic chain, and affecting system stability.
After receiving data packets, the plant-side server parses out multiple data records with the same information object address and the same timestamp, identifies the first data record, and extracts the remaining data records from the data packets and resends them to the central control side to avoid timing conflicts.
This ensures that the data packets received by the centralized control side do not contain data records that cause timing conflicts, thus maintaining the stability of the centralized control system and avoiding data loss, false alarms, and missed alarms.
Smart Images
Figure CN121864714B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of centralized control technology, and in particular to a data processing method, apparatus and computer program product. Background Technology
[0002] In a centralized control system, the central control unit needs to monitor multiple subordinate plant / station units. The plant / station units can package and send the collected data to the central control unit via data packets.
[0003] However, the data acquisition devices within the plant may generate an instantaneous remote signal and a corresponding event sequence record almost simultaneously, packaging them into a data packet and sending it to the centralized control side. When the centralized control side receives this data packet, it can parse out two time-series data records with the same information object address and data acquisition timestamp. When storing these two data records in the database, a conflict may occur, causing the later-written record to overwrite the earlier-written record. This results in the loss of a valid time-series event in the data packet, disrupting the integrity of the timing logic chain, leading to missed or false alarms, and affecting the stability of the centralized control system. Summary of the Invention
[0004] To overcome the problems existing in the related technologies, this disclosure provides a data processing method, apparatus and computer program product that can solve the above problems.
[0005] According to a first aspect of the present disclosure, a data processing method is provided, the method comprising: receiving a first data packet formed by packaging data collected from equipment at a plant / station side; parsing the first data packet, and if multiple data records with the same timestamp issued by the same information object address are parsed from the first data packet, determining a first data record among the multiple data records; extracting the remaining data records other than the first data record from the first data packet; and sending the first data packet after extracting the remaining data records to at least a centralized control side.
[0006] According to a second aspect of the present disclosure, a data processing apparatus is provided, the apparatus comprising: a receiving unit configured to receive a first data packet formed by packaging data collected from equipment at a plant side; a parsing unit configured to parse the first data packet, and, if multiple data records with the same timestamp issued by the same information object address are parsed from the first data packet, to determine a first data record among the multiple data records; a splitting unit configured to extract the remaining data records other than the first data record from the multiple data records from the first data packet; and a sending unit configured to send the first data packet after extracting the remaining data records to at least a centralized control side.
[0007] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor and a memory; the memory being used to store a computer program; and the processor being used to execute the data processing method as described in the first aspect by invoking the computer program.
[0008] According to a fourth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in the first aspect.
[0009] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:
[0010] This disclosure allows receiving and parsing a first data packet sent by the plant / station side. If the first data packet contains multiple data records with the same timestamp originating from the same information object address, the first data record can be identified from these multiple data records, and the remaining data records can be extracted from the first data packet. The first data packet with the remaining data records extracted is then resent to the central control side.
[0011] The embodiments of this disclosure can parse data packets before receiving them at the centralized control side to determine whether the data packets contain multiple data records that may cause timing conflicts. If multiple data records that may cause timing conflicts are determined to exist, the first data record can be retained, and the remaining data records can be retrieved before sending the data packet to the centralized control side. This ensures that the data packet received by the centralized control side does not contain any other data records that could cause timing conflicts with the first data record. This avoids the centralized control side obtaining multiple data records that may cause timing conflicts after parsing the same data packet, thereby avoiding overwriting and conflict problems caused by multiple data records having the same information object address and timestamp during storage in the time-series database, and thus maintaining the stability of the centralized control system.
[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0013] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0014] Figure 1 This disclosure is a schematic diagram of the architecture of a centralized control system according to an exemplary embodiment.
[0015] Figure 2 This is a schematic flowchart illustrating a data processing method according to an exemplary embodiment of the present disclosure.
[0016] Figure 3 This disclosure is a schematic diagram of the architecture of a centralized control system according to an exemplary embodiment.
[0017] Figure 4 This is a block diagram of a data processing apparatus according to an exemplary embodiment of the present disclosure.
[0018] Figure 5 This is a schematic block diagram illustrating a data processing apparatus according to an exemplary embodiment of the present disclosure. Detailed Implementation
[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0020] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0021] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0022] In centralized control systems (such as new energy centralized control systems), in order to achieve centralized monitoring of multiple subordinate plants and stations, data packets can be used to collect and transport monitoring data between the centralized control master station and each plant and station.
[0023] Some protocols (such as IEC 104) allow multiple Application Service Data Units (ASDUs) to be encapsulated within a single TCP packet. When protection and / or monitoring and control devices within a plant are triggered by the same event (such as a switch change), they can generate an instantaneous remote signaling signal and a corresponding Sequence of Events (SOE) record with millisecond-level timestamps almost simultaneously. These two data records are then encapsulated in a single packet and sent to the centralized control side.
[0024] To handle the access and storage of large volumes of high-frequency plant data, the centralized control unit typically stores the data in a time-series database. This database uses the data acquisition timestamp as the primary key to construct a time-series index, enabling millisecond-level high-speed writing and querying. However, upon receiving a message, the centralized control unit can parse two time-series data records with identical Information Object Addresses (IOAs) and identical millisecond-level data acquisition timestamps, but different type identifiers. These two records are then judged by the time-series database as duplicate data pointing to the same information object at the same time node, leading to a time-series primary key conflict. This can cause a later-written record to overwrite an earlier-written record, resulting in the loss of a valid time-series event in remote signaling or SOE (Service Execution Environment). This disrupts the integrity of the time-series logic chain, potentially causing missed or false alarms in real-time alarms, and affecting the accuracy and reliability of advanced application functions such as accident tracing and fault inversion in the centralized control system, posing a threat to the safe and stable operation of the system.
[0025] To address the aforementioned technical problems, this disclosure proposes a data processing method.
[0026] Figure 1 This is a schematic diagram of the architecture of a centralized control system according to an embodiment of the present disclosure.
[0027] like Figure 1 As shown, the centralized control system includes: a plant-side server 110, a centralized control-side server 120, and a time-series database 130.
[0028] The plant-side server 110, which can be multiple and set up in different plants, is used to collect data from sensors or measurement and control devices set up in the plant, process the data, package it and send it to the central control server 120. After receiving the instructions issued by the central control server 120, it can control the sensors, measurement and control devices or other equipment in the plant based on the instructions.
[0029] The centralized control server 120 is used to receive data packets sent by multiple plant-side servers 110 and parse the data packets; the centralized control server 120 can store the data records in the data packets into the time-series database 130 to form a time-series record, recording various events of the plant-side equipment; the centralized control server 120 can also determine instructions based on the data records in the data packets and issue instructions to the plant-side servers 110 to instruct the plant-side servers 110 to perform corresponding control according to the instructions.
[0030] The time-series database 130 is used to receive time-series data records from the central control server 120. The time-series database 130 can store the incoming time-series data records and establish a time-series index to construct a data time-series logical chain, thereby facilitating the implementation of functions such as missed alarms and false alarms, accident replay, fault inversion, and data query.
[0031] In some embodiments, the centralized control system may include a new energy centralized control system.
[0032] The data processing method proposed in this disclosure can be executed by the plant-side server 110.
[0033] Figure 2 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure. The data processing method can be executed by the plant-side server 110 in the centralized control system. This data processing method can be used to process data in data packets collected from the plant side.
[0034] like Figure 2 As shown, the data processing methods include:
[0035] In step S201, a first data packet is received, which is formed by packaging data collected from the equipment on the plant side.
[0036] In step S202, the first data packet is parsed, and if multiple data records with the same timestamp sent from the same information object address are parsed from the first data packet, the first data record among the multiple data records is determined;
[0037] In step S203, the remaining data records, excluding the first data record, are extracted from the first data packet.
[0038] In step S204, at least the first data packet after retrieving the remaining data records is sent to the central control side.
[0039] In some embodiments, a first data packet is received, which is a package of data collected from equipment on the plant side.
[0040] The plant-side equipment is responsible for collecting raw data and packaging it to send to the plant-side server 110, which then forwards the data packets to the central control server 120.
[0041] The method disclosed herein can be executed by the plant-side server 110.
[0042] In some embodiments, the first data packet is parsed.
[0043] The first data packet can be parsed to determine whether there are any conflicts in the data records contained in the first data packet.
[0044] After parsing the first data packet, all data records contained in the first data packet can be obtained, and then these data records can be judged to identify multiple data records that may cause timing conflicts.
[0045] In some embodiments, the number of data records with the same timestamp issued from the same information object address included in the first data packet can be determined.
[0046] The address of the information object corresponds to the plant-side equipment that collects the raw data, and therefore can be used to identify each monitoring point (information object) to distinguish the source of the collected data.
[0047] Normally, data records from the same information object address have different acquisition timestamps. However, in some application scenarios, the acquisition device may generate two data records with the same timestamp for the same event, such as remote signaling signals and SOEs triggered by switch changes. When the number of data records with the same timestamp from the same information object address is greater than or equal to two, conflicts may occur when these data records are stored in the time-series database on the centralized control side.
[0048] Therefore, it is possible to determine whether a data packet contains multiple data records that may cause conflicts, based on the same information object address and timestamp.
[0049] In some embodiments, if multiple data records with the same timestamp sent from the same information object address are parsed from the first data packet, the first data record among the multiple data records is determined.
[0050] If multiple data records with the same timestamp sent from the same information object address are parsed in the first data packet, these multiple data records may cause conflicts.
[0051] Therefore, the first data packet should not send all of these data records to the central control side at the same time; the data packet should only contain one of these data records.
[0052] For example, the first data record can be determined from these multiple data records. The first data record can be any one of these multiple data records.
[0053] Since multiple data records can be stored in a data packet in a specific order, the first data record can also refer to the data record that was placed into the data packet first in the order it was stored.
[0054] In some embodiments, the remaining data records, excluding the first data record, are extracted from the first data packet.
[0055] After the first data record is determined, the remaining data records in the data packet that have the same information object address and the same timestamp as the first data record can be extracted from the first data packet.
[0056] After retrieving the remaining data records, there are no other data records in the first data packet that could cause a timing conflict with the first data record.
[0057] It should be noted that the first data packet may contain multiple sets of data with the same information object address and the same timestamp. Each set of data will only retain the first data record corresponding to each set of data in the first data packet.
[0058] For example, if information object address A has two data records with the same timestamp, and information object address B has three data records with the same timestamp, then the first data packet will only contain the first data record of information object address A and the first data record of information object address B. All other data records (including one data record of information object address A and two data records of information object address B) will be extracted from the first data packet.
[0059] The remaining data records that are extracted can be processed before being sent to the central control side to avoid timing conflicts. Alternatively, after being sent to the central control side, the central control side can process different data packets to resolve timing conflicts. This disclosure does not impose any limitations on this.
[0060] For example, the remaining data records can be marked, and the central control side can sort the data records based on the marks on the data records to avoid timing conflicts; or, for example, the remaining data records can be delayed after the first data packet is sent, and when the central control side receives multiple data packets containing multiple data records with the same information object address and collection timestamp, it can process and store these data records according to the order in which the data packets are received.
[0061] In some embodiments, the first data packet after retrieving the remaining data records is sent to the central control side at least once.
[0062] After retrieving the remaining data records, the first data packet eliminates the timing conflict problem of multiple data records, and thus the processed first data packet can be sent to the central control side.
[0063] In addition to the first data packet, the remaining data records can be processed and sent to the central control side.
[0064] Alternatively, explanatory information about the temporal order of multiple data records can be generated and sent to the central control side along with the first data packet.
[0065] The technical solution disclosed herein can detect whether a first data packet sent by the plant-side equipment contains data records that may cause timing conflicts. If it is determined that multiple data records in the first data packet may cause timing conflicts, the first data record is retained, the remaining data records are extracted, and the processed first data packet is then sent to the centralized control side. Since the data records that may cause timing conflicts in the first data packet have been extracted, when the centralized control side receives the first data packet and parses the data records within it, it can only parse one data record for each information object address at each timestamp, thus preventing timing conflicts and ensuring the stability of the centralized control system.
[0066] In some embodiments, the method further includes: generating a second data packet corresponding to each of the remaining data records retrieved, wherein the timestamp of the generated second data packet is greater than the timestamp of the first data packet; and sending the second data packet to the central control side.
[0067] After retrieving the remaining data records from the first data packet, there is indeed no timing conflict issue with the first data packet. However, deleting the retrieved data records would result in data loss. If the central control side wants to store these remaining data records into the time-series database, it needs to overcome the problem of identical timestamps affecting the timing logic chain.
[0068] Therefore, as an example, the remaining data records can be used to generate a second data packet corresponding to each data record. For instance, if the first data packet contains three data records with timing conflicts, and one data record is retained, the remaining two data records are extracted from the first data packet. Since these two data records have the same information object address and timestamp, if they are packaged into another data packet, they will also cause timing conflicts and interference. Therefore, based on these two data records, two second data packets can be generated, each containing only one of the remaining data records, thus avoiding the newly generated data packets containing multiple data records that may cause timing conflicts.
[0069] However, even though multiple data records are placed in different data packets, because the timestamps of these data records and data packets are still the same, when the central control side receives these multiple data packets, timing conflicts will still occur between these multiple data records based on the same information object address and the same timestamp.
[0070] In some embodiments, a timing flag can be added to the second data packet.
[0071] By adding timing flags, the timing order between at least one second data packet and the first data packet is clearly defined, thereby avoiding timing issues.
[0072] However, adding timing flags requires the introduction of new variables and corresponding modifications to the centralized control side, making implementation complex and less robust.
[0073] Since the information object address can indicate the equipment on the plant side, it is difficult to increase the information object address.
[0074] In some embodiments, the timestamp of the generated second data packet can be made greater than the timestamp of the first data packet.
[0075] In this embodiment, the timestamp of the generated second data packet can be adjusted so that the timestamp of the second data packet is greater than the timestamp of the first data packet, thereby ensuring that the second data packet and the first data packet will not have a timing conflict.
[0076] Furthermore, if multiple second data packets exist, their timestamps will be different. For example, the timestamps of the multiple second data packets can be incremented to resolve timing conflicts between them.
[0077] In some embodiments, the method further includes: obtaining a relay timestamp when the plant-side equipment starts up; and determining a timestamp of a data packet sent to the central control side based on the relay timestamp.
[0078] In addition to using the data collection timestamp recorded in the data packet, the plant-side server 110 can also assign a unified timestamp to each data packet based on this method, thereby completely solving the timing problem of data packets (and / or data records).
[0079] For example, when the equipment on the site side starts up, the current system time can be obtained as a relay timestamp, and the timestamp of the data packet sent to the central control side can be determined based on the relay timestamp.
[0080] Since data records that may cause timing conflicts are sent from the same information object address at the same collection timestamp, and these data records will inevitably be packaged in the same data packet (i.e., the first data packet), the timestamps of the first and second data packets can be determined based on the relay timestamp, thereby distinguishing the collection timestamps of the first and second data packets.
[0081] For example, the acquisition timestamp in the first data packet can be retained, while the timestamp of the second data packet can be the sum of the relay timestamp and the acquisition timestamp, or the timestamp of the second data packet can be determined based on the difference between the relay timestamp and the acquisition timestamp.
[0082] In some embodiments, determining the timestamp of the data packet sent to the central control side based on the relay timestamp includes: comparing the data acquisition timestamp of the data packet with the relay timestamp, and assigning a unique timestamp to each data packet based on the comparison result; wherein, if the acquisition timestamp is greater than the relay timestamp, the unique timestamp is equal to the acquisition timestamp, and the relay timestamp is modified to the acquisition timestamp; if the acquisition timestamp is not greater than the relay timestamp, the relay timestamp is incremented by a first duration, and the unique timestamp is equal to the incremented relay timestamp.
[0083] The data records in the data packet contain the collection timestamp of that data. After determining multiple data records in the first data packet, the collection timestamp and the relay timestamp can be compared, and then a unique timestamp can be assigned to each data packet based on the comparison result.
[0084] If the data collection timestamp is greater than the transit timestamp, the unique timestamp is equal to the collection timestamp, and the transit timestamp is modified to the collection timestamp.
[0085] Since the transfer timestamp is obtained when the equipment on the station side starts up, and data collection can only begin after the equipment on the plant side starts up, the collection timestamp should be greater than or equal to the transfer timestamp.
[0086] When there is no risk of time-series conflict in the data records, each data record has a different collection timestamp (and / or a different information object address). Since the data collection by the equipment on the site side has a certain time sequence, the collection timestamp will be greater than the relay timestamp each time. Therefore, the unique timestamp of the data record is the collection timestamp of the data record, and the relay timestamp is updated.
[0087] When subsequent data records without time-series conflict risk are received, the collection timestamp of these data records without time-series conflict risk will necessarily be greater than the previously determined transit timestamp. Therefore, the transit timestamp can be updated with the collection timestamp of the received data records.
[0088] If the collection timestamp is not greater than the transit timestamp, the transit timestamp is increased by a first duration, and the unique timestamp is equal to the increased transit timestamp.
[0089] When multiple data records are collected at the same time, resulting in multiple data records having the same collection timestamp, this situation is the scenario mentioned above in this disclosure that leads to timing conflicts.
[0090] In this scenario, the collection timestamp of the first data record is greater than the transit timestamp. Based on the conclusion of the previous comparison, the unique timestamp is the collection timestamp, and the transit timestamp is updated to the collection timestamp. For the second data record, which has the same timestamp as the first data record, since the transit timestamp has been updated to the timestamp of the first data record, the collection timestamp of the second data record is equal to the transit timestamp. Therefore, the transit timestamp can be increased by a first duration, and the increased transit timestamp can be assigned to the second data record as its unique timestamp.
[0091] If there are more than two data records with time-series conflicts, the timestamps of subsequent data records (the third, fourth, and subsequent records) will all be smaller than the incremented transit timestamp. Therefore, in the process of processing subsequent data records one by one, the transit timestamp is continuously incremented by the first duration, and the incremented value is assigned to each subsequent data record as its corresponding unique timestamp, so that subsequent data records can also have different and incrementing unique timestamps.
[0092] Based on this example, on the one hand, the timestamps of multiple data records that may cause timing conflicts can be distinguished, thus avoiding timing conflicts; on the other hand, since the transit timestamp is incremental, the unique timestamps between multiple data records with the same original timestamp are also in an incremental relationship, which helps to improve the timing logic chain and facilitates processing and storage on the centralized control side.
[0093] In some embodiments, the first duration may be 1 millisecond.
[0094] The following example will provide further explanation of the unique timestamp-related instances mentioned above.
[0095] After the data acquisition device on the station side is started, the operating environment can be initialized, and the relay timestamp can be initialized to the current system time t0, with the timestamp precision at the millisecond level.
[0096] When there is no timing conflict in the data records in the data packet, the collection timestamp of the data records collected at times t1, t2, etc. is greater than the relay timestamp. The unique timestamp of these data records is the collection timestamp, and the relay timestamp is updated successively according to the collection timestamp.
[0097] When a first data packet is received, which contains multiple data records collected at time tx from the same information object address, the first data record among these multiple data records is retained in the first data packet based on the method disclosed herein, and the remaining data records are extracted from the first data packet and then each of these data records is assigned a unique timestamp.
[0098] The specific process is as follows: For the first data record, if the acquisition timestamp tx is greater than the relay timestamp, then the unique timestamp is tx, and the relay timestamp is also updated to tx. For subsequent data records with the same acquisition timestamp tx, such as the second data record, since the acquisition timestamp tx is not greater than the newly updated relay timestamp tx, the relay timestamp can be incremented to tx+1, and the unique timestamp of this data record is also tx+1. For the third data record, if the acquisition timestamp tx is not greater than the relay timestamp tx+1, then the relay timestamp is updated to tx+2, and the unique timestamp of this data record is tx+2. It can be seen that the original timestamps of these three data records with time-series conflicts were tx, tx, and tx, respectively. However, based on the method of this embodiment, the updated unique timestamps are tx, tx+1, and tx+2, with distinct timestamps, thus completely resolving the time-series conflict problem.
[0099] In some embodiments, a unique timestamp can be assigned to each data packet to be sent to the central control side.
[0100] A unique timestamp can be assigned to each data packet to be sent to the central control side.
[0101] In some technical scenarios, two different data packets may have the same timestamp, and these two data packets may also contain data records with the same information object address. During the parsing process, these two data packets may not detect a timing conflict. However, from the perspective of the central control side, all received data packets are parsed. Therefore, even if the data packets are different, if, after unpacking, the same information object address contains multiple data records with the same timestamp, these multiple data records will still cause a timing conflict.
[0102] Therefore, in addition to parsing the first data packet containing multiple data records as described in this disclosure to generate new data packets for the remaining data records, a unique timestamp can also be assigned to each data packet. This ensures that after the centralized control side receives all data packets, no two timestamps are the same for the same information object address.
[0103] In other embodiments, a unique timestamp may be assigned only to multiple data records in the first data packet that have the same information object address and the same timestamp.
[0104] Even if different information object addresses have the same timestamp, the data records of these two different information object addresses will not conflict when processed by the central control side.
[0105] Therefore, if no data record containing multiple data records with timing conflicts is found in the data packet, the timestamp of the data packet can be sent directly to the central control side without adjusting it.
[0106] In cases involving multiple data records that could cause data conflicts, a unique timestamp is assigned to the first identified data record and the remaining data records via a transit timestamp, so that these data records with the same information object address and the same collection timestamp are assigned different time sequences.
[0107] Compared to assigning a unique timestamp to all data records in all data packets, this embodiment only processes the first data record and the remaining data records in the first data packet, which can speed up forwarding, improve efficiency, and reduce power consumption and latency.
[0108] In some embodiments, the data packet includes an IEC104 protocol data packet; parsing the first data packet includes: parsing the first data packet into an application service data unit object; obtaining data record information in the application service data unit object; and performing aggregation statistics on the data record information to determine multiple data records sent from the same information object address contained in the first data packet.
[0109] In new energy centralized control systems, the IEC104 protocol is a commonly used communication protocol, and the technical problems proposed in this disclosure are more likely to occur in the data packets of the IEC104 protocol, thus making it more necessary to implement the data processing method proposed in this disclosure.
[0110] Different protocols can employ different parsing methods. Taking the IEC104 protocol as an example, after obtaining the raw data, it can be parsed into ASDU (Application Service Data Unit) objects according to the IEC104 protocol rules using the J60870 library (an open-source Java library that provides functions for constructing, parsing, and transmitting IEC104 protocol messages). The `getInformationObjects` method of the ASDU object is used to retrieve all InformationObjects (information objects, i.e., data records) within the ASDU object, constructing information object records and storing them in a two-dimensional data set. Aggregate and statistically analyze all information object records to determine multiple data records sent from the same information object address contained in the first data packet.
[0111] If records with the same address exist, they can be deleted by calling the `removeInformationObjects` method within the original ASDU object. Simultaneously, a new ASDU object is created using the ASDU class constructor. The constructor parameters include the ASDU type, InformationObject object, transmission reason, and common address, where the ASDU type and transmission reason are the same as the original ASDU object. After parsing and splitting, each ASDU object contains no records with duplicate addresses.
[0112] In some embodiments, the data packets sent to the centralized control side are used to enable the centralized control side to store the contents of the data packets into a time-series database according to the timestamp corresponding to the data packets upon receiving the data packets.
[0113] The central control side can parse all received data packets, obtain the data records contained therein, and then store the data records in the data packet into the time series database based on the timestamp of the data packet.
[0114] Based on the above embodiments of this disclosure, the splitting of multiple data records in the same data packet that may have time-series conflicts, and the assignment of a unique timestamp to each data packet, can ensure that no time-series conflicts occur between data records when the centralized control side stores them in the time-series database.
[0115] In some embodiments, the types of the plurality of data records include at least one of the following: single-point remote signaling information; two-point remote signaling information; single-point time-stamped remote signaling information (SOE); and two-point time-stamped remote signaling information (SOE).
[0116] Data records of the same type can be considered as the same type of data. However, if data records of different types have the same information object address and the same collection timestamp, there may be a time sequence conflict between the data records of different types, which may lead to them being overwritten or lost.
[0117] Therefore, parsing multiple data records with the same timestamp sent from the same information object address in the first data packet can also include: parsing multiple data records with the same timestamp sent from the same information object address but different data record types in the first data packet.
[0118] The type of data record can be distinguished by a type identifier. For example, in the IEC 104 protocol, a type identifier of 1 indicates single-point telemetry information; a type identifier of 3 indicates two-point telemetry information; a type identifier of 30 indicates single-point time-stamped telemetry information (SOE); and a type identifier of 31 indicates two-point time-stamped telemetry information (SOE).
[0119] In some embodiments, the method further includes: determining forwarding configuration information and sending data packets to the central control side according to the forwarding configuration information.
[0120] After the data acquisition device is started, the forwarding configuration information can be read. All data packets can be sent to the central control side according to the forwarding configuration information.
[0121] Corresponding to the embodiments of the data processing method of this disclosure, this disclosure also provides embodiments of corresponding system devices.
[0122] Please see Figure 3 , Figure 3 This is a schematic diagram of the architecture of a centralized control system in one embodiment of this disclosure.
[0123] like Figure 3 As shown, the new energy plant side includes monitoring equipment (such as Far East Equipment, DCS (Distributed Control System), and integrated automation system), including the plant side server 110 (i.e. Figure 3 (Data acquisition device in the middle).
[0124] The method disclosed herein, deployed on the plant-side server 110, can be divided into four functional modules: a data acquisition module, a parsing and splitting module, a timestamp module, and a data forwarding module.
[0125] The data acquisition module receives raw data packets; the parsing and splitting module identifies potentially conflicting data in the data packets and splits them into different data packets; the timestamp module maintains global variables and auto-incrementing logic to serialize concurrent data packets that may cause conflicts on the timeline and assign a unique timestamp to each data packet; and the data forwarding module forwards the processed data packets.
[0126] The new energy centralized control side includes data processing services and time-series databases.
[0127] like Figure 3 As shown, the data processing service can be divided into a data acquisition module, a data parsing module, and a data storage module. The specific functions of these modules will not be elaborated upon here.
[0128] Corresponding to the embodiments of the data processing method of this disclosure, this disclosure also provides embodiments of a corresponding data processing apparatus.
[0129] Please see Figure 4 , Figure 4 This is a block diagram of a data processing apparatus according to one embodiment of the present disclosure. Figure 4 As shown, the data processing device includes:
[0130] The receiving unit 410 is configured to receive a first data packet formed by packaging data collected from the plant-side equipment;
[0131] The parsing unit 420 is configured to parse the first data packet and, if multiple data records with the same timestamp sent from the same information object address are parsed from the first data packet, determine the first data record among the multiple data records;
[0132] The splitting unit 430 is configured to extract the remaining data records from the first data packet, excluding the first data record, from the plurality of data records.
[0133] The sending unit 440 is configured to send at least the first data packet after retrieving the remaining data records to the central control side.
[0134] In some embodiments, the splitting unit 430 is further configured to: generate a second data packet corresponding to each of the remaining data records extracted, wherein the timestamp of the generated second data packet is greater than the timestamp of the first data packet; and send the second data packet to the central control side.
[0135] In some embodiments, the apparatus further includes: a timestamp unit configured to acquire a relay timestamp when the plant-side equipment starts up; and to determine the timestamp of a data packet to be sent to the central control side based on the relay timestamp.
[0136] In some embodiments, determining the timestamp of the data packet sent to the central control side based on the relay timestamp includes: comparing the data acquisition timestamp of the data packet with the relay timestamp, and assigning a unique timestamp to each data packet based on the comparison result; wherein, if the acquisition timestamp is greater than the relay timestamp, the unique timestamp is equal to the acquisition timestamp, and the relay timestamp is modified to the acquisition timestamp; if the acquisition timestamp is not greater than the relay timestamp, the relay timestamp is incremented by a first duration, and the unique timestamp is equal to the incremented relay timestamp.
[0137] In some embodiments, the data packet includes an IEC104 protocol data packet; parsing the first data packet includes: parsing the first data packet into an application service data unit object; obtaining data record information in the application service data unit object; and performing aggregation statistics on the data record information to determine multiple data records sent from the same information object address contained in the first data packet.
[0138] In some embodiments, the data packets sent to the centralized control side are used to enable the centralized control side to store the contents of the data packets into a time-series database according to the timestamp corresponding to the data packets upon receiving the data packets.
[0139] In some embodiments, the types of the plurality of data records include at least one of the following: single-point remote signaling information; two-point remote signaling information; single-point time-stamped remote signaling information; and two-point time-stamped remote signaling information.
[0140] In some embodiments, the sending unit 440 is further configured to: determine forwarding configuration information and send data packets to the central control side according to the forwarding configuration information.
[0141] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.
[0142] Embodiments of this disclosure also provide an electronic device, including: a processor and a memory; the memory for storing a computer program; and the processor for executing a data processing method as described in any of the above embodiments by invoking the computer program.
[0143] Embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the data processing method as described in any of the above embodiments.
[0144] Embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.
[0145] Figure 5 This is a schematic block diagram illustrating a data processing apparatus 500 according to embodiments of the present disclosure. For example, apparatus 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0146] Reference Figure 5 The device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.
[0147] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the data processing method described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.
[0148] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of such data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0149] Power supply component 506 provides power to various components of device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.
[0150] Multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0151] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.
[0152] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.
[0153] Sensor assembly 514 includes one or more sensors for providing status assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in position of device 500 or a component of device 500, the presence or absence of user contact with device 500, orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.
[0154] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G LTE, 5G NR, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0155] In an exemplary embodiment, the apparatus 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data processing method described above.
[0156] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of the device 500 to complete the data processing method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0157] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0158] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
[0159] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0160] The methods and apparatus provided in the embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.
Claims
1. A data processing method, characterized in that, The method includes: Receive the first data packet formed by packaging data collected from equipment on the plant side; If the first data packet is parsed and multiple data records with the same timestamp sent from the same information object address are parsed from the first data packet, the first data record among the multiple data records is determined; Extract the remaining data records from the first data packet, excluding the first data record. At least send the first data packet after retrieving the remaining data records to the centralized control side; A second data packet is generated for each of the remaining data records extracted, wherein the timestamp of the generated second data packet is greater than the timestamp of the first data packet; The second data packet is sent to the central control side, and the second data packet includes at most one of the remaining data records.
2. The method according to claim 1, characterized in that, The method further includes: Obtain the transfer timestamp when the plant-side equipment starts up; The timestamp of the data packet sent to the central control side is determined based on the relay timestamp.
3. The method according to claim 2, characterized in that, The step of determining the timestamp of the data packet sent to the central control side based on the relay timestamp includes: The data acquisition timestamp of the data packet is compared with the relay timestamp, and a unique timestamp is assigned to each data packet based on the comparison result; wherein... If the collection timestamp is greater than the relay timestamp, the unique timestamp is equal to the collection timestamp, and the relay timestamp is modified to the collection timestamp; If the collection timestamp is not greater than the transit timestamp, the transit timestamp is increased by a first duration, and the unique timestamp is equal to the increased transit timestamp.
4. The method according to claim 1, characterized in that, The data packet includes IEC104 protocol data packets; the parsing of the first data packet includes: The first data packet is parsed into an application service data unit object; Obtain the data record information from the application service data unit object; The data record information is aggregated and statistically analyzed to determine multiple data records sent from the same information object address contained in the first data packet.
5. The method according to claim 1, characterized in that, The data packets sent to the centralized control side are used to enable the centralized control side to store the contents of the data packets into the time-series database according to the timestamp corresponding to the data packets upon receiving the data packets.
6. The method according to claim 1, characterized in that, The types of the multiple data records include at least one of the following: Single-point remote signaling information; two-point remote signaling information; single-point remote signaling information with time stamp; two-point remote signaling information with time stamp.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: Determine the forwarding configuration information and send data packets to the central control side according to the forwarding configuration information.
8. A data processing apparatus, characterized in that, The device includes: The receiving unit is configured to receive a first data packet formed by packaging data collected from equipment on the plant side; The parsing unit is configured to parse the first data packet and, if multiple data records with the same timestamp sent from the same information object address are parsed from the first data packet, determine the first data record among the multiple data records; The splitting unit is configured to extract the remaining data records from the first data packet, excluding the first data record, from the plurality of data records; The sending unit is configured to send at least the first data packet after retrieving the remaining data records to the central control side; The splitting unit is further configured to: generate a second data packet corresponding to each of the remaining data records extracted, wherein the timestamp of the generated second data packet is greater than the timestamp of the first data packet; and send the second data packet to the central control side, wherein the second data packet includes at most one of the remaining data records.
9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the data processing method described in any one of claims 1 to 7.