Batch data multi-level filtering collection and processing method based on configuration matrix

Abstract

This invention discloses a multi-level filtering acquisition and processing method for batch data based on a configuration matrix. Addressing the problems of request redundancy, full parsing, and rigid scheduling in existing technologies, this invention constructs a three-dimensional strategy configuration matrix and a two-dimensional periodic copying matrix to collaboratively calculate the sum of data categories and generate an optimal copying list, thus avoiding duplicate requests for the same physical channel at the same time point. During the request phase, duplicate requests are filtered by comparing the copied time with the previous copied time. At the receiving end, two levels of dynamic filtering are performed: a coarse filtering based on the divisibility relationship between the copying period and the message data time, and the sum of data categories; and a fine filtering based on the dynamic selection of measurement points to be parsed using the three-dimensional values ​​of the configuration matrix. This invention systematically solves the problems of low communication efficiency and high processing overhead, achieving efficient copying, accurate parsing, and storage of batch data.

Description

Technical Field

[0001] This invention belongs to the field of power system data acquisition technology, and relates to a batch data multi-level filtering acquisition and processing method based on a configuration matrix. Background Technology

[0002] In industrial data acquisition systems based on standards and protocols such as IEC 1107, DLMS / COSEM, and ANSI, the acquisition end needs to read various types of data from metering devices (such as smart meters and RTUs), including real-time data and historical frozen data (such as daily frozen, monthly frozen, and curve data). This data is typically stored in structured formats, such as logical channels in the IEC 1107 / DLMS protocol, where one or more logical channels are used to read all data (including instantaneous quantities, energy, and demand) at a specific point in time or time period; and data tables in the ANSI protocol (such as 0x17 energy meters and 0x1c telemetry meters), where all energy or telemetry data can be obtained by reading a single table.

[0003] However, traditional copying methods face the following common challenges in practical applications:

[0004] (1) Request redundancy and channel reuse issues. To obtain data for a complete logical unit (e.g., "positive active power curve"), it is usually necessary to request multiple sub-items such as "total, rate, and phase". These sub-items may be scattered across multiple data channels. If each sub-item initiates a request independently, a large amount of communication interaction will occur. More seriously, when it is necessary to obtain data for multiple logical units (e.g., positive and negative active power) at the same time point (e.g., 16:40) at the same time, these data may exist in the same set of channels. Traditional methods traverse all channels independently for each logical unit, resulting in repeated reading of the same physical data blocks, leading to extremely low communication efficiency.

[0005] (2) Waste of resources in "full reading and blind processing". On the requesting side, the scope of reading (such as whether to read the total score or whether to read the phase) is usually hard-coded or configured in a scattered manner, lacking a unified and flexible control mechanism. On the receiving side, all data in the returned message is usually fully parsed, regardless of whether the actual application needs it, resulting in the ineffective consumption of CPU and memory, and the inability to dynamically adapt the processing granularity according to the scenario.

[0006] (3) Rigid parsing and scheduling mechanism. The request cycle of different types of data is different. For example, electrical energy may need to be acquired every 5 minutes, while demand may need to be acquired every 15 minutes. Traditional methods may try to process all types of data in each acquisition cycle, or rely on complex timers for management, which fails to synchronize precisely with the data generation cycle, thus generating unnecessary scheduling overhead.

[0007] In summary, the core contradiction of existing technologies lies in the conflict between the diversity of data collection needs and the uniformity of data collection methods. The lack of a mechanism to dynamically map and optimize data collection needs (configuration) and physical channels (resources) leads to bottlenecks in communication efficiency, resource consumption, and scheduling flexibility. Summary of the Invention

[0008] This invention addresses the shortcomings of existing technologies by providing a batch data multi-level filtering acquisition and processing method based on a configuration matrix.

[0009] This invention includes:

[0010] S1, Initialization steps: Construct a three-dimensional strategy configuration matrix and a two-dimensional periodic reading matrix; the strategy configuration matrix is ​​used to configure the total amount, rate and phase of reading for each data subclass, and the periodic reading matrix is ​​used to configure the reading cycle for each data major class; calculate the sum of each data major class according to the strategy configuration matrix, and generate the optimal reading list by combining the periodic reading matrix and periodic matching and channel reuse verification;

[0011] S2, Data Request Step: Initiate a data request to the terminal device based on the data items in the optimal data reading list. During the request process, record the data reading time of various types of frozen data. By comparing the last data reading time of the current data item with the data reading time of the corresponding data type, filter out duplicate requests to data channels that have already been read at the same time point.

[0012] S3, Data parsing steps: Receive the response message from the terminal device and perform two-level filtering:

[0013] The first-level coarse filtering determines whether the message data time meets the period divisibility condition based on the reading cycle, and determines whether to enter the parsing process based on the sum of the data categories.

[0014] Secondary fine filtering involves dynamically selecting the measurement points to be parsed from the response messages for data extraction and storage based on the copying strategy of the corresponding data subclass in the strategy configuration matrix.

[0015] Furthermore, the step of calculating the sum of each data category based on the strategy configuration matrix includes: traversing the first two dimensions of the strategy configuration matrix, and for each coordinate of the first two dimensions, traversing the first 6 elements along the third dimension and accumulating them into the last element of the third dimension to obtain the sum of each data category;

[0016] Furthermore, the step of generating the optimal copy list through periodic matching and channel multiplexing verification includes:

[0017] Find the minimum value in the reading cycle of each data category;

[0018] Determine if the sum of the data categories corresponding to the minimum value is 0. If it is 0, continue searching for the second smallest value. If it is not 0, check if there is a copying item with the same copying data channel in the list of existing copying schemes. If it exists, determine if the minimum value is the same as or a multiple of the existing copying interval. If so, do not add it to the copying list; otherwise, add it to the copying list.

[0019] Furthermore, the periodic request in the data request step includes: initializing the next copying index to 0; retrieving each copying item and determining whether the difference between the current time and the previous copying time is greater than or equal to the copying interval; if the condition is met, recording the current copying index and the next copying index, and updating the request time; if the condition is not met, continuing the search.

[0020] Furthermore, the filtering of duplicate requests for data channels that have already been copied at the same time point includes: if the current copied item is the first channel of curve freezing, daily freezing, or monthly freezing, then compare the copied time corresponding to this type of data with the previous copied time; if they are the same, then it is determined that the frozen data of this type of channel at this time point has been copied, and the previous copied time of this type of channel and its subsequent channels of the same type at the same time is increased by a copying interval, and the search continues; if they are different, then the copied time of this type of data is updated to the previous copied time, the previous copied time is increased by a copying interval, and the search is returned as successful.

[0021] Further, after a successful retrieval, the process of assembling a request message begins: the corresponding copy item is found based on the current copy index, the corresponding OBI is obtained based on the copy channel in the copy item, and the request message is assembled based on the data type, channel OBI, and request time of the copy item.

[0022] Furthermore, the data request step also includes message request: receiving a message command, initializing the manually detected reading item, and sending a request command according to the reading item. This process does not determine the reading period and does not update the reading time of the curve freeze, daily freeze, or monthly freeze.

[0023] Furthermore, before receiving the response message from the terminal device and performing two-level filtering, a message check is also included: checking whether the response message is complete, whether it exceeds the minimum message length, and whether the buffer overflows. If the conditions are not met, the current receiving process is terminated.

[0024] Furthermore, the first-level coarse filtering includes: obtaining the corresponding copying period based on the current copying item, and determining whether the message data time msgTime can be divided by the copying period; if it can be divided, then further determining whether the sum of the corresponding data categories is not 0; if it is not 0, then proceeding to the second-level fine filtering.

[0025] Furthermore, the secondary fine filtering includes: calculating the total amount, rate, and phase reading requirements in each data subclass based on the three-dimensional data values ​​of the strategy configuration matrix, and finding the corresponding obis for the measurement point to be read; determining whether the obis exists based on the structure, and if it exists, extracting the corresponding measurement point value; if it still does not exist after filtering all similar value channels, it indicates that the measurement point is not configured.

[0026] The beneficial effects of this invention are:

[0027] 1) Initialization phase: Eliminate redundancy and optimize resource allocation

[0028] The strategy configuration matrix and the periodic copying matrix work together: By using a three-dimensional strategy configuration matrix and a two-dimensional periodic copying matrix, the optimal copying list is dynamically calculated, avoiding the problem of "multiple logical units repeatedly requesting the same physical channel at the same time" in traditional solutions.

[0029] Data category summation calculation: Traverse the first two dimensions of the configuration matrix, and accumulate the first 6 elements to the 7th element along the third dimension, eliminating duplicate or invalid copy items, which greatly reduces the number of communication messages.

[0030] 2) Sending request message stage: precise data mapping

[0031] By verifying `curvTime`, `dayTime`, `monthTime`, and `lastReadTime`, it ensures that only frozen data that has not been copied is requested. This supplements the vulnerability in the initialization process and completely avoids duplicate requests.

[0032] 3) Two-stage filtering: Focusing on effective data and reducing processing overhead.

[0033] First-level coarse filtering: First, determine whether msgTime is divisible by the reading period to exclude data from non-target periods and reduce invalid parsing; Second, further filter unnecessary parsing based on the sum of the major data categories to exclude unnecessary parsing.

[0034] Secondary fine filtering: Based on the three-dimensional values ​​of the configuration matrix (whether to read the total amount / rate / phase), the measurement points to be analyzed are dynamically selected to avoid the CPU / memory consumption of the traditional "full analysis of all returned data". That is, only the measurement points enabled in the configuration are extracted, rather than all measurement points in the channel are analyzed. Attached Figure Description

[0035] Figure 1 The curve freezing data reading item diagram of the present invention;

[0036] Figure 2 : A diagram of daily frozen data reading items in this invention;

[0037] Figure 3 The periodic copying matrix diagram of this invention;

[0038] Figure 4 The strategy configuration matrix diagram of this invention;

[0039] Figure 5 : A diagram of the channel configuration structure of this invention;

[0040] Figure 6 The channel numerical structure diagram of this invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0042] This invention provides a batch data multi-level filtering acquisition and processing method based on a configuration matrix, applicable to industrial data acquisition systems for multi-protocol terminal devices such as IEC 1107, DLMS / COSEM, and ANSI. This method systematically solves the problems of communication redundancy, resource waste, and scheduling rigidity in existing technologies through structured configuration, intelligent merging, and dynamic filtering.

[0043] Step S1: Initialization Step

[0044] When the system starts up or is reloaded, initialization steps are performed, including building the policy configuration matrix and periodic reading matrix, generating the optimal reading list, and initializing the channel configuration.

[0045] S1.1: Constructing the strategy configuration matrix and periodic copying matrix

[0046] A three-dimensional strategy configuration matrix is ​​constructed to configure the data reading strategy for each data subclass under different application scenarios, including whether to read the total amount, rate, and phase data. This matrix defines the three-dimensional position and corresponding value of the data subclass. For example, for curve-frozen data, it can be configured whether to read the total amount of positive active energy, whether to read each rate, and whether to read phase data.

[0047] A two-dimensional periodic meter reading matrix is ​​constructed to configure the meter reading cycle for various data categories (such as instantaneous quantity, electrical energy, demand, and four quadrants) under different application scenarios. The freeze time for each data category of curve data is directly set; the freeze time for each data category of daily and monthly frozen data is calculated based on the number of settlements per day / month for each data category. For example, if daily frozen electrical energy is settled three times a day, the meter reading cycle is 8 hours; if four quadrants are settled four times a day, the meter reading cycle is 6 hours.

[0048] S1.2: Generate the optimal copy list

[0049] When the system starts up or is reloaded, initialize all values ​​of the policy configuration matrix and the periodic reading matrix to 0. Integrate all data items that need to be collected (such as phase voltages of A / B / C, various rates, and total amount) into a unified scheme matrix, calculate the sum of major categories, and initialize the reading items according to the sum of major categories and the reading cycle.

[0050] The curve freeze data copying item in this application (see...) Figure 1 ), Daily frozen data reading items (see Figure 2 A single line of description, such as (positive active energy), can be called a logical unit. Each logical unit has three parts: total amount, rate, and phase. Each part has several measurement points. For example, positive active energy phase includes positive active energy A, positive active energy B, and positive active energy C measurement points. Rate includes positive active energy at rate 1, ..., positive active energy at rate 6 measurement points, etc. Each measurement point is a reading item.

[0051] Specifically, it includes the following sub-steps:

[0052] Step 1: Obtain the reading cycle. Obtain or calculate the reading cycle for each data category (instantaneous quantity, electrical energy, demand, and four quadrants) under the three application scenarios, and generate a cycle reading matrix. See [link / reference]. Figure 3 .

[0053] Step 2: Obtain the data collection and dissemination plan. Obtain the data collection and dissemination plan for each data subclass, including whether to collect the total amount, the rate, and the phase. Calculate the value for each data subclass and initialize the configuration matrix based on its three-dimensional position and value. For example, a 3*4*7 three-dimensional configuration matrix. The meaning of the three-dimensional matrix is ​​as follows: Figure 4 As shown.

[0054] Step 3: Calculate the sum of each category. Traverse the first two dimensions of the matrix. For each coordinate in the first two dimensions, traverse the first 6 elements along the third dimension and add them to the last element of the third dimension to obtain the sum of each data category.

[0055] Step 4: Calculate and add the reading items. Calculate the reading items based on the total sum of the major categories and the reading cycle. Find the minimum value (ReadCycle) in the reading cycle and check if the total sum of the major categories corresponding to the minimum value is 0. If it is 0, continue searching for the second smallest value in the reading cycle; otherwise, search the existing reading scheme list. If a scheme has the same reading data channel as the one to be added, compare it with the reading interval (ReadInterval) of the existing reading scheme. If ReadCycle is the same as ReadInterval or a multiple of ReadInterval, do not add it to the reading list; otherwise, add it to the reading list. The reading cycle is ReadCycle.

[0056] For example, there are 8 data channels for curve freezing and reading, represented by ReadChannel1 to ReadChannel8; the reading cycles are: 2 minutes for electrical energy, 15 minutes for four quadrants, 3 minutes for demand, and 5 minutes for instantaneous quantity. The major categories of reading curves in the configuration matrix are: electrical energy 6 (coordinates [0][0][6]), four quadrants 3 (coordinates [0][1][6]), demand 0 (coordinates [0][2][6]), and instantaneous quantity 7 (coordinates [0][3][6]). The calculation process is as follows:

[0057] First, find the electrical energy with the shortest reading cycle. The corresponding data category reading sum is 6. Search the existing reading list. If there is no reading item with the reading data channel ReadChannel1, add the reading item to the reading list with the reading data channel ReadChannel1 and the reading cycle of 2 minutes. Continue in this way to add all 8 channels to the reading list.

[0058] Secondly, find the second smallest required quantity in the copying cycle. If the corresponding data category is 0, then no copying is needed.

[0059] Next, continue searching for instantaneous data with a reading cycle of 5 minutes. The total number of reading categories is 7. Traverse the reading list. If there is a reading item with the reading data channel ReadChannel1 and a reading interval of 2 minutes (not a multiple of 5 minutes), continue traversing. If no reading item meets the requirements, add a new reading item with the reading data channel ReadChannel1 and a reading cycle of 5 minutes. Continue adding items in this manner.

[0060] Finally, the four quadrants with a copying cycle of 15 minutes are searched, corresponding to a copying category sum of 3. The copying list is then iterated through. Data items in ReadChannel1 have a copying interval of 2 minutes, which does not satisfy the multiple or identical relationship. The copying interval for data items in ReadChannel1 is then continued, and it is found to be 5 minutes, satisfying the multiple relationship; therefore, no new copying item is added. Data items in ReadChannel2 are then iterated through again, processed similarly to ReadChannel1.

[0061] Similarly, the process of generating the optimal copy list for daily and monthly freezing is similar to that for curve freezing.

[0062] It should be noted that if the data reading periods are not multiples of each other and are not the same (e.g., periods of 2 minutes and 3 minutes), and the sum of the corresponding major data categories is not zero, then both types of data items will be added to the data reading list. At the 6th minute, this will lead to a problem where multiple logical units repeatedly request the same channel at the same time. This issue will be resolved in subsequent data request steps.

[0063] S1.3: Channel Configuration Initialization

[0064] After the power data acquisition terminal is initialized, the channel configuration is initialized. The number of data reading channels configured for each of the curve freeze, daily freeze, and monthly freeze modes corresponds to the number of data reading channels available. For example... Figure 5 As shown, the identifiers of the curve freeze, daily freeze, and monthly freeze configuration channels, as well as the numerical channels, can be read from the specification parameter configuration; these are the channel OBIS. The channel OBIS are saved for later use; each channel contains the OBIS of several measurement points.

[0065] Specifically, it includes the following sub-steps:

[0066] Step 1: Request channel configuration. After the protocol is started or reloaded, the initialization flag is set to FALSE, and a request message is sent according to the obtained channel obis.

[0067] Step 2: Parse the channel configuration. After obtaining the channel response message, parse the measurement point obis in the channel and save it for later use, including the total number of measurement points in each channel and the measurement point obis.

[0068] Step 3: Set the initialization flag. After all channel configurations have been successfully obtained, set the initialization flag to TRUE.

[0069] The purpose of the initialization flag is to prevent multiple requests for channel configuration, thus avoiding unnecessary resource waste. Since this process involves interaction with the meter, it cannot be guaranteed that the configuration will be obtained in a single request. For example, if the meter is not powered on when the protocol starts, the request may fail, and the initialization flag will be set to FALSE, but subsequent requests will continue. Once the meter is powered on, the configuration is obtained, parsed, and saved, and then the flag is initialized to TRUE. No further requests will be made during subsequent communication.

[0070] Step S2: Requesting Data

[0071] Data requests are initiated to the terminal device based on the data transfer items in the optimal data transfer list. Data transfer items include the data transfer interval (ReadInterval), the last data transfer time (lastReadTime), the data type (real-time data, curve-frozen data, daily frozen data, monthly frozen data), and the data transfer channel. The OBI (Optimal Biscuit Number) of the data transfer channel can be obtained through the channel. Simultaneously, `curvTime`, `dayTime`, and `monthTime` record the time point at which the requested frozen data is transferred when reading the first data channel for curve-frozen, daily frozen, and monthly frozen data, respectively. For example, if the current time is 10:21 and the request is for frozen data at 10:20 from the meter, and it is curve-frozen, then `curvTime` will be recorded as the timestamp of 10:20 when reading the first data channel.

[0072] The data request process includes three methods: periodic request, message request, and rewrite request.

[0073] S2.1: Periodic Request

[0074] Step 1: Initialization. Before performing the cyclic reading, initialize the next reading index (nextReadIndex) to 0.

[0075] Step 2: Retrieve Data Items. Enter the retrieval process, set the second-level copying flag to FALSE, obtain the current time (currentTime), and retrieve the difference between the current time (currentTime) and the last copying time (lastReadTime) of the copied item. That is, determine if currentTime - lastReadTime is greater than or equal to ReadInterval. If the condition is not met, continue searching for the next copied item; if the condition is not met after one round of searching, set nextReadIndex to 0, exit the retrieval process, and return a retrieval failure message; otherwise, proceed to Step 3.

[0076] Step 3: Record status values. Retrieve data items that meet the conditions, record currentReadIndex as the current index item for use in assembling request messages and parsing response messages; increment nextReadIndex by 1 to start the next search from nextReadIndex to increase search efficiency; update requestTime to lastReadTime.

[0077] Step 4: Filter duplicate data entries. If the data entry is the first channel of a curve-frozen, daily-frozen, or monthly-frozen channel, compare the corresponding data entry time (curvTime, dayTime, or monthTime) with lastReadTime. If they are different, it means that the frozen data for that channel at that time point has not been copied. Update curvTime, dayTime, or monthTime to lastReadTime, and increment the last read time (lastReadTime) by one ReadInterval for the next round of data entry calculation; exit the search process and return a success message. If they are the same, it means that the channel at that time point has already been copied. Increase the lastReadTime of the current channel and subsequent channels of the same type with the same lastReadTime as curvTime, dayTime, or monthTime by one ReadInterval; continue the search. If the search fails, process other business or wait for the next cycle.

[0078] This process addresses a legacy issue from the initialization step: when the reading cycles of different types of the same channel are not multiples of each other. For example, if the curve freeze data has an energy cycle of 2 minutes and an instantaneous data cycle of 3 minutes, and the time is 2026-05-16 17:24:00 (equivalent to a timestamp of 1778923440 seconds, or 29648724 minutes), this time point is divisible by 6 minutes, meaning it's both a 2-minute and a 3-minute reading time point. Without Step 4, the channel data at that time point would be repeatedly read; filtering resolves this legacy issue.

[0079] Step 5: Assemble the request message. If the retrieval is successful, proceed to the message assembly process. Locate the corresponding read item based on currentReadIndex, obtain the corresponding obis based on the read channel in the read item, and assemble the request message based on the data type, channel obis, and requestTime in the data item.

[0080] S2.2: Message Request

[0081] The message module is the interaction module between the human interface and the power energy acquisition terminal, and it is also the interaction module between the northbound request message and the power energy acquisition terminal.

[0082] Step 1: Receive message request. The message receiving interface receives message commands, including message type, task number, response timeout, whether a response is required, message content, message content length, etc.; after receiving the message, it modifies the message processing flag to TRUE.

[0083] Step 2: Processing Message Requests. In the message module, manual meter readings require requesting curve freeze, daily freeze, and monthly freeze data from the meter. This process involves initializing the manual reading items, which include message type, total number of reading items, reading index, and the time point from which the data is frozen. Request commands are sent based on the reading items. The process is similar to periodic meter readings, except that the period is not considered, and the values ​​of `curvTime`, `dayTime`, and `monthTime` are not updated.

[0084] S2.3: Request for Supplementary Copying

[0085] Similar to periodic requests, rewrite requests will eventually send a data channel request message; the specific process will not be elaborated here.

[0086] Step S3: Data parsing step

[0087] Upon receiving a request, the terminal device assembles a response packet based on the request message information. For example, if a request is made for data frozen at 10:20, and the meter has frozen data at 10:20, a data packet response is assembled; otherwise, an empty message is returned. When frozen data for a specific frozen channel is available, the message format is as follows: Figure 6 As shown, the measurement point values ​​are placed in the channel, and the OBIS of the measurement point is consistent with the corresponding configured channel.

[0088] After receiving the response message, the terminal performs two-level filtering, which includes the following sub-steps:

[0089] Step 1: Message Inspection. After receiving the acknowledgment message, the receiving end performs inspections, such as checking whether the message is complete, whether it exceeds the minimum message length, and whether the buffer overflows. If the conditions are not met, the receiving process is terminated.

[0090] Step 2: Check the value channel OBIS. Parse the message to obtain the OBIS of the value channel. Use the currentReadIndex to find the requested read item and obtain the OBIS of the requested value channel. Check if the two OBISs are consistent. If they are inconsistent, end the parsing and notify the sender to re-request the message.

[0091] Step 3: Parse the message. Based on the current reading item from the requesting end, obtain the corresponding configuration channel, map the numerical channel (val) to the configuration channel (obis), and store this mapping in a structure. The structure includes the total number of measurement points for each channel, the val for each measurement point, and the corresponding obis. Simultaneously, convert the measurement point clock value to minutes and update it in msgTime.

[0092] Step 4: First-level coarse filtering. First, obtain the corresponding reading period based on the current reading item from the requesting end, and determine if the reading period is divisible by `msgTime`. If it is divisible, then this reading belongs to the reading period of that data category. Second, determine if the data category is empty based on the total sum of the data categories. If not empty, proceed to the corresponding function for fine filtering.

[0093] For example, the curve freeze reading cycles are: 2 minutes for electrical energy, 15 minutes for the four quadrants, 3 minutes for demand, and 5 minutes for instantaneous quantity. The msgTime is 29625190 minutes. If the reading item belongs to the curve freeze data channel, the electrical energy reading cycle 2 is divisible by 29625190. The system checks if the sum of the curve freeze electrical energy data categories is 0. If not, it enters the curve freeze electrical energy interface for fine filtering. Similarly, the four quadrant reading cycle 15 is not divisible by 29625190 and does not belong to the four quadrant reading cycle. Even if it contains four quadrant-related measurement point values, it will not be processed.

[0094] Step 5: Secondary Fine Filtering. After entering the corresponding interface, the reading requirements for different subcategories within the same data category are different. For example, in electrical energy, the total amount and phase of forward active energy are read, while all reverse active energy is read. Based on the three-dimensional data values ​​of the configuration matrix, it is calculated which data for each data subcategory needs to be read (phase, total, rate). The corresponding obis is searched based on the initialization. The presence of the obis can be determined from the structure in Step 3. If it exists, the corresponding value can be obtained; if not, it may be in a subsequent similar value channel. If it is still not found after filtering all similar value channels, it means that the measurement point was not configured when configuring the channel.

[0095] Step 6: Subsequent processing. After obtaining the data values ​​and corresponding OBIS, perform processing such as writing to the test points, saving to the database, and generating response messages.

[0096] Through the coordinated execution of steps S1 to S3 above, this invention achieves full-process batch data collection optimization from source configuration optimization and request phase anti-duplicate to receiving end dynamic filtering, effectively improving communication efficiency, reducing resource consumption, and enhancing scheduling flexibility.

[0097] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that these embodiments are merely illustrative, and those skilled in the art can make various modifications and improvements without departing from the spirit and scope of the present invention; all such modifications and improvements fall within the protection scope of the present invention. The protection scope of the present invention is defined by the claims.

Claims

1. A batch data multi-level filtering acquisition and processing method based on a configuration matrix, applied to an industrial data acquisition system for multi-protocol terminal devices, characterized in that, include: S1, Initialization steps: Construct a three-dimensional strategy configuration matrix and a two-dimensional periodic reading matrix; the strategy configuration matrix is ​​used to configure the total amount, rate and phase of reading for each data subclass, and the periodic reading matrix is ​​used to configure the reading cycle for each data major class; calculate the sum of each data major class according to the strategy configuration matrix, and generate the optimal reading list by combining the periodic reading matrix and periodic matching and channel reuse verification; S2, Data Request Step: Initiate a data request to the terminal device based on the data items in the optimal data reading list. During the request process, record the data reading time of various types of frozen data. By comparing the last data reading time of the current data item with the data reading time of the corresponding data type, filter out duplicate requests to data channels that have already been read at the same time point. S3, Data parsing steps: Receive the response message from the terminal device and perform two-level filtering: The first-level coarse filtering determines whether the message data time meets the period divisibility condition based on the reading cycle, and determines whether to enter the parsing process based on the sum of the data categories. Secondary fine filtering involves dynamically selecting the measurement points to be parsed from the response messages for data extraction and storage based on the copying strategy of the corresponding data subclass in the strategy configuration matrix.

2. The method according to claim 1, characterized in that, The step of calculating the sum of each data category based on the strategy configuration matrix includes: traversing the first two dimensions of the strategy configuration matrix, and for each coordinate in the first two dimensions, traversing the first 6 elements along the third dimension and adding them to the last element of the third dimension to obtain the sum of each data category.

3. The method according to claim 1 or 2, characterized in that, The process of generating the optimal copy list through periodic matching and channel multiplexing verification includes: Find the minimum value in the reading cycle of each data category; Determine if the sum of the data categories corresponding to the minimum value is 0. If it is 0, continue searching for the second smallest value. If it is not 0, check if there is a copying item with the same copying data channel in the list of existing copying schemes. If it exists, determine if the minimum value is the same as or a multiple of the existing copying interval. If so, do not add it to the copying list; otherwise, add it to the copying list.

4. The method according to claim 1, characterized in that, The periodic request in the data request step includes: initializing the next copying index to 0; retrieving each copying item and determining whether the difference between the current time and the previous copying time is greater than or equal to the copying interval; if the condition is met, recording the current copying index and the next copying index, and updating the request time; if the condition is not met, continuing the retrieval.

5. The method according to claim 1 or 4, characterized in that, The filtering of duplicate requests for data channels that have already been copied at the same time point includes: if the current copied item is the first channel of curve freezing, daily freezing, or monthly freezing, then compare the copied time corresponding to this type of data with the previous copied time; if they are the same, then it is determined that the frozen data of this type of channel at this time point has been copied, and the previous copied time of this type of channel and its subsequent channels of the same type at the same time is increased by a copying interval, and the search continues; if they are different, then the copied time of this type of data is updated to the previous copied time, the previous copied time is increased by a copying interval, and the search is returned as successful.

6. The method according to claim 5, characterized in that, After successful retrieval, the assembly request message process begins: the corresponding copy item is found based on the current copy index, the corresponding OBI is obtained based on the copy channel in the copy item, and the request message is assembled based on the data type, channel OBI, and request time of the copy item.

7. The method according to claim 1, characterized in that, The data request step also includes message request: receiving message command, initializing manual recall and reading items, and sending request command according to the reading items. This process does not determine the reading period and does not update the reading time of curve freeze, daily freeze, and monthly freeze.

8. The method according to claim 1, characterized in that, Before receiving the response message from the terminal device and performing two-level filtering, the process also includes message checking: checking whether the response message is complete, whether it exceeds the minimum message length, and whether the buffer overflows. If the conditions are not met, the current receiving process is terminated.

9. The method according to claim 1, characterized in that, The first-level coarse filtering includes: obtaining the corresponding reading period based on the current reading item, and determining whether the message data time msgTime can be divided by the reading period; if it can be divided, further determining whether the sum of the corresponding data categories is not 0; if it is not 0, then proceeding to the second-level fine filtering.

10. The method according to claim 1 or 9, characterized in that, The secondary fine filtering includes: calculating the total amount, rate, and phase reading requirements in each data subclass based on the three-dimensional data values ​​of the strategy configuration matrix, and finding the corresponding obis for the measurement point to be read; determining whether the obis exists based on the structure, and if it exists, extracting the corresponding measurement point value; if it still does not exist after filtering all similar value channels, it means that the measurement point is not configured.

Images