Streaming computation rule engine system and data processing method of power internet of things

Abstract

The invention discloses a streaming computation rule engine system and a data processing method of an electric power internet of things, and relates to the technical field of the electric power internet of things, the streaming computation rule engine system comprises an initialization module, a rule loading module, an analysis module, a data receiving module and a data processing module; wherein the initialization module is used for initializing a system and checking whether a rule meeting a target condition exists in a database or not; the rule loading module is used for reading and loading the rules in the database and starting an independent process for each rule; the analysis module is used for analyzing the corresponding rule in the independent process to generate a corresponding logic tree structure; the data receiving module is used for receiving streaming data to be processed; and the data processing module is used for performing data processing on the to-be-processed streaming data based on the logic tree structure to obtain a processing result. The technical problem that a traditional rule engine is low in efficiency in the complex data stream processing process is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power internet of things, and particularly to a stream computing rule engine system of power internet of things and a data processing method. BACKGROUND

[0002] With the growth of the scale and complexity of the internet of things, edge computing has become increasingly important due to its ability to process data close to the data source. Edge computing not only improves network service response speed, but also reduces the burden on the cloud center. However, the internet of things edge computing faces many challenges in practical application, especially in efficiently processing a large amount of business data in a heterogeneous device environment and quickly responding to abnormal situations, which poses a severe test to the continuity, stability and availability of the business.

[0003] The traditional rule engine is deficient in computing and response ability, and cannot meet the stringent requirements of complex data stream processing and is inefficient. SUMMARY

[0004] The present application aims to solve at least one of the above technical problems by providing a stream computing rule engine system of power internet of things and a data processing method.

[0005] In a first aspect, an embodiment of the present application provides a stream computing rule engine system of power internet of things, deployed in an edge internet of things agent; the system comprises an initialization module, a rule loading module, an analysis module, a data receiving module and a data processing module; wherein the initialization module is configured to initialize the system and check whether there is a rule meeting a target condition in a database; the rule loading module is configured to read and load the rules in the database and start an independent process for each rule; the analysis module is configured to analyze the corresponding rules in the independent process and generate a corresponding logical tree structure; the data receiving module is configured to receive stream data to be processed; and the data processing module is configured to process the stream data to be processed based on the logical tree structure to obtain a processing result.

[0006] Further, the system further comprises a rule adding module configured to obtain new rules added by a user based on a friendly interface or an HTTP server and store the new rules in the database.

[0007] Further, the system further comprises a feedback module configured to feed back the processing result to an edge computing platform in communication connection with the edge internet of things agent.

[0008] Further, the analysis module is further configured to analyze the rules based on SQL statements and generate a corresponding logical tree structure.

[0009] Further, the data receiving module is further configured to receive the to-be-processed streaming data based on the MQTT broker.

[0010] In a second aspect, the embodiments of the present application further provide a data processing method of a streaming computing rule engine system based on power Internet of Things, the method comprising: initializing the system and checking whether there is a rule meeting a target condition in a database; reading and loading the rules in the database and starting an independent process for each rule; parsing the corresponding rule in the independent process to generate a corresponding logical tree structure; receiving to-be-processed streaming data; and performing data processing on the to-be-processed streaming data based on the logical tree structure to obtain a processing result.

[0011] Further, the method further comprises: obtaining an added rule added by a user based on a friendly interface or an HTTP server and storing the added rule in the database.

[0012] Further, the method further comprises: feeding back the processing result to an edge computing platform in communication connection with the edge Internet of Things agent.

[0013] In a third aspect, the embodiments of the present application further provide an electronic device, comprising: a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method provided by the embodiments of the present application when executing the computer program.

[0014] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium stores computer instructions, and the computer instructions are executed by a processor to implement the method provided by the embodiments of the present application.

[0015] The present application provides a streaming computing rule engine system and a data processing method of power Internet of Things, acquires streaming data through the basic service provided by the edge computing framework, stores the processing result of the streaming data and reports to the edge computing platform after processing according to specific rules through the rule engine, and constructs an efficient, flexible and easy-to-manage rule engine system. The present application not only can realize efficient processing of streaming data, but also can provide strong support for real-time monitoring and decision-making of the edge computing platform. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0017] Figure 1 A deployment schematic diagram of a stream computing rule engine system of a power Internet of Things is provided for an embodiment of the present application. Figure 2 A schematic diagram of a stream computing rule engine system of a power Internet of Things is provided for an embodiment of the present application. Figure 3 A schematic diagram of generating a corresponding logic tree structure based on a logic expression is provided for an embodiment of the present application. Figure 4 A stream data processing timing diagram is provided for an embodiment of the present application. Figure 5 A flowchart of a data processing method of a stream computing rule engine system based on a power Internet of Things is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0019] Embodiment one Figure 1 A deployment schematic diagram of a stream computing rule engine system of a power Internet of Things is provided for an embodiment of the present application. As shown in the figure, Figure 1 the rule engine system is deployed in an edge Internet of Things agent, the edge Internet of Things agent further includes an edge computing framework and a database, and the edge Internet of Things agent is in communication connection with various edge devices and edge computing platforms.

[0020] Figure 2 A schematic diagram of a stream computing rule engine system of a power Internet of Things is provided for an embodiment of the present application. As shown in the figure, Figure 2 the system includes an initialization module 10, a rule loading module 20, an analysis module 30, a data receiving module 40, and a data processing module 50.

[0021] Specifically, the initialization module 10 is configured to initialize the system and check whether there is a rule meeting a target condition in the database; the rule loading module 20 is configured to read and load the rules in the database and start an independent process for each rule; the analysis module 30 is configured to analyze the corresponding rules in the independent process and generate a corresponding logic tree structure; the data receiving module 40 is configured to receive stream data to be processed; The data processing module 50 is configured to perform data processing on the to-be-processed streaming data based on the logic tree structure, and obtain a processing result.

[0022] Specifically, as shown in the figure, Figure 2 The system further includes a rule adding module 60 configured to acquire a newly-added rule added by a user based on a friendly interface or an HTTP server, and store the newly-added rule in a database.

[0023] In an optional embodiment of the embodiment of the present application, the rule structure of the server is as follows: { Name string `json:"name"` SQL string `json:"sql"` HandlerTypes int `json:"type"` Url string `json:"url"` Method string `json:"method"` Topic string `json:"topic"` Body string `json:"body"` Username string `json:"username"` Password string `json:"password"` } The "Name" field stores a rule name, and the rule name must be unique and non-repetitive; The "Url" field indicates an api interface of the server: For example, / api / v1 / ping: a server connectivity test interface; / api / v1 / config: a configuration file management interface; / api / v1 / rule: a rule management interface; The "Method" field specifies a request mode of the api interface: "GET": the content of the rule is returned to the client; "PUT": the body of the request is parsed into the rule, and if the rule name does not exist in the database, the rule is added; "DELETE": a rule with a specified rule name is deleted.

[0024] Specifically, the initialization module 10 initializes the system, including configuration loading, setting necessary parameters, resource allocation and other preparations to ensure that the system can respond quickly when receiving data. Connect to the database and prepare to read rule data.

[0025] Specifically, the rule loading module 20 reads the stored rules from the database and starts a separate process for each rule. After initialization, the rule engine opens the database, selects the corresponding collection, queries and reads all stored rules. Create a separate processing process for each compliant rule and load the rule content. This process ensures that each rule can process data independently, improving the parallel processing capability of the system.

[0026] In the embodiment of the present application, the parsing module 30 is also used to parse the rules based on the SQL statement to generate the corresponding logical tree structure.

[0027] Specifically, each rule is parsed in a separate process to generate the corresponding logical tree structure. The parsing module 30 in the rule engine system splits the rule expressed in SQL statements into basic syntax units, performs lexical analysis and syntax analysis, and then converts it into a logical tree form that is easy to calculate. The condition judgment statement of the rule adopts the form of a SQL-like statement, as follows: SELECT {VAR1,VAR2...} FROM DevId WHERE {Calculation-Sentence} WITH WINDOW (X1,X2) Where SELECT {VAR1,VAR2,...} indicates the variables VAR1, VAR2, etc. participating in rule calculation, which need to be matched to the fields in the received data. FROME DevId indicates the device identifier participating in rule calculation, which corresponds to the deviceId field in the received data; WHERE {Calculation-Sentence} indicates the calculation expression that needs to be followed, where Calculation-Sentence is the specific expression content, referring to the various operators of SQL statements and extending various operation functions as shown in Table 1: Table 1. Extended functions of rule engine calculation expression Function Function MAX(col) Returns the maximum value in the data array col MIN(col) Returns the minimum value in the data array col COUNT(col) Returns the number of elements in the data array col SUM(col) Returns the sum of the data array col AVG(col) Returns the average of the data array col SQRT(col) Returns the square root POWER(col1, col2) Returns col1 to the power of col2 ABS(col) Returns the absolute value CEIL(col) Returns the ceiling of col EXP(col1) Returns E to the power of col1 LN(col) Returns the natural logarithm of col LOG(col) Returns the base 10 logarithm of col ARRAY(col, index) Extracts the value from a certain array field in the data POINT(col, Field_Name) Extracts the field from a structure in the data WITH indicates that the above defined temporary result set is applied to the following logical expression; WINDOW (X1, X2) is a representation of a time window, the window contains two parameters: X1, X2, X1 represents the window type, X2 represents the window size in seconds, X1 can be 1, 2, 3, and X2=60 is taken as an example: X1=1: every 30 (X2 / 2) seconds, keep the data in 60 (X2) seconds, and perform calculation once; X1=2: every 60 (X2) seconds, keep the data in 60 (X2) seconds, and perform calculation once; X1=3: every time data arrives, keep the data in 60 (X2) seconds, and perform calculation once; The common time window also has a HOPWINDOW (X1, X2) field, which indicates that a hop window is used, the following parameter X1 indicates the size of the window (that is, how long the data will be kept when calculating), and X2 indicates how long the calculation is performed once every time. The units of both are seconds, and the integer greater than 0 is required.

[0028] Figure 3 is a schematic diagram of generating a corresponding logical tree structure based on a logical expression according to an embodiment of the present application. As shown in Figure 3 , the logical expression is an example: SQL statement: MAX (Ua) > 20 & SQRT (MIN (Ub)) < 10. Then start the data processing service with time window as the cycle period in the rule.

[0029] Specifically, the data receiving module 40 is also used for receiving the to-be-processed stream data based on the MQTT agent.

[0030] In the embodiment of the present application, the rule engine system acts as a subscriber of MQTT and receives the to-be-processed stream data pushed from the MQTT agent. The to-be-processed stream data is transmitted in Json format, and the structure is as follows: { "deviceId": "XXXXX", "param": { "data": { "XX":..., "XX":..., "XX": {...}, "XX": [... ] } } } The data stored in data under param is in the form of k-v key-value pairs, wherein k is in the form of string, indicating the name of the value, and k is case-sensitive, for example, "Ua" and "ua", which are two different variables; v can be a basic data type such as bool, int, float, string, or an array of basic types, a nested structure (structure body). The specific variables in the rule statement described in the parsing module 30 need to be matched with the variables in the data structure.

[0031] Specifically, the data processing module 50 processes the received stream data to be processed in the data processing period according to the pre-generated logical tree structure. The processing logic includes operations such as filtering, calculation and analysis of data. The data is traversed according to the logical tree structure, and records meeting the conditions are filtered. The filtered data is calculated, such as sum, average, etc. The calculation result is matched with the predefined rule to determine whether the rule condition is met. The time sequence is as shown in Figure 4

[0032] Specifically, as shown in Figure 2 , the system further comprises a feedback module 70 for feeding back the processing result to the edge computing platform in communication connection with the edge IoT agent, realizing real-time monitoring and decision-making. This module ensures that the processing result can be timely uploaded to the edge computing platform for further analysis and decision support. The processing result is packaged into a predefined format. The result is sent to the edge computing platform through HTTP or MQTT. It is confirmed that the edge computing platform receives the result and performs corresponding processing.

[0033] The present application constructs an efficient, flexible and easy-to-manage rule engine system. The system not only can realize efficient processing of stream data, but also can provide strong support for real-time monitoring and decision-making of edge computing platform.

[0034] Embodiment two Figure 5 is a flow chart of a data processing method of a stream computing rule engine system based on power Internet of Things according to an embodiment of the present application. As shown in Figure 5 , the method specifically comprises the following steps: Step S502, initialize the system, and check whether there is a rule meeting the target condition in the database.

[0035] Step S504, read and load the rules in the database, and start an independent process for each rule.

[0036] Step S506, parse the corresponding rule in the independent process, and generate the corresponding logical tree structure.

[0037] Step S508, receive the stream data to be processed.​

[0038] In step S510, the data processing is performed on the to-be-processed streaming data based on the logic tree structure, and a processing result is obtained.

[0039] Specifically, the method provided by the embodiment of the present application further includes: obtaining an added rule based on a friendly interface or an HTTP server, and storing the added rule in a database.

[0040] Specifically, the method provided by the embodiment of the present application further includes: feeding back the processing result to an edge computing platform in communication connection with the edge Internet-of-Things agent.

[0041] The present application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method provided by the embodiment of the present application when executing the computer program.

[0042] The present application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by a processor to implement the method provided by the embodiment of the present application.

[0043] It is apparent for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and range of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the involved claims.

[0044] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can be properly combined to form other embodiments which can be understood by those skilled in the art.

Claims

1. A streaming computing rule engine system for the power Internet of Things, characterized by: Deployed in the edge IoT agent; the system includes: an initialization module, a rule loading module, a parsing module, a data receiving module and a data processing module; wherein, The initialization module is used to initialize the system and check whether there are rules that meet the target conditions in the database; The rule loading module is used to read and load the rules in the database and start an independent process for each rule; The parsing module is used to parse the corresponding rules in the independent process and generate the corresponding logical tree structure; The data receiving module is used to receive the streaming data to be processed; The data processing module is used to process the streaming data to be processed based on the logical tree structure to obtain a processing result.

2. The system according to claim 1, wherein: It also includes a rule adding module for acquiring new rules added by the user based on a friendly interface or HTTP server, and storing the new rules in the database.

3. The system according to claim 1, wherein: It also includes a feedback module for feeding back the processing results to an edge computing platform that is communicatively connected to the edge IoT agent.

4. The system according to claim 1, wherein: The parsing module is further used to parse the rules based on the SQL statements and generate a corresponding logical tree structure.

5. The system according to claim 1, wherein: The data receiving module is further configured to receive streaming data to be processed based on the MQTT agent.

6. A data processing method based on the stream computing rule engine system of the power Internet of Things according to any one of claims 1 to 5, characterized in that: The method comprises: Initializing the system and checking whether there are rules meeting the target conditions in the database; Read and load the rules from the database and start a separate process for each rule; Parsing corresponding rules in the independent process to generate corresponding logic tree structures; Receive streaming data to be processed; The to-be-processed streaming data is processed based on the logical tree structure to obtain a processing result.

7. The method according to claim 6, characterized in that: The method further includes: acquiring new rules added by the user based on a friendly interface or an HTTP server, and storing the new rules in the database.

8. The method according to claim 6, wherein: The method further includes: feeding back the processing result to an edge computing platform that is communicatively connected to the edge IoT agent.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 6 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 6 to 8 is implemented.

Images