Large-scale Internet of Things data processing method based on object model and rule engine

Abstract

The invention relates to the technical field of Internet of Things data processing, and discloses a large-scale Internet of Things data processing method based on an object model and a rule engine, and the method comprises the following steps: equipment access and data collection: firstly, enabling a large number of Internet of Things equipment and sensors to access an edge gateway, and enabling the edge gateway to be responsible for the preprocessing of equipment data, equipment data formats are unified, then Internet of Things data are collected through Internet of Things message middleware, and large-scale data analysis is carried out subsequently based on an object model. According to the method, the object model is adopted as a unified data analysis standard, unstructured data generated by the Internet of Things is converted into meaningful structured data, semantic information is given to the data, the data understanding difficulty of a user is lowered, the data analysis speed is greatly increased, the data analysis difficulty and the equipment access threshold are lowered, and the user experience is improved. Therefore, the problems that the collection standards of heterogeneous devices of the Internet of Things are different, and collected data are difficult to analyze and standardize are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things data processing, and in particular to a large-scale Internet of Things data processing method based on a physical model and a rule engine. Background Art

[0002] The core of traditional IoT data processing includes data collection, data cleaning, data preprocessing, data storage and other processes. However, in the process of collecting massive IoT data, due to heterogeneous devices and data description methods, data standards vary during cross-role collaboration, and the collected data is difficult to parse and standardize, which results in the loss of much of the data's value. How to standardize and parse the data and add semantic information to the data has become a major problem.

[0003] The existing patent discloses a terminal control method, device, equipment and medium based on the IoT rule engine (publication number CN113918361A). In the technology disclosed in the patent, during the data cleaning and data preprocessing process, due to the complexity and diversity of IoT business rules, customized data processing logic needs to be developed each time, which consumes a lot of manpower and economic costs, and it is difficult to decouple business processing rules and applications to improve the mining data value of processing massive IoT data. Summary of the invention

[0004] The main technical problem solved by the present invention is to provide a large-scale Internet of Things data processing method based on a physical model and a rule engine, which solves the problems in the above-mentioned background technology.

[0005] To solve the above technical problems, according to one aspect of the present invention, more specifically, a large-scale Internet of Things data processing method based on a thing model and a rule engine comprises the following steps:

[0006] S1. Device access and data collection: First, a large number of IoT devices and sensors are connected to the edge gateway. The edge gateway is responsible for pre-processing the device data and unifying the device data format. Then, the IoT data is collected through the IoT message middleware, and then large-scale data analysis is performed based on the object model.

[0007] S2. Object model definition and parsing data: First, configure several object models according to the physical devices of the Internet of Things. Each object model corresponds to a product, and the configured products are stored in a relational database. Then, create a virtual device on the platform, select the corresponding object model product for mapping, and realize a one-to-one association mapping relationship between the digital virtual device and the actual physical device through the object model and the physical device ID;

[0008] S3. Design the rule calculation module of the rule engine: First, select the object model to which the device belongs to configure the data source, then configure the rules, and finally configure the time interval for rule execution to determine the frequency of rule execution. The specific time interval needs to be selected according to business requirements;

[0009] S4. Design a rule statistics module of the rule engine. The rule statistics is used to perform statistical calculations for a certain attribute value of a specific device over a certain period of time to obtain a statistical value.

[0010] S5. Design a rule fault module of the rule engine, where the rule fault module is used to monitor the device by judging the attribute value of the device according to the rules configured on the front end;

[0011] S6. Design a rule control module of the rule engine: The rule control module is used to trigger corresponding control commands according to the rules configured by the front end, and then send the control commands to the devices to realize the control of the devices and the control linkage between the devices;

[0012] S7, Message push: When the trigger of the rule engine is activated, it will trigger pre-defined actions, including sending emails, text messages, and push notifications, and push the execution results to the backend service, which will process the messages.

[0013] Furthermore, in the step S1, unifying the device data format specifically unifies the heterogeneous data formats of IoT devices of Opcua, Modbus, and S7 communication protocols into the data format of the MQTT protocol.

[0014] Furthermore, in step S2, when parsing the physical model data, the connection channel to which the physical device belongs is determined based on the device id; based on the connection channel client_id, the Topic of the subscription data can be determined, and the corresponding device data can be obtained by subscribing to this Topic. The unique device can be determined through the device identification code and the client_id of the connection channel.

[0015] Furthermore, the physical model parses the data flow as follows:

[0016] Topic:iot / {client_id} / attrbute / sync

[0017] {

[0018] "device_id": / / device id

[0019] "ts": / / timestamp

[0020] "method":"Attribute.Sync", / / Attribute synchronization method

[0021] "params":{ / / property parameters of the object model

[0022] "key1":value,

[0023] "key2":value,

[0024] "key3":value

[0025] }

[0026] }.

[0027] Furthermore, in step S3, by supporting the use of JavaScript script configuration rules, complex calculations are implemented by configuring complex rules, so as to clean, preprocess and denoise the data, including:

[0028]

[0029] In the formula, H represents the standardized data, R represents the original data, u represents the mean of the data, and a represents the sampling rate of the data; then:

[0030]

[0031] Where a represents the sampling rate of the data, N represents the number of sampling points, and T represents the total time (seconds).

[0032] Furthermore, in step S4, the supported statistical methods mainly include count, max, min, sum, avg, stdev, first, last, and spread.

[0033] Furthermore, the specific steps in step S5 are:

[0034] 1) Write rules by using logical operations such as AND, OR, >, =, <, ! =, > =, < =, in, between, and contain.

[0035] 2) Type of configuration failure;

[0036] 3) Configure the execution time interval of the rules according to actual business needs;

[0037] 4) When the attribute value of the device meets the rules for the occurrence of a fault, an alarm will be triggered, and then the alarm information will be pushed to other business systems for corresponding processing.

[0038] Furthermore, the specific steps in step S6 are:

[0039] 1) Configure rules. The rule control module configures rules by using JavaScript scripting language;

[0040] 2) When the control module is triggered, the rule control will send a control command to the designated device at the edge to complete the device control function.

[0041] Beneficial effects:

[0042] 1. The present invention adopts the object model as a unified data analysis standard, converts the unstructured data generated by the Internet of Things into meaningful structured data, and gives the data semantic information, thereby reducing the difficulty for users to understand the data, greatly improving the data analysis speed, and reducing the difficulty of data analysis and the threshold for device access, thereby solving the problem that heterogeneous devices in the Internet of Things have different collection standards, and the collected data is difficult to parse and standardize.

[0043] 2. The present invention designs a rule engine through an embodiment, and provides external functions of real-time data calculation, device scenario control linkage and fault monitoring by configuring rule calculation, rule statistics, rule fault, rule control, etc. The rule engine decouples business processing rules and applications, and can basically cover the common business logic of IoT data processing, improving data processing efficiency while ensuring high concurrency and low latency requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a large-scale IoT data flow graph based on object model and rule engine;

[0045] Figure 2 It is a large-scale IoT flow chart based on object model and rule engine;

[0046] Figure 3 This is the rule failure data flow diagram of the rule engine. DETAILED DESCRIPTION

[0047] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Example 1

[0049] like Figure 1-3 As shown, according to one aspect of the present invention, a large-scale Internet of Things data processing method based on a physical model and a rule engine is provided.

[0050] Device access and data collection: The edge gateway pre-processes the device data and unifies the device data format. Specifically, it unifies the heterogeneous data formats of IoT devices using communication protocols such as Opcua, Modbus, and S7 into the data format of the MQTT protocol. The IoT data is then collected through the IoT message middleware, and subsequently large-scale data analysis is performed based on the object model.

[0051] To define and parse data in the object model, first configure several object models (including attributes, services, and events) based on the physical devices of the Internet of Things. Taking the parking space product as an example, the attributes of the object model include the parking space type (SpaceType) and the parking space status (ParkingStatus). The service can count the parking time according to the parking space status, and the event can define faults, alarms, information, etc. When parsing the object model data, first determine which connection channel the physical device belongs to based on the device id. Based on the connection channel client_id, the topic of the subscription data can be determined accordingly. Subscribe to this topic to obtain the corresponding device data. The unique device can be determined through the device identification code and the connection channel client_id. The object model parsing data flow is as follows:

[0052] Topic:iot / {client_id} / attrbute / sync

[0053] {

[0054] "device_id":"f888e02b-6b28-4a84-8cd5-ad26dfe705b0" / / device id

[0055] "ts":1724921140 / / timestamp

[0056] "method":"Attribute.Sync", / / Attribute synchronization method

[0057] "params":{ / / property parameters of the object model

[0058] "SpaceType":value,

[0059] "ParkingStatus":value

[0060] }

[0061] }

[0062] Example 2

[0063] Rule configuration for rule calculation generally supports two methods. One is to support only simple addition, subtraction, multiplication and division operations to reduce the difficulty of rule configuration. The other is to support the use of JavaScript scripts to configure rules. Complex calculations are achieved by configuring complex rules. For example, to calculate the power value of the device, first add an attribute of power P to the object model. Then, specify the current I and voltage U of a specific device in the rule calculation configuration. The name is configured as power P in the rule, the calculated new attribute is selected as P, the rule is I×U, and the execution time interval is the actual required time.

[0064] In step S3, by supporting the use of JavaScript script configuration rules, complex calculations are implemented by configuring complex rules, so as to clean, preprocess and denoise the data, including:

[0065]

[0066] In the formula, H represents the standardized data, R represents the original data, u represents the mean of the data, and a represents the sampling rate of the data; then:

[0067]

[0068] Where a represents the sampling rate of the data, N represents the number of sampling points, and T represents the total time (seconds).

[0069] The statistical methods supported by rule statistics mainly include count, max, min, sum, avg, stdev, first, last, spread, etc. Take the average power consumption of an IoT device in the current month as an example. First, add the power consumption attribute to the object model, then use the avg function in the front-end configuration, and select the current month as the time range. If the maximum power consumption of the current month is counted, the max function can be used for the power consumption attribute. The linear time interval is one month, and the required value can be counted.

[0070] Rules for rule faults are mainly written by using logical operations such as AND, OR, >, =, <, ! =, > =, < =, in, between, and contain. Taking fire alarm as an example, smoke detectors, temperature detectors, and sound and light alarms can be configured to be associated through OR. As long as an event is triggered, an alarm will be issued. Then, the fault type such as serious fault, warning, and negligible can be configured. Finally, the execution time interval of the rule can be configured according to actual business needs.

[0071] Example 3

[0072] The main function of the rule control module is to trigger the corresponding control command through the rules configured by the front end, and then send the control command to the device to realize the control of the device and the control linkage between devices. Take the fire alarm as an example. For example, if the configured rule is if param1>0and param2>0then{param3=true}, param1 and param2 are input parameters, and param3 is the output parameter, that is, the control command that needs to be sent to the device. When param1>0and param2>0 are both true, the action of param3 can be executed. Param3 can be to turn on the smoke alarm or call the alarm.

[0073] Message push: When the trigger of the rule engine is activated, it will trigger a series of predetermined actions. These actions are pre-defined in the rule engine configuration. Taking alarm notification as an example, when the trigger is activated, the rule engine will push the execution result to the backend alarm center service in the form of an alarm, and the alarm center will process the message.

[0074] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A large-scale Internet of Things data processing method based on object model and rule engine, characterized in that: The following steps are involved: S1. Device access and data collection: First, a large number of IoT devices and sensors are connected to the edge gateway. The edge gateway is responsible for pre-processing the device data and unifying the device data format. Then, the IoT data is collected through the IoT message middleware, and then large-scale data analysis is performed based on the object model. S2. Object model definition and parsing data: First, configure several object models according to the physical devices of the Internet of Things. Each object model corresponds to a product, and the configured products are stored in a relational database. Then, create a virtual device on the platform, select the corresponding object model product for mapping, and realize a one-to-one association mapping relationship between the digital virtual device and the actual physical device through the object model and the physical device ID; S3. Design the rule calculation module of the rule engine: First, select the object model to which the device belongs to configure the data source, then configure the rules, and finally configure the time interval for rule execution to determine the frequency of rule execution. The specific time interval needs to be selected according to business requirements; S4. Design a rule statistics module of the rule engine. The rule statistics is used to perform statistical calculations for a certain attribute value of a specific device over a certain period of time to obtain a statistical value. S5. Design a rule fault module of the rule engine, where the rule fault module is used to monitor the device by judging the attribute value of the device according to the rules configured on the front end; S6. Design a rule control module of the rule engine: The rule control module is used to trigger corresponding control commands according to the rules configured by the front end, and then send the control commands to the devices to realize the control of the devices and the control linkage between the devices; S7, Message push: When the trigger of the rule engine is activated, it will trigger pre-defined actions, including sending emails, text messages, and push notifications, and push the execution results to the backend service, which will process the messages.

2. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 1, characterized in that: In the step S1, unifying the device data format specifically unifies the heterogeneous data formats of IoT devices of Opcua and Modbus communication protocols into the data format of the MQTT protocol.

3. The large-scale Internet of Things data processing method based on object model and rule engine according to claim 1 is characterized in that: In step S2, when parsing the physical model data, the connection channel to which the physical device belongs is determined based on the device id; based on the connection channel client_id, the Topic of the subscription data can be determined, and the corresponding device data can be obtained by subscribing to this Topic. The unique device can be determined through the device identification code and the client_id of the connection channel.

4. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 3 is characterized in that: The physical model parsing data flow is as follows: Topic:iot / {client_id} / attrbute / sync { "device_id": / / device id "ts": / / timestamp "method":"Attribute.Sync", / / Attribute synchronization method "params":{ / / property parameters of the object model "key1":value, "key2":value, "key3":value } }。 5. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 1, characterized in that: In step S3, by supporting the use of JavaScript script configuration rules, complex calculations are implemented by configuring complex rules, so as to clean, preprocess and denoise the data, including: In the formula, H represents the standardized data, R represents the original data, u represents the mean of the data, and a represents the sampling rate of the data; then: Where a represents the sampling rate of the data, N represents the number of sampling points, and T represents the total time (seconds).

6. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 1, characterized in that: In step S4, the supported statistical methods mainly include count, max, min, sum, avg, stdev, first, last, and spread.

7. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 1, characterized in that: The specific steps in step S5 are: 1) Write rules by using logical operations such as AND, OR, >, =, <, ! =, > =, < =, in, between, and contain. 2) Type of configuration failure; 3) Configure the execution time interval of the rules according to actual business needs; 4) When the attribute value of the device meets the rules for the occurrence of a fault, an alarm will be triggered, and then the alarm information will be pushed to other business systems for corresponding processing.

8. The method for processing large-scale Internet of Things data based on object model and rule engine according to claim 1, characterized in that: The specific steps in step S6 are: 1) Configure rules. The rule control module configures rules by using JavaScript scripting language; 2) When the control module is triggered, the rule control will send a control command to the designated device at the edge to complete the device control function.

Images