Real-time data processing system based on object model and calculation expression configuration
By centrally processing and managing object models and computing expressions in the cloud, the problems of insufficient data consistency and computing power in industrial equipment monitoring systems are solved, and efficient real-time data processing and simplified system management are achieved.
Patent Information
- Application Number
- CN202411091829.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-08
AI Technical Summary
In the existing industrial equipment monitoring system, the data processing and management of edge equipment are complex, resulting in difficulty in data consistency and update, insufficient computing power, and affecting real-time data processing efficiency and system stability.
Using a real-time data processing system based on object model and calculation expression configuration, it performs calculation and processing in the cloud. Through object model configuration module and expression configuration module, real-time data calculation and processing are realized to ensure data consistency and integrity. Simplify distributed data management.
Improves the efficiency of data management and system stability, reduces the computational burden and maintenance complexity of edge devices, ensures consistency of computational expressions and processing logic, supports flexible expression configurations and multiple devices and data formats, Adapt to a variety of application scenarios.
Smart Images

Figure CN118916171A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet of Things, and more particularly to a real-time data processing system configured based on a physical model and a calculation expression. Background Art
[0002] In the prior art, the cloud-based system embeds calculation expressions on the edge device side to perform data calculations before uploading it to the cloud. There are a large number of edge devices, which are widely distributed and have high maintenance and update costs. Especially when the calculation expressions need to be updated uniformly, the operation is complicated and error-prone. It has become a challenge to ensure that the software versions of all edge devices are consistent and updated in a timely manner. Version incompatibility issues are prone to occur, thus affecting the overall stability of the system. Coordinating data management between edge devices and the cloud can partially solve the problem of edge device updates and coordination, but it also increases the complexity and management costs of the system. It can be seen that the decentralized management method currently used in traditional industrial equipment monitoring systems leads to difficulties in data consistency and updating. At the same time, insufficient computing power on the device side leads to low efficiency in real-time data processing, as well as complexity and unreliability in data storage and analysis.
[0003] Therefore, how to improve the coordination and real-time data processing capability of the system is an urgent problem to be solved by those skilled in the art. Summary of the invention
[0004] In view of this, the present invention provides a real-time data processing system based on physical model and calculation expression configuration, which performs calculation and processing centrally in the cloud, realizes physical model configuration and expression configuration, and performs calculation in time according to the data collected by edge devices to obtain calculation results.
[0005] In order to achieve the above object, the present invention adopts the following technical solution:
[0006] A real-time data processing system based on physical model and calculation expression configuration, comprising: a physical model configuration module, an expression configuration module and a real-time calculation engine arranged in the cloud;
[0007] The physical model configuration module creates several physical models and configures the properties of the physical models;
[0008] An expression configuration module sets the corresponding expression for the configured physical model;
[0009] The real-time computing engine receives valid monitoring data, retrieves expressions and performs calculations, obtains calculation results, and maps the calculation results to the configured physical model.
[0010] Preferably, a uniquely identified data key value and data type configuration attribute are added in the object model, and saved to the memory database after verification.
[0011] Preferably, corresponding object model tables and fields are created in the in-memory database according to the configured object model.
[0012] Preferably, the expression configuration module selects a physical model and a mathematical operator, introduces physical model attributes through the data key value of the physical model, combines the physical model attributes with the mathematical operator, and constructs an expression; the expression is saved to the memory database after verification.
[0013] Preferably, when the expression configuration module verifies the legality of the expression, the simulated numerical value is substituted into the expression and executed. If the execution is normal, the verification is passed and the expression is saved to the memory database. Otherwise, the expression is adjusted and the verification continues. The expression is adjusted by judging the error content of the expression. If the physical model attribute in the expression does not exist or the data key value is modified, the physical model attribute is reintroduced. If the mathematical operator is wrong, the mathematical operator is modified.
[0014] Preferably, the real-time computing engine includes an expression parsing module and a physical model parsing module; the expression parsing module reads the expression from the memory database, replaces the effective monitoring data into the expression according to the physical model attributes written in the expression, and converts the mathematical operator into an operator function to obtain an operation function, and executes the operation function to obtain the calculation result;
[0015] The object model data parsing module reads the object model from the memory database, parses it to build SQL statements, maps the object model, the field types of the object model table and the real-time monitoring data, uses PreparedStatement to fill the real-time monitoring data of the field types of the object model table, and obtains the stored data.
[0016] The acquired valid monitoring data is replaced into the expression through the data key value of the physical model, the mathematical operators are converted into JavaScript expressions, and the JavaScript engine performs the operation to obtain the calculation result.
[0017] Preferably, the system is also provided with a data reporting module for receiving collected data from edge devices; the data reporting module includes an Internet of Things middleware and a distributed message queue system, the Internet of Things middleware receives the collected data from the edge device and transmits it to the distributed message queue system, and the real-time computing engine reads the collected data from the message queue of the distributed message queue system.
[0018] Preferably, the real-time computing engine also includes a data source module and a filter. The data source module receives the collected data in the message queue and transmits it to the filter for filtering, removes abnormal data, generates valid monitoring data and transmits it to the expression parsing module. The valid monitoring data is a string of semi-structured JSON data.
[0019] Preferably, a distributed database is also included, and the calculation results and stored data are written into the distributed database for persistent storage, which is convenient for subsequent analysis and processing.
[0020] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a real-time data processing system based on physical model and calculation expression configuration, which performs calculation and processing in the cloud, reduces the data synchronization problem between the edge and the cloud, and ensures the consistency and integrity of the data. It simplifies the complexity of distributed data management and improves the efficiency of data management. It can feedback the results immediately, reduces the complexity and error probability of on-site operations, and quickly obtains accurate results; expression updates can be uniformly performed in the cloud to ensure the consistency of all calculation expressions and processing logic; through the physical model configuration module and the expression configuration module, it is possible to easily define and modify the calculation expression according to different business needs, flexibly configure different expressions, support different types of equipment and data formats, and adapt to a variety of application scenarios. The present invention can be applied to the real-time monitoring of the operating status of industrial equipment. Through the key technical means such as centralized management of physical model attributes, flexible configuration of calculation expressions, and real-time processing of high-quality data, it effectively solves the various technical challenges of decentralized management difficulties, limited configuration of calculation expressions, low efficiency of real-time data processing, insufficient data mapping and computing power, and difficulty in ensuring data storage and analysis in traditional industrial equipment monitoring systems, and improves the efficiency, reliability and management convenience of industrial monitoring systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0022] Figure 1 A schematic diagram of a real-time data processing system configured based on a physical model and a calculation expression according to an embodiment of the present invention;
[0023] Figure 2 A schematic diagram of a real-time data processing system configured based on a physical model and a calculation expression according to another embodiment of the present invention. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] The embodiment of the present invention discloses a real-time data processing system based on a physical model and a calculation expression configuration, as follows Figure 1 As shown, it includes a physical model configuration module, an expression configuration module and a real-time computing engine deployed in the cloud; the physical model configuration module creates several physical models and configures various properties of the physical models; the expression configuration module sets the corresponding calculation expression for the configured physical model; the real-time computing engine receives valid monitoring data, calls the expression and performs the calculation, obtains the calculation result, and maps the calculation result to the configured physical model.
[0026] In a specific embodiment, a real-time data processing system configured based on a physical model and a computational expression is as follows: Figure 2 As shown, including:
[0027] 1. Physical model configuration module, to configure physical model attributes: The process of configuring physical model includes creating physical model and physical model attributes. When creating physical model attributes, uniquely identified data key values and data types are added, and the configuration is verified and saved. Various attributes in the physical model are configured, such as current, voltage, temperature, humidity, etc. After the configuration is completed, the physical model configuration is updated to the in-memory database, and the corresponding physical model tables and fields are created in the database for persistent storage of real-time data to ensure the consistency and accessibility of all device data.
[0028] 2. Expression configuration module, perform expression configuration: select and reference the object model attributes in the configuration interface, enter the relevant mathematical operators to define the expression, wherein the object model attributes are referenced through the data key value and combined with the mathematical operators to form an expression that can be arbitrarily combined; after the configuration is completed, verify the legality of the expression. If the verification passes, update the expression to the memory database. If the verification fails, adjust the expression and continue to verify until it passes.
[0029] Furthermore, when verifying the legality of an expression, simulated numerical values are substituted to verify whether the expression can be executed normally and process real data; if the attribute referenced in the expression does not exist or the data key value is modified, the attribute needs to be re-referenced; if characters other than mathematical operators are entered, the expression is judged to be illegal, and errors in the mathematical operators are manually corrected.
[0030] 3. Data reporting module: The edge computing device reports the collected raw data to the IoT middleware according to the format configured by the object model, and then passes it to the distributed message queue system; the edge computing device collects raw data through sensors, including current, voltage, temperature, humidity, etc.
[0031] 4. The real-time computing engine includes a data source module, a filter, an expression parsing module, and a physical model parsing module, which are used for real-time data reading and filtering, as well as real-time computing and processing. Among them:
[0032] The collected raw data is read from the message queue of the distributed message queue system through the data source module. The real-time computing engine reads the real-time monitoring data from the distributed message queue system through the data source module. The filter filters the data to obtain the real-time monitoring data for subsequent computing and processing, and filters out abnormal data to ensure that the data output to the data processing layer is high-quality data.
[0033] The expression parsing module reads the latest configured expression from the memory database, replaces the real-time monitoring data corresponding to the physical model attributes mapped by the expression into the expression, converts the mathematical operators in the expression into operator functions, and calculates the results by the JavaScript engine; among them, the real-time monitoring data is a string of semi-structured JSON data, and the data is obtained through the data key value and replaced into the expression, and the custom symbols are converted into JavaScript expressions.
[0034] The object model parsing module reads the latest configured object model from the memory database, parses the configuration of the object model to build an insert SQL statement, maps the object model, field types of the object model table and real-time monitoring data, uses PreparedStatement to fill in the real-time monitoring data of the object model table field type, obtains the stored data, and writes the stored data into the distributed database.
[0035] 5. Distributed database for real-time data storage: Write the calculation results and storage data into the database to ensure persistent storage and subsequent analysis of the data.
[0036] The beneficial effects of the system in this embodiment include: centralized computing: all data processing and computing tasks are performed in the cloud, and the edge devices are only responsible for data collection and uploading, reducing the computing requirements for edge devices; unified management: calculation expressions and processing logic are uniformly configured and updated in the cloud, avoiding the tedious work of configuring and updating separately on multiple edge devices; simplified maintenance: since the computing tasks are concentrated in the cloud, the maintenance and update of edge devices are greatly simplified, reducing the complexity and error probability of on-site operations; consistency guarantee: the update of calculation expressions and processing logic is uniformly performed in the cloud, ensuring the consistency and accuracy of all computing tasks.
[0037] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0038] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A real-time data processing system based on physical model and calculation expression configuration, characterized in that: include: The physical model configuration module, expression configuration module and real-time computing engine deployed in the cloud; The physical model configuration module creates several physical models and configures the properties of the physical models; An expression configuration module sets the corresponding expression for the configured physical model; The real-time computing engine receives valid monitoring data, retrieves expressions and performs calculations, obtains calculation results, and maps the calculation results to the configured physical model.
2. A real-time data processing system based on physical model and calculation expression configuration according to claim 1, characterized in that: Configure the properties by adding uniquely identified data key values and data types in the object model and save them after verification.
3. A real-time data processing system based on physical model and calculation expression configuration according to claim 2, characterized in that: It also includes an in-memory database, which stores the verified object models and creates corresponding object model tables and fields according to the configured object models.
4. A real-time data processing system based on physical model and calculation expression configuration according to claim 3, characterized in that: The expression configuration module selects the physical model and mathematical operators, introduces the physical model attributes through the data key values of the physical model, combines the physical model attributes with the mathematical operators, and constructs the expression; after the expression is verified, it is saved to the memory database.
5. A real-time data processing system based on physical model and calculation expression configuration according to claim 4, characterized in that: When the expression configuration module verifies the legality of an expression, it substitutes the simulated numerical value into the expression and executes it. If it executes normally, the verification passes and the expression is saved to the memory database. Otherwise, the expression is adjusted and verification continues. The expression is adjusted by judging the error content of the expression. If the physical model attribute in the expression does not exist or the data key value is modified, the physical model attribute is reintroduced. If the mathematical operator is wrong, the mathematical operator is modified.
6. A real-time data processing system based on physical model and calculation expression configuration according to claim 4, characterized in that: The real-time computing engine includes an expression parsing module and a physical model parsing module; The expression parsing module reads the expression from the memory database, replaces the valid monitoring data into the expression according to the physical model attributes in the expression, converts the mathematical operators into operator functions to obtain operation functions, and executes the operation functions to obtain the calculation results; The object model data parsing module reads the object model from the memory database, parses it to build SQL statements, maps the object model, the field types of the object model table and the real-time monitoring data, uses PreparedStatement to fill the real-time monitoring data of the field types of the object model table, and obtains the stored data.
7. A real-time data processing system based on physical model and calculation expression configuration according to claim 1, characterized in that: The system is also equipped with a data reporting module for receiving collected data from edge devices; The data reporting module includes the Internet of Things middleware and the distributed message queue system. The Internet of Things middleware receives the collected data from the edge device and transmits it to the distributed message queue system. The real-time computing engine reads the collected data from the message queue of the distributed message queue system.
8. A real-time data processing system based on physical model and calculation expression configuration according to claim 7, characterized in that: The real-time computing engine also includes a data source module and a filter. The data source module receives the collected data in the message queue and transmits it to the filter for filtering, removes abnormal data, generates valid monitoring data and transmits it to the expression parsing module.
9. A real-time data processing system based on physical model and calculation expression configuration according to claim 6, characterized in that: It also includes a distributed database to save calculation results and store data.
