Multi-mode multi-parameter intelligent operation and maintenance platform with real-time front-end sensing and cloud side-end cooperation

Abstract

The invention relates to a multi-mode multi-parameter intelligent operation and maintenance platform with real-time front-end sensing and cloud side end cooperation, and belongs to the field of operation and maintenance systems. The terminal equipment layer is used for collecting and transmitting bottom layer data in real time; the edge computing layer is used for enhancing real-time performance and expandability; the cloud platform layer is used for global analysis, optimization and coordination of the system; the operation and maintenance platform comprises a multi-modal data classification module which is used for dividing data into time sequence data, relational data and coded data. The multi-modal data model under the cloud side-end collaborative architecture not only can support flexible storage of various types of data, but also can ensure efficient management and utilization of the data through accurate query and update operations.

Description

Technical Field

[0001] The present invention belongs to the field of operation and maintenance systems, and in particular relates to a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration. Background Art

[0002] Multimodal data usually refers to data types from different sources, such as relational data, graph data, time series data, images, videos, etc. Existing research mainly focuses on fusing multimodal data to improve the performance of machine learning models. However, there is relatively little research on the access and operation of multimodal data, especially the lack of theoretical research on data operations and integrity constraints.

[0003] Traditional data models, such as relational models and XML models, often have compatibility issues when facing multimodal data, resulting in data loss, redundancy, or damage to data integrity. Therefore, existing data storage and query systems face many challenges when processing multimodal data.

[0004] Based on the above problems, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge collaboration is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration, which is used to improve the processing speed of large-scale terminal multi-modal and multi-parameter data.

[0006] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] A multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration. The system framework adopts a layered design, including: a terminal device layer for real-time collection and transmission of underlying data; an edge computing layer for enhancing real-time performance and scalability; a cloud platform layer for global analysis, optimization and coordination of the system;

[0008] The operation and maintenance platform includes a multimodal data classification module for classifying data into time series data, relational data and coded data, specifically including:

[0009] Time series data storage module, which is used to store time series data based on time series base class and high-dimensional time series base class. Data points contain timestamp, sensor ID, data value fields, and are stored in the database in the form of tuples;

[0010] The relational data storage module is used to store device status data based on the attribute base class, including device ID, device status, start time, and end time fields, which are stored in the device status table in the form of tuples;

[0011] The image and video data storage module is used to store image and video data based on the coding base class. The image data is stored as an image file reference through coding compression and is associated with the image ID, device ID, and shooting time metadata. The video data is stored in the cloud computing layer and is stored in fragments, with timestamps and device ID metadata attached.

[0012] Beneficial effects:

[0013] The multimodal data model under the cloud-edge-end collaborative architecture of this application can not only support flexible storage of various types of data, but also ensure efficient management and utilization of data through precise query and update operations.

[0014] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0016] Figure 1 This is a diagram of the multi-modal and multi-parameter storage architecture of an embodiment of the present disclosure.

[0017] Figure 2 This is a system architecture diagram of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] A multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration. The system framework adopts a layered design and includes the following three core layers:

[0020] Terminal device layer: responsible for the real-time collection and transmission of underlying data, and is the data input of large-scale terminal systems. The terminal device layer includes: Terminal devices: including sensors, actuators, etc., used to monitor key parameters in industrial production processes, such as temperature, pressure, flow, etc. Data acquisition module: a lightweight software / hardware module embedded in the device to complete data preprocessing and upload. Network connection interface: supports multiple communication protocols to interact with edge computing nodes or the cloud.

[0021] The specific operation steps of the terminal device layer are as follows: Terminal device is responsible for obtaining the operation data of various equipment in the industrial system, such as real-time parameters such as temperature, pressure, and position.

[0022] The data acquisition module pre-processes the device and performs preliminary processing capabilities to reduce the amount of transmitted data. The preliminary processing includes denoising and compression.

[0023] The network connection interface realizes device control and performs specific operations under edge computing or cloud instructions, including adjusting device status, shutting down, etc.

[0024] Edge computing layer: The edge computing layer is the system's regulation and deployment layer, providing a bridge between the terminal and the cloud to enhance real-time performance and scalability. The edge computing layer includes: edge gateway devices, which are responsible for local data aggregation and transmission, provide protocol conversion functions, and unify the data formats of different terminals. Edge computing nodes deploy lightweight containerized microservices to achieve real-time data analysis, anomaly detection, and some control logic. Support equipment health diagnosis and early warning functions. Task distribution management module dynamically allocates computing tasks to ensure efficient use of edge resources.

[0025] The specific operation steps of the edge computing layer are: process and analyze local data through edge computing nodes to reduce the frequency of data upload to the cloud platform. Through the task distribution management module: tasks with high real-time requirements are completed locally to avoid response delays caused by network delays. The tasks with real-time requirements include anomaly detection and device protection. The edge gateway unifies the data protocol of terminal devices to prepare for subsequent analysis.

[0026] Cloud platform layer: The cloud platform layer is responsible for the system's computational processing, global analysis, optimization and coordination, and is the core of intelligent operation and maintenance. The cloud platform layer includes a microservice management module, which manages various services in a containerized manner based on a cloud-native architecture to achieve elastic expansion and high availability. Data storage and management, including a distributed storage system for efficient storage and rapid access to massive amounts of data, supports the storage of structured and unstructured data. The analysis engine uses artificial intelligence and machine learning algorithms to analyze historical and real-time data, identify equipment operation trends and potential risks, and provide intelligent optimization recommendations. User interface layer: supports multiple access terminals for data monitoring and operation. Furthermore, the display terminal can provide real-time dashboards, report generation and alarm push functions.

[0027] The operation steps of the cloud platform layer are as follows: Data storage and management module, which stores and deeply analyzes massive data to identify equipment operation modes and trends. Analysis engine, which provides decision support through artificial intelligence and big data analysis. Microservice management module, which realizes centralized management of all equipment, including fault prediction, operation and maintenance planning, and resource allocation.

[0028] Classify the multimodal data in the system; the classified multimodal data includes:

[0029] Time series data: sensor data such as temperature, humidity, and pressure, presented in the form of time series.

[0030] Relational data: equipment status information, such as equipment start and stop status, work efficiency, etc.

[0031] Encoded data: Image and video data captured by surveillance cameras for fault detection and security monitoring.

[0032] In order to support the storage and query of multimodal data, different designs are used to provide flexible storage methods for different data types, as follows:

[0033] Time series data storage based on time series base class and high-dimensional time series base class: At the terminal device layer and edge computing layer, the time series data generated by temperature, humidity, and pressure sensors will be mapped to the time series base class. Each data point contains fields such as timestamp, sensor ID, and data value. For example, the data collected by the temperature sensor every minute will be stored through the time series base class and saved in the database in the form of tuples for subsequent query and analysis.

[0034] Relational data storage based on attribute base classes: The start and stop status of the equipment, equipment operating efficiency and other information are stored as relational data. Equipment information will be extended through attribute base classes, and the data of each device will be stored in the form of tuples. This data will be stored in the device status table, which contains fields such as device ID, status, start time, end time, etc., and supports cross-device queries.

[0035] Image and video data storage based on the encoding base class: The image and video data captured by the surveillance camera are stored through the encoding base class. When the image data is stored, it is encoded and compressed and stored as a reference to the image file. The associated tuple contains fields such as image ID, device ID, and shooting time. The storage of image and video data is mainly concentrated in the cloud computing layer. Due to the large volume of image and video data, it needs to be stored in shards and metadata such as timestamps and device IDs are attached to facilitate query and analysis.

[0036] To support efficient querying of multimodal data, two types of queries are adopted: ad hoc queries and continuous queries.

[0037] Ad hoc query: Ad hoc query is often used in tasks such as equipment maintenance and troubleshooting. Suppose the user needs to query all records of temperature sensors exceeding 40 degrees in the past week. The query process includes the following steps:

[0038] Query object: The target data is the time series data of all temperature sensors, and the query condition is that the temperature value exceeds 40 degrees.

[0039] Query mode: The data mode is in the form of a time series, which contains timestamp, sensor ID, and temperature value fields.

[0040] Query conditions: The query conditions are that the temperature value (A[temperature]) is greater than 40 and the time range is within the past week.

[0041] The query will be completed in the cloud, which will filter the data items that meet the requirements according to the query conditions and return the records that meet the conditions.

[0042] Among them, continuous query: continuous query is used for real-time data monitoring, usually involving time series data. For example, real-time monitoring of temperature sensor data and triggering an alarm, when the temperature value exceeds a certain threshold (such as 60 degrees), the system will automatically perform data updates or trigger an alarm. Continuous query can be performed at the edge computing layer or the cloud computing layer, and query tasks are executed periodically according to preset conditions.

[0043] In actual applications, in addition to query operations, data also needs to be updated and modified. For example, when the device status changes, the device status data needs to be updated. The operation steps include:

[0044] Insert operation: When a new device joins the system, the edge device inserts information such as device status and attribute data into the relational database.

[0045] Deletion operation: When a device is removed or no longer used, the edge computing device deletes the corresponding device data from the database.

[0046] Modification operation: After a device fails, the device status will be marked as "faulty". At this time, the cloud will update the device status information and trigger the alarm system.

[0047] When querying, updating and modifying, data needs to be retrieved. The specific steps of data retrieval are as follows: classify data extraction;

[0048] A Extraction: Extracting some data items from a data table that meet the user-specified attribute set and rule set. The result of the extraction is a new data set in which each data item meets the specified attribute conditions. For example, if you want to extract products with a qualified rate greater than a certain threshold.

[0049] TopK extraction: select the top k data items.

[0050] K-important extraction: K-important extraction is used to extract the most important k data items from a data table, which have significant time differences or large deviations from other data items.

[0051] The specific extraction of A is as follows;

[0052] Input: Data table: raw data containing multiple data items; Attribute set: user-defined attribute set to be queried, which may include, for example, product qualification rate, etc.; Rule set: conditions for attributes in each attribute set. For example, it may include a condition that a certain attribute is greater than a certain threshold.

[0053] Output: Extraction result: the data subset that meets the conditions.

[0054] The TopK extraction is as follows;

[0055] Input: Data table: original data containing multiple data items, k: specifies the first k data items to be extracted.

[0056] Output: Extraction result: a data subset containing the top k data items.

[0057] kThe important extractions are as follows;

[0058] Input: Data table: raw data containing multiple data items. k: specifies the k most important data items to be extracted. Time information: the timestamp of each data item. Difference evaluation: an indicator used to measure the difference between data items.

[0059] Output: Extraction result: Contains the most important k data items.

[0060] The multimodal data model under the cloud-edge-end collaborative architecture can not only support the flexible storage of various types of data, but also ensure the efficient management and utilization of data through precise query and update operations. This architecture is particularly suitable for scenarios such as large-scale industrial Internet of Things, and can effectively solve the challenges of multimodal data in storage, query, and update, and improve the overall performance of the system.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration, characterized by: The system framework adopts a layered design, including: a terminal device layer for real-time collection and transmission of underlying data; an edge computing layer for enhancing real-time performance and scalability; a cloud platform layer for global analysis, optimization and coordination of the system; The operation and maintenance platform includes a multimodal data classification module for classifying data into time series data, relational data and coded data, specifically including: Time series data storage module, which is used to store time series data based on time series base class and high-dimensional time series base class. Data points contain timestamp, sensor ID, data value fields, and are stored in the database in the form of tuples; The relational data storage module is used to store device status data based on the attribute base class, including device ID, device status, start time, and end time fields, which are stored in the device status table in the form of tuples; The image and video data storage module is used to store image and video data based on the coding base class. The image data is stored as an image file reference through coding compression and is associated with the image ID, device ID, and shooting time metadata. The video data is stored in the cloud computing layer and is stored in fragments, with timestamps and device ID metadata attached.

2. According to claim 1, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: Relational data, including the start and stop status of the equipment, and work efficiency equipment status information; relational data, including the start and stop status of the equipment, and work efficiency equipment status information; coded data, including images and video data captured by surveillance cameras, which are used for fault detection and security monitoring.

3. According to claim 1, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: The system supports two types of query operations, namely: Instant query: users can query historical data according to query conditions for equipment maintenance and troubleshooting; Continuous query supports real-time data monitoring. The system executes query tasks regularly or in real time according to preset conditions, and issues alarms or data updates.

4. According to claim 3, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: Ad hoc query includes the following steps: determine the query object, set the query conditions, filter the data items that meet the requirements in the cloud according to the query conditions, and return the records that meet the conditions.

5. According to claim 3, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: The continuous query consists of the following steps: Real-time monitoring of sensor data. When the sensor parameter acquisition value exceeds the set threshold, the system automatically updates the data or triggers an alarm. Query tasks can be performed at the edge computing layer or the cloud computing layer, and the query execution level can be dynamically selected according to real-time requirements.

6. According to claim 1, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: The system supports data update and modification operations, including: Insert operation: When a new device is added, the edge computing device inserts the device status and attribute data information into the relational database; Deletion operation: When a device is removed or no longer used, the edge computing device deletes the corresponding device data from the database; Modify the operation. When a device failure occurs, the cloud will update the device status and trigger an alarm.

7. According to claim 3, a multi-modal and multi-parameter intelligent operation and maintenance platform with real-time front-end perception and cloud-edge-end collaboration is characterized in that: Retrieve data when querying, updating, and modifying, and classify data extraction; A Extraction: Extracting some data items that meet the user-specified attribute set and rule set from a data table. The result of the extraction is a new data set, in which each data item meets the specified attribute conditions; TopK extraction: select the top k data items; K-important extraction: K-important extraction is used to extract the most important k data items from a data table, which have significant time differences or large deviations from other data items.

Images