Whole plant equipment operation condition monitoring data storage method

Abstract

The invention relates to the field of industrial automation and intelligent manufacturing, and discloses a whole-plant equipment operation condition monitoring data storage method which comprises the following steps: S1, data collection and real-time pushing: collecting state and position information of equipment in a plant and pushing the state and position information to a front end for monitoring; s2, data encoding and compression: encoding and compressing the collected data to reduce the storage space; s3, data storage: organizing the encoded data according to time frames and storing the data as a binary data file; s4, data partition storage and indexing: improving data storage and retrieval efficiency through partition storage and index establishment; s5, historical data backtracking and display: carrying out backtracking display by decoding the historical data; and S6, data security and fault tolerance. According to the method, the device state and the position information are subjected to structured coding by using a Protocol Buffer coding scheme, and data compression technologies such as Huffman coding and differential coding are combined, so that the occupation of a storage space is effectively reduced, and the data storage is more compact and efficient.

Description

Technical Field

[0001] The present invention relates to the fields of industrial automation and intelligent manufacturing, and in particular to a method for storing data on monitoring the operation of equipment in a whole plant. Background Art

[0002] In industrial manufacturing and production processes, real-time monitoring and recording of equipment operating status is a key means to ensure efficient, safe and stable operation of the factory. It helps to discover potential problems, optimize production processes, extend equipment life, and take corrective measures in time when equipment fails, thereby reducing downtime and improving production efficiency and quality.

[0003] Existing equipment operation monitoring systems usually save the equipment's operating status by directly storing large amounts of raw data or recording videos. Although these methods can capture a large amount of information, they take up a huge amount of storage space, resulting in great pressure on system storage. Especially in the case of long-term operation, storage devices need to be continuously expanded, increasing hardware investment and maintenance costs. Summary of the invention

[0004] In order to make up for the above shortcomings, the present invention provides a method for storing data monitoring the operation status of equipment in the whole plant, aiming to improve the problem that the existing equipment operation monitoring system usually saves the operating status of the equipment by directly storing a large amount of raw data or recording video, which takes up a huge amount of storage space.

[0005] In a first aspect, the present invention provides the following technical solution: a method for storing data on monitoring the operation of equipment in a whole plant, comprising the following steps:

[0006] S1. Data collection and real-time push: collect the status and location information of equipment in the factory and push it to the front end for monitoring;

[0007] S2, data encoding and compression: encode and compress the collected data to reduce storage space;

[0008] S3, data storage: organize the encoded data by time frame and store it as a binary data file;

[0009] S4, Data partition storage and indexing: Improve data storage and retrieval efficiency through partition storage and indexing;

[0010] S5. Historical data backtracking and display: backtracking and displaying by decoding historical data;

[0011] S6. Data security and fault tolerance: Ensure the security and fault tolerance of data storage.

[0012] Preferably, the data collection and real-time push include regularly collecting real-time status information of each device in the factory, running track status information and transportation equipment location information through a sensor network and an industrial bus system, wherein the sensor network includes ultrasonic sensors, infrared sensors and lidars; removing outliers and noise through data preprocessing, and pushing the processed data in real time to the front end through industrial Ethernet for rendering of the monitoring screen.

[0013] Preferably, the data encoding and compression process includes using the Protocol Buffers encoding scheme to define the structure of the device status information and location information data, converting the status information into integer data, encoding the data of adjacent time points by differential encoding, and compressing the encoded data in combination with Huffman coding;

[0014] The data encoding process adopts data hierarchical encoding, and different encoding methods are used for different types of data, and the different types of data include status information, location information and energy consumption information;

[0015] The data is compressed by combining Huffman coding and differential coding.

[0016] Preferably, the data storage process includes compiling the data structure defined by ProtocolBuffers into a corresponding class object, storing the monitoring data as a binary data file, and organizing the data file according to the concept of time frames. Each frame records the complete monitoring data of a time point, and distinguishes data frames at different time points by newline separators.

[0017] Preferably, the data partition storage and indexing includes partitioning the data files according to preset time periods or device types; establishing timestamp and device identifier indexes during data storage, storing the index data as a compact structure, and updating it regularly.

[0018] Preferably, the historical data backtracking and display process includes, when historical data backtracking is needed, restoring the historical monitoring data into a structured object by reading the data frame and decoding it using ProtocolBuffers, and passing it to the front end; the front end locates the data at a specific time point through the timeline, loads the data frame by frame to simulate the historical monitoring effect, and analyzes the historical data in combination with a machine learning algorithm, identifies abnormal situations and generates a report.

[0019] Preferably, the data security and fault-tolerance process includes generating a check code for each data frame during data storage; adopting a multi-copy redundant storage strategy to store monitoring data on multiple storage nodes; encrypting data using an AES encryption algorithm; and recording each step of data storage and reading operations through a logging mechanism. When an error occurs, the data is restored to a normal state through a log rollback operation.

[0020] In a second aspect, the present invention provides the following technical solution: a system for monitoring and storing data on the operation status of equipment in a whole plant, comprising:

[0021] Data collection and push module, used to collect equipment status and location information from within the factory and push the data to the front end for monitoring;

[0022] Data encoding and compression module, responsible for encoding and compressing the collected data;

[0023] A data storage module, for organizing and storing the encoded data by time frame, and storing it as a binary data file;

[0024] Data partition storage and index module, used to partition data and create indexes;

[0025] The historical data backtracking and display module is responsible for backtracking and displaying by decoding historical data;

[0026] The data security and fault tolerance module is used to provide data storage security and fault tolerance.

[0027] In a third aspect, the invention provides the following technical solution: a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for storing data on the operation status of equipment in the entire plant when executing the computer program.

[0028] In a fourth aspect, the present invention provides the following technical solution: a readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the above-mentioned method for storing data for monitoring the operation status of equipment in the entire plant.

[0029] The present invention has the following beneficial effects:

[0030] 1. In the present invention, the device status and location information are structuredly encoded by using the Protocol Buffers encoding scheme, and combined with data compression technologies such as Huffman coding and differential coding, which effectively reduces the storage space occupied and makes data storage more compact and efficient.

[0031] 2. In the present invention, by encoding the data into a compact binary format, the amount of data is reduced, thereby greatly improving the efficiency of transmission and processing. Differential coding further reduces data redundancy between adjacent time points, making data processing more efficient and controllable.

[0032] 3. In the present invention, data is organized by time frame, and timestamp and device identifier indexes are established, so that data can be retrieved quickly and accurately. At the same time, the data backtracking function is combined with the timeline control, which can easily play back historical data frame by frame, greatly improving the user's understanding and analysis of the device operating status.

[0033] 4. In the present invention, AES encryption technology is used in the process of data storage and transmission to ensure the security of data and prevent unauthorized access. In order to enhance the fault tolerance of data, we introduce a multi-copy redundant storage strategy to synchronously store data in multiple nodes. When a node fails, the data can be restored through other nodes to ensure the reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A method flow chart of the method for storing data on the operation status of equipment in the whole plant proposed by the present invention;

[0035] Figure 2 This is a system architecture diagram of the plant-wide equipment operation status monitoring data storage system proposed by the present invention. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Embodiment 1

[0038] Reference Figure 1 In a first embodiment of the present invention, the present invention provides a method for storing data on monitoring the operation of equipment in a whole plant, comprising the following steps:

[0039] S1. Data collection and real-time push: collect the status and location information of equipment in the factory and push it to the front end for monitoring;

[0040] S2, data encoding and compression: encode and compress the collected data to reduce storage space;

[0041] S3, data storage: organize the encoded data by time frame and store it as a binary data file;

[0042] S4, Data partition storage and indexing: Improve data storage and retrieval efficiency through partition storage and indexing;

[0043] S5. Historical data backtracking and display: backtracking and displaying by decoding historical data;

[0044] S6. Data security and fault tolerance: Ensure the security and fault tolerance of data storage.

[0045] Data collection and real-time push include regularly collecting real-time status information of each device in the factory, running track status information and transportation equipment location information through sensor networks and industrial bus systems. The sensor network includes ultrasonic sensors, infrared sensors and lidars; outliers and noise are removed through data preprocessing, and the processed data is pushed to the front end in real time through industrial Ethernet for rendering of monitoring screens.

[0046] The data encoding and compression process includes using the Protocol Buffers encoding scheme to define the structure of device status information and location information data, converting the status information into integer data, encoding the data of adjacent time points through differential encoding, and compressing the encoded data in combination with Huffman coding;

[0047] The data layering coding is used in the data coding process, and different coding methods are used for different types of data. Different types of data include status information, location information and energy consumption information;

[0048] Data compression uses a combination of Huffman coding and differential coding.

[0049] The data storage process includes compiling the data structure defined by ProtocolBuffers into the corresponding class object, storing the monitoring data as a binary data file. The data file is organized according to the concept of time frames. Each frame records the complete monitoring data of a time point, and the data frames at different time points are distinguished by the newline separator.

[0050] Data partition storage and indexing include partitioning data files according to preset time periods or device types; establishing timestamp and device identifier indexes during data storage, storing index data in a compact structure, and updating it regularly.

[0051] The historical data backtracking and display process includes: when historical data backtracking is needed, by reading the data frame and decoding it with ProtocolBuffers, restoring the historical monitoring data into a structured object and passing it to the front end; the front end locates the data at a specific time point through the timeline, loads the data frame by frame to simulate the historical monitoring effect, and analyzes the historical data in combination with machine learning algorithms, identifies abnormal situations and generates reports.

[0052] The data security and fault-tolerance process includes generating a check code for each data frame during the data storage process; adopting a multi-copy redundant storage strategy to store the monitoring data on multiple storage nodes; using the AES encryption algorithm to encrypt the data; and recording each step of the data storage and reading operation through a logging mechanism. When an error occurs, it is restored to normal through a log rollback operation.

[0053] Specifically, S1, data collection and real-time push

[0054] In the daily operation of factory equipment, in order to grasp the status and location information of the equipment in real time, it is necessary to collect data through sensor networks and industrial bus systems. The sensor network is composed of various types of sensors, including ultrasonic sensors, infrared sensors, and lidar, which are used to accurately collect equipment operation data. The data is transmitted to the central control system through the industrial bus system, and then the data is cleaned by the data preprocessing module to remove outliers and noise to ensure the accuracy and reliability of the data. Finally, the data is pushed to the front-end monitoring system in real time through the industrial Ethernet to realize the real-time display of the operation status of the factory equipment.

[0055] S2. Data encoding and compression

[0056] Data encoding and compression are to reduce the space required for storage. In this method, the Protocol Buffers encoding scheme is used to define the collected device status information, location information, etc. in a structured manner, and the raw data is converted into integer data to reduce the storage space occupied. Differential encoding technology is used to encode data that does not change much at adjacent time points to further reduce data redundancy. Subsequently, Huffman coding is used to compress the encoded data to ensure more efficient data storage.

[0057] S3, Data Storage

[0058] The encoded data is organized by time frame and stored as a binary data file. Each frame of data records the complete device operation information at a time point, so that historical data can be traced back and analyzed later. The data file is compiled into the corresponding class object through the data structure defined by ProtocolBuffers and stored in the file in chronological order. Each frame of data is separated by a line break or other specific delimiter to facilitate subsequent data retrieval and parsing.

[0059] S4, data partition storage and indexing

[0060] In order to improve the efficiency of data storage and retrieval, data files are partitioned and stored according to preset time periods or device types. The amount of data in each data partition is relatively small, which is convenient for fast reading. At the same time, during the data storage process, the system creates a timestamp and device identifier index for each data, and the index data is stored in a compact structure to speed up retrieval. The index is also updated regularly to ensure that the data retrieval efficiency is always in the best state.

[0061] S5. Historical data review and display

[0062] The purpose of retracing and displaying historical data is to reproduce the operating status of the equipment when needed. When a user requests to view historical data, the system reads the data frame of the corresponding time point from the storage file, decodes it using ProtocolBuffers, and restores it to a structured object. The front-end system can accurately select the time point to be viewed through the timeline positioning function, and load the data frame by frame for playback. At the same time, the machine learning algorithm is combined to analyze historical data, identify abnormal situations, and help users judge the status of the equipment and make maintenance decisions.

[0063] S6. Data security and fault tolerance

[0064] In the process of data storage, data security and fault tolerance are crucial. Before each data frame is written to the storage, a checksum generation module will generate a checksum to verify the integrity of the data. In addition, the system adopts a multi-copy redundant storage strategy to store data in multiple storage nodes to prevent data loss due to hardware failure. To ensure the security of data transmission and storage, the system uses the AES encryption algorithm to encrypt data. The logging mechanism is used to record each step of data storage and reading operations so that when an error occurs, the system can restore the state to normal through the log rollback mechanism.

[0065] Moreover, in the coding and compression stage, the combination of Huffman coding and differential coding can effectively reduce data redundancy. Huffman coding efficiently compresses data through variable-length coding, while differential coding encodes the changes between adjacent data, thereby minimizing the amount of data that needs to be stored. The combination of the two coding methods can not only ensure the integrity of the data, but also significantly reduce the storage space occupied by the data.

[0066] The core idea of ​​this method is to achieve the purpose of low storage occupation of historical monitoring data by compressing the monitoring screen frames; the most important part is to store structured data as binary data through an efficient and reversible encoding method.

[0067] The current solution mainly uses Google's open source ProtocolBuffers encoding and decoding solution to achieve an efficient and reversible data encoding solution.

[0068] Specific workflow:

[0069] When the software is running, usually, it is necessary to push the data required for the monitoring screen in real time to the front end for rendering the monitoring screen;

[0070] The software collects all the data of the monitoring screen regularly (usually once every n [1 <= n <= 5] seconds), such as the status information of all equipment, the status information of all running tracks, the current location data of the transportation equipment (AGV, OHT)...;

[0071] Using Google's ProtocolBuffers, define all the data attributes that need to be recorded. At the same time, pay attention to the basic idea of ​​data conversion: because numbers take up fewer bytes than strings when encoding data, all status information is converted into integer structures by defining mapping relationships. For example, the idle state is represented by the number 1, the running state is represented by the number 2, the pickup state is represented by the number 3, and the delivery state is represented by the number 4... In a similar way, all attribute data that can be represented by numbers are converted into digital representations and encapsulated into a new data format;

[0072] Through the above steps, after determining the data structure that needs to be stored, write the ProtocolBuffers definition file, and use the relevant compilation tools to compile the definition file into the required class. In the program, all the data of the monitoring screen is stored in the objects of the relevant class, converted into binary data, and written into the file.

[0073] The above steps implement the main process of encoding a frame of data. At the same time, the time point of the frame of data needs to be encoded and written into the data to accurately display the monitoring situation at each moment during historical tracing.

[0074] Separate different data frames by adding a data frame separator, such as a newline, to the file.

[0075] When historical backtracking is required, the encoded data frame is decoded into a structured object by reading the data line by line in the file and the defined state transition mapping relationship, and passed to the front end for rendering of the monitoring screen.

[0076] Since each row of data frame has its own data time and is complete information, when displaying history, the system can drag the time axis to find data frames at similar time points and start displaying them row by row (frame by frame) at certain intervals to simulate the historical monitoring effect. This solution can 100% restore the operation of the factory at that time.

[0077] Embodiment 2:

[0078] Reference Figure 2 In a second embodiment of the present invention, the present invention provides a plant-wide equipment operation status monitoring data storage system, comprising:

[0079] Data collection and push module, used to collect equipment status and location information from within the factory and push the data to the front end for monitoring;

[0080] Data encoding and compression module, responsible for encoding and compressing the collected data;

[0081] A data storage module, for organizing and storing the encoded data by time frame, and storing it as a binary data file;

[0082] Data partition storage and index module, used to partition data and create indexes;

[0083] The historical data backtracking and display module is responsible for backtracking and displaying by decoding historical data;

[0084] The data security and fault tolerance module is used to provide data storage security and fault tolerance.

[0085] Specifically, data collection and push module:

[0086] Includes a variety of sensors, such as ultrasonic sensors, infrared sensors, and lidar, to collect equipment operating status and location data.

[0087] The collected data is transmitted to the central control system through the industrial bus system, and processed by the data preprocessing module to remove outliers and noise to ensure data accuracy.

[0088] Industrial Ethernet is used to push the processed data to the front-end monitoring system in real time, realizing real-time monitoring and feedback of the operating status of equipment in the factory.

[0089] Data encoding and compression module:

[0090] Use ProtocolBuffers to structure the device status and location information and convert the raw data into compact integer data.

[0091] The differential encoding method is introduced to perform differential processing on data with small changes at adjacent time points to reduce redundancy.

[0092] Combined with Huffman coding technology, the encoded data is further compressed to ensure that the data occupies the smallest storage space while maintaining decoding efficiency and data integrity.

[0093] Data storage module:

[0094] The encoded data is organized and stored according to the concept of time frames, and each time frame represents a complete monitoring cycle.

[0095] Generate the corresponding class object through the data structure defined by ProtocolBuffers and convert it into a binary data file for storage.

[0096] In the data file, use specific delimiters, such as newline characters or markers, to distinguish frame data at different time points for subsequent retrieval and parsing.

[0097] Data partition storage and index module:

[0098] Data files are partitioned and stored according to time periods or device types, reducing the size of a single data file and thus improving access efficiency.

[0099] Create a timestamp index and device identifier index for the data to ensure that the data can be quickly located when queried.

[0100] The index structure is compact and will be updated regularly according to data growth to maintain efficient data retrieval capabilities and ensure the system's responsiveness.

[0101] Historical data backtracking and display module:

[0102] It provides the decoding and display functions of historical data, and can accurately trace the operation status of equipment through the timeline.

[0103] When a user requests to view historical data, the corresponding time frame data is read from the storage system and decoded and restored to structured data using Protocol Buffers.

[0104] The front end uses the timeline control to locate the data at a specific time point and loads it frame by frame, combining machine learning algorithms to analyze historical data to assist in anomaly detection and maintenance decisions.

[0105] Data security and fault tolerance module:

[0106] Before data is stored, a checksum is generated for each data frame to ensure that the data has not been tampered with or damaged, thereby ensuring data integrity.

[0107] Use multi-copy redundant storage to store the same data on multiple nodes to avoid data loss caused by hardware failure and increase the reliability of data storage.

[0108] During data storage and transmission, the AES encryption algorithm is used to prevent unauthorized access and ensure data security.

[0109] By recording each operation step in the log, when the system fails or the data is damaged, the log can be used to roll back and restore the system to its normal state.

[0110] Embodiment 3

[0111] The third embodiment of the present invention is based on the same inventive concept and proposes a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the plant equipment operation status monitoring data storage method of the above embodiment are implemented.

[0112] Embodiment 4

[0113] The fourth embodiment of the present invention is based on the same inventive concept. The present invention proposes a terminal, which includes: a processor and a memory; the processor and the memory communicate with each other; the memory is used to store instructions; the processor is used to execute the instructions in the memory, and execute the plant equipment operation status monitoring data storage method of the above embodiment.

[0114] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0115] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for storing data on the operation status of equipment in the whole plant, characterized in that: The following steps are involved: S1. Data collection and real-time push: collect the status and location information of equipment in the factory and push it to the front end for monitoring; S2, data encoding and compression: encode and compress the collected data to reduce storage space; S3, data storage: organize the encoded data by time frame and store it as a binary data file; S4, Data partition storage and indexing: Improve data storage and retrieval efficiency through partition storage and indexing; S5. Historical data backtracking and display: backtracking and displaying by decoding historical data; S6. Data security and fault tolerance: Ensure the security and fault tolerance of data storage.

2. The method for storing monitoring data of the operation status of the whole plant equipment according to claim 1 is characterized in that: The data collection and real-time push include regularly collecting real-time status information of each device in the factory, running track status information and transportation equipment location information through a sensor network and an industrial bus system. The sensor network includes ultrasonic sensors, infrared sensors and lidars; outliers and noise are removed through data preprocessing, and the processed data is pushed to the front end in real time through industrial Ethernet for rendering of the monitoring screen.

3. The method for storing monitoring data of the operation status of the whole plant equipment according to claim 1 is characterized in that: The data encoding and compression process includes using the Protocol Buffers encoding scheme to define the structure of the device status information and location information data, converting the status information into integer data, encoding the data of adjacent time points through differential encoding, and compressing the encoded data in combination with Huffman encoding; The data encoding process adopts data hierarchical encoding, and different encoding methods are used for different types of data, and the different types of data include status information, location information and energy consumption information; The data is compressed by combining Huffman coding and differential coding.

4. The method for storing monitoring data of the operation status of the whole plant equipment according to claim 1, characterized in that: The data storage process includes compiling the data structure defined by ProtocolBuffers into a corresponding class object, storing the monitoring data as a binary data file, and organizing the data file according to the concept of time frames. Each frame records the complete monitoring data of a time point, and distinguishes data frames at different time points by newline separators.

5. The method for storing monitoring data of the operation status of the whole plant equipment according to claim 1, characterized in that: The data partition storage and indexing includes partitioning and storing data files according to preset time periods or device types; establishing timestamp and device identifier indexes during data storage, storing index data as a compact structure, and updating them regularly.

6. The method for storing monitoring data of the whole plant equipment operation status according to claim 1 is characterized in that: The historical data backtracking and display process includes: when historical data backtracking is required, by reading the data frame and decoding it using ProtocolBuffers, restoring the historical monitoring data into a structured object and passing it to the front end; the front end locates the data at a specific time point through the timeline, loads the data frame by frame to simulate the historical monitoring effect, and analyzes the historical data in combination with a machine learning algorithm, identifies abnormal situations and generates a report.

7. The method for storing monitoring data of the operation status of the whole plant equipment according to claim 1 is characterized in that: The data security and fault tolerance process includes generating a check code for each data frame by generating a check code during data storage; A multi-copy redundant storage strategy is adopted to store monitoring data on multiple storage nodes. The AES encryption algorithm is used to encrypt the data. Each step of data storage and reading operations is recorded through a logging mechanism. When an error occurs, it is restored to normal through log rollback operations.

8. The whole plant equipment operation monitoring data storage system is characterized by: The method for storing data for monitoring the operation status of equipment in the whole plant as claimed in any one of claims 1 to 7 comprises: Data collection and push module, used to collect equipment status and location information from within the factory and push the data to the front end for monitoring; Data encoding and compression module, responsible for encoding and compressing the collected data; A data storage module, for organizing and storing the encoded data by time frame, and storing it as a binary data file; Data partition storage and index module, used to partition data and create indexes; The historical data backtracking and display module is responsible for backtracking and displaying by decoding historical data; The data security and fault tolerance module is used to provide data storage security and fault tolerance.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for storing data on the operation status of equipment in the entire plant as described in any one of claims 1 to 7 is implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for storing data on the operation status of equipment in the entire plant as described in any one of claims 1 to 7 is implemented.

Images