Power equipment data acquisition and processing system and method

Abstract

The invention discloses a power equipment data acquisition and processing system, which comprises a multi-protocol compatible acquisition module, an intelligent network detection module, a system service performance monitoring module and a high-concurrency processing module, the multi-protocol compatible acquisition module is responsible for adaptation and data acquisition of different power equipment protocols and normalization processing of the acquired data; the intelligent network detection module monitors a network link state, diagnoses a network fault reason and pushes fault information in real time; the system service performance monitoring module collects key performance indexes of a server, displays performance data through a visual interface, and triggers an alarm when the indexes are abnormal; the high-concurrency processing module compresses the collected data, realizes efficient pushing and priority processing of the data through a message queue, and guarantees the data processing efficiency in a high-concurrency scene; comprehensive monitoring of power equipment communication and system performance is realized, and data acquisition efficiency and system reliability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power data processing, and in particular to a power equipment data acquisition and processing system and method. Background Art

[0002] In the existing technology, some data collection systems only support a limited number of power equipment communication protocols. For example, they can only connect to some devices that use ModbusTCP or RTU protocols. They are not compatible with various protocols such as PLC's OPCUA and S7 protocols, electricity meter's DL645 protocol, WF102 protocol, IEC104 protocol, etc. This results in the inability to comprehensively and completely collect data from various types of equipment when facing a complex power equipment networking environment. It is difficult for power plant operation and maintenance personnel to obtain all key information from a single system, which increases operation and maintenance costs and management difficulties.

[0003] Moreover, most existing systems focus on data collection itself, and have obvious shortcomings in monitoring the stability of the collection network. During the long-term operation of the equipment, network link disconnection and interruption often occur, but the existing technology is difficult to accurately track the cause of the disconnection and count the number of disconnections. Power plant personnel are unable to accurately determine the root cause of the fault in a timely manner, and the efficiency of fault detection and repair is low, which greatly affects the continuity and reliability of power production.

[0004] Furthermore, as the core support of the entire collection and processing process, the performance status of the server is crucial. However, the existing system generally lacks the real-time monitoring function of the use of key resources such as CPU, memory, and hard disk of the system service. It is difficult for operation and maintenance personnel to grasp the server load in time. When high-concurrency data collection tasks come, it is very easy to cause data loss and processing delays due to server overload, further affecting the stability of power system operation. Summary of the invention

[0005] In view of the many problems existing in the above-mentioned prior art, the present invention proposes a power equipment data acquisition and processing system and method, constructs a collection framework that is fully compatible with multiple power equipment communication protocols, and realizes comprehensive collection of power data.

[0006] The technical solution of the present invention is achieved in this way:

[0007] An electric power equipment data acquisition and processing system, comprising a multi-protocol compatible acquisition module, an intelligent network detection module, a system service performance monitoring module and a high-concurrency processing module;

[0008] The multi-protocol compatible acquisition module is responsible for the adaptation and data acquisition of different power equipment protocols and normalization of the collected data;

[0009] The intelligent network detection module monitors the network link status, diagnoses the cause of network failure, and pushes failure information in real time;

[0010] The system service performance monitoring module collects key performance indicators of the server and displays the performance data through a visual interface, triggering an alarm when the indicators are abnormal;

[0011] The high-concurrency processing module compresses the collected data and implements efficient data push and priority processing through the message queue, thereby ensuring data processing efficiency in high-concurrency scenarios.

[0012] Further, the intelligent network detection module includes a link monitoring unit and a fault diagnosis unit;

[0013] The link monitoring unit uses a heartbeat packet mechanism to periodically send a detection packet to the acquisition device and determines the link status based on the device response;

[0014] When the fault diagnosis unit link is interrupted, network parameters and device logs are collected, the fault reasoning algorithm is used to accurately locate the cause of the disconnection, the number of disconnections is counted and the fault information is pushed to the system service performance monitoring module.

[0015] Furthermore, the system service performance monitoring module includes a resource collection unit and a visual display unit;

[0016] The resource collection unit collects key performance indicators of the server regularly through the operating system API interface and updates the data at fixed time intervals.

[0017] The visualization display unit visualizes the collected performance data and displays it in the form of intuitive charts on the monitoring interface. When the performance index exceeds the threshold, an audible and visual alarm is triggered to remind the operation and maintenance personnel.

[0018] Furthermore, the high-concurrency processing module includes a data compression unit and an MQ push and service consumption unit;

[0019] The data compression unit generates a unique identifier for each collected data, and compresses the data using a lossless compression algorithm based on data characteristics and historical rules to reduce the amount of transmission;

[0020] The MQ push and service consumption unit pushes the compressed data to the MQ message queue, and the service consumer obtains data from the queue according to the priority rules for processing, ensuring that high-priority data is processed first, and ensuring the timeliness and orderliness of data processing in high-concurrency scenarios.

[0021] A method for collecting and processing data of electric power equipment comprises the following steps:

[0022] S1. Dynamically load the predefined adapter module according to the protocol type of the power equipment, establish a communication connection with the power equipment, and collect equipment data according to the protocol standard;

[0023] S2. Process the collected raw data through a preset normalization algorithm and convert it into a unified standardized data structure within the system;

[0024] S3. Generate a unique identifier for each data collection, and compress the data using a lossless compression algorithm based on the data characteristics and historical collection rules to reduce the amount of data transmission;

[0025] S4, encapsulate the compressed data into a preset message format and push it to the MQ message queue, which sorts and distributes the messages according to the preset priority rules;

[0026] S5. Periodically send a heartbeat packet in a preset format to the power equipment, determine the current link connection status based on the equipment response time and content, and mark the link as normal or abnormal;

[0027] S6. Count the number of link interruptions per unit time. When it exceeds a preset threshold, generate link failure warning information and push it to the system service performance monitoring module;

[0028] S7, through the operating system API interface, regularly collect the key performance indicators of the server to form time series data;

[0029] S8. Input the collected performance indicator data into a preset visualization engine, generate a real-time monitoring chart, and update it to the system monitoring interface.

[0030] Furthermore, the process of collecting device data according to the protocol standard in step S1 is specifically as follows:

[0031] According to the protocol type of the power equipment, the corresponding adapter module path is searched in the system configuration file, and the adapter module is dynamically loaded;

[0032] In the adapter module, the protocol standards of the power equipment are parsed to extract the communication parameters and data collection rules;

[0033] Use the extracted communication parameters to establish a network connection with the power equipment and complete the initialization configuration of the communication link;

[0034] According to the protocol standard, send data collection instructions to the power equipment and wait for the equipment to return response data;

[0035] Verify the data returned by the device and determine the validity of the data by comparing the verification code of the data with the expected value.

[0036] In step S2, the collected raw data is converted into a unified standardized data structure within the system:

[0037] After collecting the original data, the preset data type mapping rules are called to map the extracted field information to the standard field names defined within the system;

[0038] According to the value range and data type of the field, perform type verification and range validation on the original data to remove abnormal values;

[0039] Use data format conversion algorithm to convert the timestamp format of original data into a standardized time format unified by the system;

[0040] Fill the normalized data into the corresponding field position according to the data structure format defined within the system;

[0041] Check data integrity based on the mandatory fields in the data structure, and add default values ​​or marks for missing data;

[0042] The converted standardized data structure is obtained by processing with a preset normalization algorithm.

[0043] Furthermore, the process of compressing the data in step S3 to reduce the amount of data transmission is specifically as follows:

[0044] According to the standardized data structure defined within the system, extract the key field information, including device identification, acquisition timestamp and data value;

[0045] Use the hash algorithm to concatenate the device ID and the collection timestamp to generate a unique identifier for the data collected. Obtain historical collection data, count the frequency of occurrence of each field value, and generate a frequency distribution table;

[0046] According to the frequency information in the frequency distribution table, a binary coding tree is constructed using the Huffman coding algorithm;

[0047] Traverse the constructed binary coding tree and assign corresponding variable-length binary codes to each field value;

[0048] Generate a mapping rule between field values ​​and Huffman codes, and replace the field values ​​in the standardized data with corresponding Huffman codes according to the mapping rule;

[0049] Using bit manipulation technology, the replaced variable-length coding sequence is spliced ​​into a continuous binary data stream, the spliced ​​binary data stream is byte-aligned, padded to a fixed length, and the padded binary data is associated with the unique identifier generated in the first step to form a compressed data packet;

[0050] The compressed data packet is stored in a specified storage location, and a mapping relationship between the compressed data packet and the original standardized data is established through a unique identifier to facilitate subsequent decompression and data restoration.

[0051] Furthermore, the process of encapsulating the compressed data into a preset message format in step S4 is specifically as follows:

[0052] Extract the unique identifier and data stream information contained in the compressed data packet as basic data for message encapsulation;

[0053] Generate a message header according to the system predefined message format, which includes the message type, priority identifier and timestamp;

[0054] Combine the data stream information in the compressed data packet with the message header to construct a complete message body;

[0055] Use message serialization technology to convert the message body into binary format;

[0056] Calculate the checksum of the serialized message to generate the integrity check value of the message;

[0057] The checksum is added to the end of the message body to form a final message data packet, and the priority level of the message is determined according to the priority identifier in the message header;

[0058] The process of sorting and distributing messages is as follows:

[0059] Push the message data packets to the corresponding partitions of the MQ message queue according to the priority level;

[0060] The MQ message queue sorts and distributes messages according to preset priority rules. If the message priority is high, it will be processed and forwarded first. If the message priority is low, processing and forwarding will be delayed to ensure that high-priority messages are transmitted and processed in a timely manner.

[0061] Furthermore, the process of determining the current link connection status in step S5 is specifically as follows:

[0062] According to the protocol type of the power equipment, the adapter module is loaded to generate a heartbeat packet in a preset format, the adapter module is used to establish a connection with the power equipment, and the generated heartbeat packet is sent through the communication protocol;

[0063] Obtain the response data of the power equipment to the heartbeat packet and record the time point of receiving the response data;

[0064] Parse the received response data, extract the device status information and response content, and determine whether the recorded response time exceeds the preset threshold. If so, mark the link status as abnormal.

[0065] If the response time is within the preset threshold, it is further determined whether the response content meets expectations. If it meets expectations, the link status is marked as normal;

[0066] Generate a link status record according to the marked link status, and store the record in a system log;

[0067] Count the number of link interruptions within a certain period of time. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated;

[0068] Push the generated link failure warning information to the system service performance monitoring module and update the link health related indicators;

[0069] The process of generating link failure warning information in step S6 is specifically as follows:

[0070] Parse the response data, extract the device status information and response content, and if the response time exceeds the preset threshold, mark the link status as abnormal; if the response time is within the preset threshold and the response content is as expected, mark the link status as normal;

[0071] Generate link status records based on link status and store them in system logs;

[0072] Count the number of link interruptions per unit time, and determine whether the number of interruptions reaches a preset number continuously. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated;

[0073] Push link failure warning information to the system service performance monitoring module and update link health indicators.

[0074] Furthermore, the process of collecting the key performance indicators of the server and forming time series data in step S7 is specifically as follows:

[0075] According to the type and version of the server operating system, load the corresponding system performance collection API interface module;

[0076] Use the system performance collection API interface module to call the function interface provided by the operating system to obtain the original data of CPU usage, memory occupancy, and disk read and write rates;

[0077] By calling the system performance collection function, the real-time performance index data of the server is obtained;

[0078] According to the characteristics of the original performance data, the corresponding data structure is designed, and the original data is mapped to the fields of the structure to form a standardized performance data format;

[0079] Generate a unique timestamp identifier for each collected performance data and sort it in time series;

[0080] Through the timestamp generation algorithm, a unique identifier is generated for each performance data record, and the performance data is sorted in ascending order according to the timestamp to construct time series data.

[0081] Furthermore, the process of generating the real-time monitoring chart in step S8 is specifically as follows:

[0082] According to the preset visualization engine configuration, load the template library of chart types such as line charts and bar charts;

[0083] Use the parsing module of the visualization engine to identify the structure and type of performance data and determine the appropriate chart display format;

[0084] Use the data mapping function of the visualization engine to bind performance data to the data source fields of the chart;

[0085] According to the time series characteristics of the data, the horizontal axis of the chart is set to the time dimension, and the vertical axis is set to the performance indicator value;

[0086] Using visualization engine, it generates monitoring charts including line charts and bar charts;

[0087] The generated charts are displayed on the system monitoring interface through the interface update interface of the visualization engine. If the performance data is updated, the refresh mechanism of the visualization engine is triggered to regenerate the charts and update the interface display;

[0088] According to the interactive events of the monitoring interface, respond to the user's zooming and filtering operations on the chart and dynamically adjust the chart content;

[0089] The log recording function of the visualization engine is used to save the generation and update records of monitoring charts to form an operation history.

[0090] Furthermore, the electric power equipment data collection and processing method further includes:

[0091] S9. Based on the data change trend in the visual chart and the preset anomaly detection algorithm, real-time monitoring and early warning are carried out;

[0092] The process of real-time monitoring and early warning combined with the preset anomaly detection algorithm is specifically as follows:

[0093] According to the visual chart of the system monitoring interface, the data acquisition module is used to extract the real-time change value of the performance indicator data, and the original performance data is converted into a unified format within the system according to the preset data structure;

[0094] Use the preset anomaly detection algorithm to analyze the extracted performance indicator data and identify data fluctuation characteristics. If the performance indicator data fluctuation exceeds the preset threshold range, the alarm mechanism of the anomaly detection algorithm is triggered;

[0095] Based on the output of the anomaly detection algorithm, a detailed description of the performance anomaly is generated, including the anomaly type and time point, and the processed performance data is stored in the system performance database, organized and indexed according to time series;

[0096] Push performance anomaly information to the alarm service module through the message queue, set the alarm priority, and use the processing logic of the alarm service module to match the anomaly information with historical fault records to determine whether it is a known fault type. If it is a known fault type, call the preset fault handling process to obtain corresponding handling suggestions; if it is an unknown fault type, trigger the fault analysis module to conduct in-depth analysis of the anomaly data and extract potential fault features;

[0097] Load the performance data visualization module in the monitoring system, generate a line chart of performance indicator data based on time series data, and realize real-time monitoring and early warning of system performance;

[0098] According to the abnormal status of performance data, push performance alarm information to the system service performance monitoring module, update the server health index, and determine the overall operation status of the system;

[0099] The entire process of anomaly detection and fault analysis is recorded in system logs to form a complete performance monitoring and early warning record.

[0100] Compared with the prior art, the present invention has the following beneficial effects:

[0101] 1. The present invention can dynamically load predefined adapter modules for different protocol types of power equipment, and can flexibly support multiple protocols. By searching and loading the corresponding adapter module path in the system configuration file, it achieves full compatibility with different protocol devices in various complex power equipment networking environments, thereby solving the problem of only supporting limited protocols and difficulty in collecting diversified equipment data in the prior art;

[0102] 2. By analyzing the protocol standards of power equipment, extracting communication parameters and data collection rules, and then using these parameters to establish network connections and complete the initial configuration of communication links, accurately establish communication connections with power equipment of different protocols, effectively collect data from various types of equipment, and provide comprehensive key information for power plant operation and maintenance personnel, reducing operation and maintenance costs and management difficulties;

[0103] 3. By sending heartbeat packets in a preset format to power equipment at regular intervals and judging the link connection status based on the equipment response time and content, the normality of the communication link can be monitored in real time, and network link disconnection, interruption and other faults can be discovered in a timely manner;

[0104] 4. By counting the number of link interruptions per unit time, when it exceeds the preset threshold, link fault warning information is generated and pushed to the system service performance monitoring module, so that power plant personnel can promptly and accurately determine the root cause of the fault, quickly respond to the fault, and improve the efficiency of fault detection and repair, thereby ensuring the continuity and reliability of power production;

[0105] 5. Through the operating system API interface, key performance indicators such as CPU usage, memory occupancy, disk read and write rates of the server are collected regularly and time series data is formed, so that operation and maintenance personnel can grasp the server load in real time;

[0106] 6. Input the collected performance indicator data into the preset visualization engine, generate real-time monitoring charts and update them to the system monitoring interface. Operation and maintenance personnel can understand the server performance status through intuitive charts, discover potential overload risks in advance, avoid data loss and processing delays caused by server overload, and further ensure the stability of power system operation;

[0107] In summary, the present invention dynamically loads adaptation modules for different power equipment protocol types, establishes communication connections and collects data. The collected raw data is normalized, a unique identifier is generated and compressed using Huffman coding, and then encapsulated as a message and pushed to the MQ queue. The link status is monitored through heartbeat packets, and warning information is generated when the number of link interruptions exceeds a threshold. Server performance indicators are regularly collected and data is input into a visualization engine to generate real-time monitoring charts. Based on data change trends and anomaly detection algorithms, the present invention realizes comprehensive monitoring of power equipment communications and system performance, improves data collection efficiency and system reliability, and provides strong support for power equipment management. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 This is a system framework diagram of a power equipment data acquisition and processing system in Example 1;

[0109] Figure 2 It is a framework diagram of the intelligent network detection module in Example 1;

[0110] Figure 3 This is a framework diagram of the system service performance monitoring module in Example 1;

[0111] Figure 4 This is a framework diagram of the high-concurrency processing module in Example 1;

[0112] Figure 5 This is a flow chart of a method for collecting and processing data of electric power equipment according to Example 1. DETAILED DESCRIPTION

[0113] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0114] Example 1

[0115] like Figure 1-4 As shown, this embodiment is a power equipment data acquisition and processing system, including a multi-protocol compatible acquisition module, an intelligent network detection module, a system service performance monitoring module and a high concurrency processing module;

[0116] The multi-protocol compatible acquisition module is responsible for the adaptation and data acquisition of different power equipment protocols and normalization of the collected data;

[0117] The intelligent network detection module monitors the network link status, diagnoses the cause of network failure, and pushes failure information in real time;

[0118] The system service performance monitoring module collects key performance indicators of the server and displays the performance data through a visual interface, triggering an alarm when the indicators are abnormal;

[0119] The high-concurrency processing module compresses the collected data and implements efficient data push and priority processing through the message queue, thereby ensuring data processing efficiency in high-concurrency scenarios.

[0120] Further, the intelligent network detection module includes a link monitoring unit and a fault diagnosis unit;

[0121] The link monitoring unit uses a heartbeat packet mechanism to periodically send a detection packet to the acquisition device and determines the link status based on the device response;

[0122] When the fault diagnosis unit link is interrupted, network parameters and device logs are collected, the fault reasoning algorithm is used to accurately locate the cause of the disconnection, the number of disconnections is counted and the fault information is pushed to the system service performance monitoring module.

[0123] Furthermore, the system service performance monitoring module includes a resource collection unit and a visual display unit;

[0124] The resource collection unit collects key performance indicators of the server regularly through the operating system API interface and updates the data at fixed time intervals.

[0125] The visualization display unit visualizes the collected performance data and displays it in the form of intuitive charts on the monitoring interface. When the performance index exceeds the threshold, an audible and visual alarm is triggered to remind the operation and maintenance personnel.

[0126] Furthermore, the high-concurrency processing module includes a data compression unit and an MQ push and service consumption unit;

[0127] The data compression unit generates a unique identifier for each collected data, and compresses the data using a lossless compression algorithm based on data characteristics and historical rules to reduce the amount of transmission;

[0128] The MQ push and service consumption unit pushes the compressed data to the MQ message queue, and the service consumer obtains data from the queue according to the priority rules for processing, ensuring that high-priority data is processed first, and ensuring the timeliness and orderliness of data processing in high-concurrency scenarios.

[0129] In this embodiment, the multi-protocol compatible acquisition module performs protocol adaptation and data acquisition. According to the protocol type of the power equipment, the predefined adapter module is dynamically loaded. The adapter module parses the protocol standard of the power equipment, extracts communication parameters and data acquisition rules, uses the extracted communication parameters to establish a network connection with the power equipment, sends data acquisition instructions, and waits for the device to return response data; after collecting the original data, the preset data type mapping rules are called to map the extracted field information to the standard field name defined in the system; the original data is type checked and range verified, abnormal values ​​are removed, and the timestamp format is converted into a unified standardized time format of the system; the normalized data is filled into the corresponding field position according to the data structure format defined in the system, and the data integrity is checked, and the missing data is supplemented with default values ​​or marks.

[0130] The intelligent network detection module periodically sends a heartbeat packet in a preset format to the power equipment, and sends the generated heartbeat packet through the communication protocol; obtains the response data of the power equipment to the heartbeat packet, and records the time point of receiving the response data; parses the response data, extracts the device status information and response content, and determines whether the response time exceeds the preset threshold;

[0131] The fault diagnosis unit collects network parameters and device logs, uses fault inference algorithms to accurately locate the cause of disconnection, counts the number of disconnections, pushes fault information to the system service performance monitoring module, generates link fault warning information, and updates link health related indicators;

[0132] In the system service performance monitoring module, the resource collection unit collects key performance indicators of the server at regular intervals through the operating system API interface, such as CPU usage, memory occupancy, disk read and write rates, etc., converts the collected performance indicator data into time series data, and updates the data at fixed time intervals;

[0133] The performance data collected by the visualization display unit is input into the preset visualization engine to generate real-time monitoring charts (such as line charts and bar charts). When the performance indicators exceed the threshold, the sound and light alarms are triggered to remind the operation and maintenance personnel, ensuring that the operation and maintenance personnel can discover abnormal situations in time;

[0134] In the high-concurrency processing module, the data compression unit generates a unique identifier for each collected data, and compresses the data using a lossless compression algorithm (such as Huffman coding) based on the data characteristics and historical collection rules to reduce the amount of data transmission, stores the compressed data in a specified location, and establishes a mapping relationship between the compressed data packet and the original data through the unique identifier;

[0135] The MQ push and service consumption unit encapsulates the compressed data into a preset message format, generates a message header (including message type, priority identifier and timestamp), converts the message body into binary format, and calculates the checksum value and appends it to the end of the message to form the final message data packet. The message data packet is pushed to the MQ message queue. The service consumer obtains data from the queue for processing according to the priority rules, ensuring that high-priority data is processed first, and ensuring the timeliness and orderliness of data processing in high-concurrency scenarios.

[0136] Example 2

[0137] like Figure 5 As shown, this embodiment provides a method for collecting and processing data of electric power equipment, comprising the following steps:

[0138] S1. Dynamically load the predefined adapter module according to the protocol type of the power equipment, establish a communication connection with the power equipment, and collect equipment data according to the protocol standard;

[0139] S2. Process the collected raw data through a preset normalization algorithm and convert it into a unified standardized data structure within the system;

[0140] S3. Generate a unique identifier for each data collection, and compress the data using a lossless compression algorithm based on the data characteristics and historical collection rules to reduce the amount of data transmission;

[0141] S4, encapsulate the compressed data into a preset message format and push it to the MQ message queue, which sorts and distributes the messages according to the preset priority rules;

[0142] S5. Periodically send a heartbeat packet in a preset format to the power equipment, determine the current link connection status based on the equipment response time and content, and mark the link as normal or abnormal;

[0143] S6. Count the number of link interruptions per unit time. When it exceeds a preset threshold, generate link failure warning information and push it to the system service performance monitoring module;

[0144] S7, through the operating system API interface, regularly collect the key performance indicators of the server to form time series data;

[0145] S8. Input the collected performance indicator data into a preset visualization engine, generate a real-time monitoring chart, and update it to the system monitoring interface.

[0146] Furthermore, the process of collecting device data according to the protocol standard in step S1 is specifically as follows:

[0147] According to the protocol type of the power equipment, the corresponding adapter module path is searched in the system configuration file, and the adapter module is dynamically loaded;

[0148] In the adapter module, the protocol standards of the power equipment are parsed to extract the communication parameters and data collection rules;

[0149] Use the extracted communication parameters to establish a network connection with the power equipment and complete the initialization configuration of the communication link;

[0150] According to the protocol standard, send data collection instructions to the power equipment and wait for the equipment to return response data;

[0151] Verify the data returned by the device and determine the validity of the data by comparing the verification code of the data with the expected value.

[0152] Specifically, this step example can be described as follows:

[0153] Define the search path of the adapter module as " / modules / adapters / " in the system configuration file, and dynamically load the adapter module "ModbusRtuAdapter.class" matching the protocol type "MODBUS-RTU" through the Java reflection mechanism. The adapter module parses the protocol standard and extracts communication parameters including baud rate 9600bps, data bit 8 bits, parity bit even parity, stop bit 1 bit, and acquisition period of 5 seconds;

[0154] Use the extracted communication parameters to establish a TCP connection through Socket. If the connection times out, log it and try again after a delay of 1 second. Retry up to 3 times. After the connection is successful, send the acquisition command and set the response timeout to 1 second. If the timeout occurs, resend the command up to 2 times.

[0155] Perform CRC16 check on the response data returned by the device. If the check fails, resend the acquisition instruction. Parse the voltage, current, power and other field values ​​from the valid response data in 16-bit integer format. Convert the timestamp field to Unix timestamp. Verify the value range of the voltage field. The normal range is 0 to 380V. If it exceeds the range, it is marked as invalid.

[0156] Convert the voltage value from volts to kilovolts, retaining 2 decimal places, fill the parsed and converted data into the structure, generate a 36-byte unique ID as the data packet identifier in the format of "E_device number_Unix timestamp", and write the data packet to the Kafka message queue topic "electric_data". If the number of queue messages exceeds 10,000, the earliest 1,000 low-priority data will be discarded.

[0157] In step S2, the collected raw data is converted into a unified standardized data structure within the system:

[0158] After collecting the original data, the preset data type mapping rules are called to map the extracted field information to the standard field names defined within the system;

[0159] According to the value range and data type of the field, perform type verification and range validation on the original data to remove abnormal values;

[0160] If the data verification passes, the normalization algorithm is called to convert the data value into a unified numerical range and unit format; if the data verification fails, an error log is generated and the data status is marked as invalid, and the normalization processing of the current data is skipped;

[0161] Use data format conversion algorithm to convert the timestamp format of original data into a standardized time format unified by the system;

[0162] Fill the normalized data into the corresponding field position according to the data structure format defined within the system;

[0163] Check data integrity based on the mandatory fields in the data structure, and add default values ​​or marks for missing data;

[0164] Through the preset normalization algorithm, a converted standardized data structure is obtained;

[0165] The complete standardized data structure is output to the next link for use by the data identification and binding module. According to the protocol standard parsed by the adapter module, the specific field information of the equipment operation status and performance indicators is extracted from the standardized data structure for subsequent analysis.

[0166] Specifically, this step example can be described as follows:

[0167] After collecting the original equipment operation data, the system will map "dev_status" to "equipment status" and "cpu_usage" to "CPU usage" and other standard field names according to the preset field mapping rules. Then, the original data will be verified, for example, to determine whether the value of the temperature field is between -50℃ and 100℃. If it is out of range, it will be eliminated.

[0168] The data that passes the verification will be normalized, such as converting the temperature data to Celsius and adjusting the range to between 0 and 100. At the same time, the UNIX timestamp in the original data will be converted to a standard format.

[0169] The normalized data will be filled in according to the JSON format defined by the system, and the missing fields will be filled with the default value 0;

[0170] Finally, the complete standardized data structure will be output to the data identification module, which will extract the status and performance indicators such as CPU utilization rate of 85% for subsequent equipment monitoring and anomaly analysis.

[0171] Furthermore, the process of compressing the data in step S3 to reduce the amount of data transmission is specifically as follows:

[0172] According to the standardized data structure defined within the system, extract the key field information, including device identification, acquisition timestamp and data value;

[0173] Use the hash algorithm to concatenate the device ID and the collection timestamp to generate a unique identifier for the data collected. Obtain historical collection data, count the frequency of occurrence of each field value, and generate a frequency distribution table;

[0174] According to the frequency information in the frequency distribution table, a binary coding tree is constructed using the Huffman coding algorithm;

[0175] Traverse the constructed binary coding tree and assign corresponding variable-length binary codes to each field value;

[0176] Generate a mapping rule between field values ​​and Huffman codes, and replace the field values ​​in the standardized data with corresponding Huffman codes according to the mapping rule;

[0177] Using bit manipulation technology, the replaced variable-length coding sequence is spliced ​​into a continuous binary data stream, the spliced ​​binary data stream is byte-aligned, padded to a fixed length, and the padded binary data is associated with the unique identifier generated in the first step to form a compressed data packet;

[0178] The compressed data packet is stored in a specified storage location, and a mapping relationship between the compressed data packet and the original standardized data is established through a unique identifier to facilitate subsequent decompression and data restoration.

[0179] Specifically, this step example can be described as follows:

[0180] According to the defined data structure, key fields such as device ID and timestamp are extracted, and the hash value is calculated using the SHA-256 algorithm as the unique identifier of the data;

[0181] Count the frequency of each field value in the historical data, such as "normal" appears 80 times, "warning" appears 15 times, and "fault" appears 5 times. Based on this, construct a Huffman tree and assign codes "0", "10", and "11". Replace the field values ​​in the data with codes, convert them into binary streams, store them after byte alignment, and associate them with unique identifiers;

[0182] When restoring, the compressed package is retrieved using the identifier, the binary stream is parsed, restored to the original field value according to the encoding mapping table, the standardized data structure is reconstructed, and finally passed to the data processing module to achieve efficient compression storage and lossless restoration.

[0183] Furthermore, the process of encapsulating the compressed data into a preset message format in step S4 is specifically as follows:

[0184] Extract the unique identifier and data stream information contained in the compressed data packet as basic data for message encapsulation;

[0185] Generate a message header according to the system predefined message format, which includes the message type, priority identifier and timestamp;

[0186] Combine the data stream information in the compressed data packet with the message header to construct a complete message body;

[0187] Use message serialization technology to convert the message body into binary format;

[0188] Calculate the checksum of the serialized message to generate the integrity check value of the message;

[0189] The checksum is added to the end of the message body to form a final message data packet, and the priority level of the message is determined according to the priority identifier in the message header;

[0190] The process of sorting and distributing messages is as follows:

[0191] Push the message data packets to the corresponding partitions of the MQ message queue according to the priority level;

[0192] The MQ message queue sorts and distributes messages according to preset priority rules. If the message priority is high, it will be processed and forwarded first. If the message priority is low, processing and forwarding will be delayed to ensure that high-priority messages are transmitted and processed in a timely manner.

[0193] Specifically, this step example can be described as follows:

[0194] When encapsulating the message, the 32-bit device ID and 1KB data stream contained in the compressed data packet are extracted as the basic data. When generating the message header, the CRC32 algorithm is used to calculate the checksum, and the message type is encoded as a 2-byte integer, the priority identifier is a 1-byte integer, and the timestamp is a 64-bit long integer. By combining the message header and the data stream in TLV format, a message body with an average size of 1.5KB is constructed. The message body is serialized using ProtocolBuffers and converted into binary format. The serialized message size is reduced by 20%;

[0195] Calculate the 128-bit MD5 checksum of the serialized message as a means of message integrity verification, and append the checksum to the end of the message body to form a message data packet with an average size of 2KB. According to the 1-byte priority identifier in the message header, the message is divided into three levels: high, medium, and low, corresponding to the three priority partitions of the MQ message queue. The message data packet is evenly pushed to each partition through the load balancing algorithm, and the push rate reaches 5,000 messages per second.

[0196] The MQ message queue uses a priority queue algorithm to sort messages. For high-priority messages, a heap sort algorithm is used for priority processing, with an average delay of no more than 10 milliseconds. For low-priority messages, a first-in-first-out algorithm is used, with a 5-second timeout set. If a timeout occurs, the messages are put back to the end of the queue, with an average delay of no more than 1 second.

[0197] Furthermore, the process of determining the current link connection status in step S5 is specifically as follows:

[0198] According to the protocol type of the power equipment, the adapter module is loaded to generate a heartbeat packet in a preset format, the adapter module is used to establish a connection with the power equipment, and the generated heartbeat packet is sent through the communication protocol;

[0199] Obtain the response data of the power equipment to the heartbeat packet and record the time point of receiving the response data;

[0200] Parse the received response data, extract the device status information and response content, and determine whether the recorded response time exceeds the preset threshold. If so, mark the link status as abnormal.

[0201] If the response time is within the preset threshold, it is further determined whether the response content meets expectations. If it meets expectations, the link status is marked as normal;

[0202] Generate a link status record according to the marked link status, and store the record in a system log;

[0203] Count the number of link interruptions within a certain period of time. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated;

[0204] Push the generated link failure warning information to the system service performance monitoring module and update the link health related indicators;

[0205] Specifically, this step example can be described as follows:

[0206] According to the Modbus protocol adopted by the power equipment, the corresponding Modbus adapter module is loaded, and a heartbeat packet data frame containing fields such as device address, function code, data, and CRC check is generated according to the protocol specification. A communication link is established with the power equipment through a TCP connection, and a heartbeat packet is sent to the device every 5 seconds. The sending timestamp is recorded and a 500-ms response timeout timer is started. The response frame returned by the device is received, and information such as the response time and the device operation status code is extracted. If the response time exceeds 500 ms or the status code is not 0x00, the link status is marked as abnormal, otherwise it is marked as normal.

[0207] The link status, response time, status code and other information are written into the system log database, and the number of link abnormalities per minute is counted. If the abnormality lasts for 3 consecutive minutes, an alarm event is generated and the link health is updated to 60%. It is pushed to the monitoring interface and triggers a text message notification to the operation and maintenance personnel.

[0208] The process of generating link failure warning information in step S6 is specifically as follows:

[0209] Parse the response data, extract the device status information and response content, and if the response time exceeds the preset threshold, mark the link status as abnormal; if the response time is within the preset threshold and the response content is as expected, mark the link status as normal;

[0210] Generate link status records based on link status and store them in system logs;

[0211] Count the number of link interruptions per unit time, and determine whether the number of interruptions reaches a preset number continuously. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated;

[0212] Push link failure warning information to the system service performance monitoring module and update link health indicators.

[0213] Specifically, this step example can be described as follows:

[0214] According to the Modbus protocol type of the power equipment, load the Modbus protocol adapter module, generate a heartbeat packet with a length of 8 bytes, including the device address, function code, data and other fields, use the adapter module to establish a connection with the power equipment through the TCP / IP protocol, send the heartbeat packet at a baud rate of 9600bps, obtain the response data of the power equipment to the heartbeat packet, and record the response time as 15ms. Parse the response data, extract the device status information as "normal", and the response content is "heartbeat response". If the response time exceeds the preset threshold of 20ms, mark the link status as abnormal; if the response time is within the preset threshold and the response content is as expected, mark the link status as normal;

[0215] According to the link status, a link status record is generated, including fields such as timestamp, link ID, and status, and stored in the system log table of the MySQL database. The number of link interruptions within a unit time of 5 minutes is counted to determine whether the number of interruptions reaches 3 times in a row. If the number of interruptions reaches 3 times in a row, then a link fault warning information is generated, including fields such as warning level, link ID, and fault time. The link fault warning information is pushed to the system service performance monitoring module through the RESTAPI interface, and the link health indicator is updated. When the health is lower than 80%, an alarm notification is triggered.

[0216] Furthermore, the process of collecting the key performance indicators of the server and forming time series data in step S7 is specifically as follows:

[0217] According to the type and version of the server operating system, load the corresponding system performance collection API interface module;

[0218] Use the system performance collection API interface module to call the function interface provided by the operating system to obtain the original data of CPU usage, memory occupancy, and disk read and write rates;

[0219] By calling the system performance collection function, the real-time performance index data of the server is obtained;

[0220] According to the characteristics of the original performance data, the corresponding data structure is designed, and the original data is mapped to the fields of the structure to form a standardized performance data format;

[0221] Generate a unique timestamp identifier for each collected performance data and sort it in time series;

[0222] Through the timestamp generation algorithm, a unique identifier is generated for each performance data record, and the performance data is sorted in ascending order according to the timestamp to construct time series data.

[0223] Specifically, this step example can be described as follows:

[0224] For Windows servers, you can load the Windows performance counter API module. For Linux servers, you can load the / proc virtual file system interface module. You can call the GetSystemTimes function to obtain the CPU usage, the GlobalMemoryStatusEx function to obtain the memory occupancy, and the GetDiskPerformanceInfo function to obtain the disk read and write rates. The raw data is converted into a standardized structure containing fields such as CPU usage, used memory, free memory, disk read rate, and disk write rate. The 64-bit Unix timestamp format is used, accurate to milliseconds, to uniquely identify and time-series sort the performance data.

[0225] Apply the sliding average algorithm, use 5 minutes as the time window, smooth the performance time series data, set the CPU usage threshold to 80%, and the memory usage threshold to 90%. If the threshold is exceeded, it is marked as abnormal, and the abnormal time, abnormal indicator and abnormal value are recorded. InfluxDB time series database is used to store performance data, with timestamp as the primary index, to support efficient data writing and query;

[0226] Use the Grafana visualization panel to draw line graphs of CPU usage, memory occupancy, and disk read and write rates based on time series data to display server performance dynamics in real time. When more than three consecutive performance anomalies occur, an abnormal alarm notification is sent to the system service monitoring module, and the server health index is reduced by 10 points until the anomaly is resolved and the health score is restored.

[0227] Furthermore, the process of generating the real-time monitoring chart in step S8 is specifically as follows:

[0228] According to the preset visualization engine configuration, load the template library of chart types such as line charts and bar charts;

[0229] Use the parsing module of the visualization engine to identify the structure and type of performance data and determine the appropriate chart display format;

[0230] Use the data mapping function of the visualization engine to bind performance data to the data source fields of the chart;

[0231] According to the time series characteristics of the data, the horizontal axis of the chart is set to the time dimension, and the vertical axis is set to the performance indicator value;

[0232] Using visualization engine, it generates monitoring charts including line charts and bar charts;

[0233] The generated charts are displayed on the system monitoring interface through the interface update interface of the visualization engine. If the performance data is updated, the refresh mechanism of the visualization engine is triggered to regenerate the charts and update the interface display;

[0234] According to the interactive events of the monitoring interface, respond to the user's zooming and filtering operations on the chart and dynamically adjust the chart content;

[0235] The log recording function of the visualization engine is used to save the generation and update records of monitoring charts to form an operation history.

[0236] Specifically, this step example can be described as follows:

[0237] The configuration file of the visualization engine defines a variety of chart templates, such as LineChart, BarChart, etc. The structure, style, and interaction mode of the chart are described in JSON format. The parsing module uses regular expressions to match keywords in the data structure and selects the appropriate chart template based on data type judgment. The data mapping function extracts the timestamp and indicator value of the performance data through XPath syntax and fills them into the corresponding fields of the chart data source.

[0238] The chart is drawn based on HTML5Canvas. The time axis uses uniform scale values. The vertical axis adjusts the scale adaptively according to the value range of the performance indicator. The generated SVG format chart is sent to the monitoring interface through Ajax request. The chart container is updated in real time using DOM operations. If the performance data is updated, the WebSocket notification is triggered, the data is pulled again and the chart is refreshed.

[0239] The event callback function of the monitoring interface captures the mouse wheel and drag operations, calls the chart zoom and pan interface, passes in the zoom ratio and offset parameters, and dynamically updates the chart display range. The logging function writes the timestamp, operation type, data source and other information of chart generation and update into MongoDB in a fixed format for easy tracing and analysis.

[0240] Furthermore, the electric power equipment data collection and processing method further includes:

[0241] S9. Based on the data change trend in the visual chart and the preset anomaly detection algorithm, real-time monitoring and early warning are carried out;

[0242] The process of real-time monitoring and early warning combined with the preset anomaly detection algorithm is specifically as follows:

[0243] According to the visual chart of the system monitoring interface, the data acquisition module is used to extract the real-time change value of the performance indicator data, and the original performance data is converted into a unified format within the system according to the preset data structure;

[0244] Use the preset anomaly detection algorithm to analyze the extracted performance indicator data and identify data fluctuation characteristics. If the performance indicator data fluctuation exceeds the preset threshold range, the alarm mechanism of the anomaly detection algorithm is triggered;

[0245] Based on the output of the anomaly detection algorithm, a detailed description of the performance anomaly is generated, including the anomaly type and time point, and the processed performance data is stored in the system performance database, organized and indexed according to time series;

[0246] Push performance anomaly information to the alarm service module through the message queue, set the alarm priority, and use the processing logic of the alarm service module to match the anomaly information with historical fault records to determine whether it is a known fault type. If it is a known fault type, call the preset fault handling process to obtain corresponding handling suggestions; if it is an unknown fault type, trigger the fault analysis module to conduct in-depth analysis of the anomaly data and extract potential fault features;

[0247] Load the performance data visualization module in the monitoring system, generate a line chart of performance indicator data based on time series data, and realize real-time monitoring and early warning of system performance;

[0248] According to the abnormal status of performance data, push performance alarm information to the system service performance monitoring module, update the server health index, and determine the overall operation status of the system;

[0249] The entire process of anomaly detection and fault analysis is recorded in system logs to form a complete performance monitoring and early warning record.

[0250] Specifically, this step example can be described as follows:

[0251] The visual chart on the system monitoring interface collects performance indicator data every 5 seconds, including CPU usage, memory occupancy, and disk read and write rates. The original data format is JSON, which is converted into the system's internal binary format after being processed by the data conversion module. Each time the performance data is collected, a 64-bit timestamp identifier is generated, accurate to milliseconds. The data is sorted in the order of the timestamps, and then the moving average algorithm is used to smooth the data. The smoothing window size is 10 data points.

[0252] The anomaly detection algorithm is based on the 3-sigma principle, that is, if a data point deviates from the mean by more than 3 standard deviations, it is considered an anomaly and triggers an alarm mechanism. The anomaly information includes the anomaly type (such as high CPU usage) and the time when the anomaly occurred. This information will be written into the anomaly record table of the system performance database, and the original performance data will also be written into the corresponding time series data table.

[0253] Receive exception information through the message queue and process it according to the preset alarm rules. If the exception information matches the known fault mode, call the preset fault handling process. For example, when the CPU usage rate exceeds 90% for 5 consecutive minutes, the restart operation of the application service is automatically triggered. If the exception information cannot match the known fault mode, the fault analysis module is triggered for in-depth analysis, and machine learning algorithms (such as support vector machines) are used to classify and extract features of the abnormal data;

[0254] The line graph of performance indicators is refreshed every 30 seconds. The horizontal axis of the graph is time, and the vertical axis is the value of the performance indicator. Different performance indicators are distinguished by different colors. If an abnormal point is detected on the line graph, the location of the abnormal point is marked on the graph, and an alarm message pops up on the interface. The system service performance monitoring module updates the health indicator of the server based on the received performance alarm information. The calculation formula of the health indicator is: (1-alarm number / total collection number)*100%. When the health indicator is lower than 60%, the system automatically sends a text message notification to the administrator.

[0255] The specific embodiments of the invention are described in detail above, but they are only examples. The present invention is not limited to the specific embodiments described above. Those skilled in the art should understand that the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A power equipment data acquisition and processing system, characterized in that: It includes multi-protocol compatible acquisition module, intelligent network detection module, system service performance monitoring module and high-concurrency processing module; The multi-protocol compatible acquisition module is responsible for the adaptation and data acquisition of different power equipment protocols and normalization of the collected data; The intelligent network detection module monitors the network link status, diagnoses the cause of network failure, and pushes failure information in real time; The system service performance monitoring module collects key performance indicators of the server and displays the performance data through a visual interface, triggering an alarm when the indicators are abnormal; The high-concurrency processing module compresses the collected data and implements efficient data push and priority processing through the message queue, thereby ensuring data processing efficiency in high-concurrency scenarios.

2. The power equipment data acquisition and processing system according to claim 1, characterized in that: The intelligent network detection module includes a link monitoring unit and a fault diagnosis unit; The link monitoring unit uses a heartbeat packet mechanism to periodically send a detection packet to the acquisition device and determines the link status based on the device response; When the fault diagnosis unit link is interrupted, it collects network parameters and device logs, uses fault reasoning algorithms to accurately locate the cause of the disconnection, counts the number of disconnections, and pushes the fault information to the system service performance monitoring module; The system service performance monitoring module includes a resource collection unit and a visual display unit; The resource collection unit collects key performance indicators of the server regularly through the operating system API interface and updates the data at fixed time intervals; The visualization display unit visualizes the collected performance data and displays it in the form of intuitive charts on the monitoring interface, and triggers an audible and visual alarm to remind the operation and maintenance personnel when the performance index exceeds the threshold; The high-concurrency processing module includes a data compression unit and an MQ push and service consumption unit; The data compression unit generates a unique identifier for each collected data, and compresses the data using a lossless compression algorithm based on data characteristics and historical rules to reduce the amount of transmission; The MQ push and service consumption unit pushes the compressed data to the MQ message queue, and the service consumer obtains data from the queue according to the priority rules for processing, ensuring that high-priority data is processed first, and ensuring the timeliness and orderliness of data processing in high-concurrency scenarios.

3. A method for collecting and processing data of electric power equipment, characterized in that: The following steps are involved: S1. Dynamically load the predefined adapter module according to the protocol type of the power equipment, establish a communication connection with the power equipment, and collect equipment data according to the protocol standard; S2. Process the collected raw data through a preset normalization algorithm and convert it into a unified standardized data structure within the system; S3. Generate a unique identifier for each data collection, and compress the data using a lossless compression algorithm based on the data characteristics and historical collection rules to reduce the amount of data transmission; S4, encapsulate the compressed data into a preset message format and push it to the MQ message queue, which sorts and distributes the messages according to the preset priority rules; S5. Periodically send a heartbeat packet in a preset format to the power equipment, determine the current link connection status based on the equipment response time and content, and mark the link as normal or abnormal; S6. Count the number of link interruptions per unit time. When it exceeds a preset threshold, generate link failure warning information and push it to the system service performance monitoring module; S7, through the operating system API interface, regularly collect the key performance indicators of the server to form time series data; S8. Input the collected performance indicator data into a preset visualization engine, generate a real-time monitoring chart, and update it to the system monitoring interface.

4. A method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of collecting device data according to the protocol standard in step S1 is specifically as follows: According to the protocol type of the power equipment, the corresponding adapter module path is searched in the system configuration file, and the adapter module is dynamically loaded; In the adapter module, the protocol standards of the power equipment are parsed to extract the communication parameters and data collection rules; Use the extracted communication parameters to establish a network connection with the power equipment and complete the initialization configuration of the communication link; According to the protocol standard, send data collection instructions to the power equipment and wait for the equipment to return response data; Verify the data returned by the device and determine the validity of the data by comparing the verification code of the data with the expected value. In step S2, the collected raw data is converted into a unified standardized data structure within the system: After collecting the original data, the preset data type mapping rules are called to map the extracted field information to the standard field names defined within the system; According to the value range and data type of the field, perform type verification and range validation on the original data to remove abnormal values; Use data format conversion algorithm to convert the timestamp format of original data into a standardized time format unified by the system; Fill the normalized data into the corresponding field position according to the data structure format defined within the system; Check data integrity based on the mandatory fields in the data structure, and add default values ​​or marks for missing data; The converted standardized data structure is obtained by processing with a preset normalization algorithm.

5. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of compressing data and reducing the amount of data transmission in step S3 is specifically as follows: According to the standardized data structure defined within the system, extract the key field information, including device identification, acquisition timestamp and data value; Use the hash algorithm to concatenate the device ID and the collection timestamp to generate a unique identifier for the data collected. Obtain historical collection data, count the frequency of occurrence of each field value, and generate a frequency distribution table; According to the frequency information in the frequency distribution table, a binary coding tree is constructed using the Huffman coding algorithm; Traverse the constructed binary coding tree and assign corresponding variable-length binary codes to each field value; Generate a mapping rule between field values ​​and Huffman codes, and replace the field values ​​in the standardized data with corresponding Huffman codes according to the mapping rule; Using bit manipulation technology, the replaced variable-length coding sequence is spliced ​​into a continuous binary data stream, the spliced ​​binary data stream is byte-aligned, padded to a fixed length, and the padded binary data is associated with the unique identifier generated in the first step to form a compressed data packet; The compressed data packet is stored in a specified storage location, and a mapping relationship between the compressed data packet and the original standardized data is established through a unique identifier to facilitate subsequent decompression and data restoration.

6. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of encapsulating the compressed data into a preset message format in step S4 is specifically as follows: Extract the unique identifier and data stream information contained in the compressed data packet as basic data for message encapsulation; Generate a message header according to the system predefined message format, which includes the message type, priority identifier and timestamp; Combine the data stream information in the compressed data packet with the message header to construct a complete message body; Use message serialization technology to convert the message body into binary format; Calculate the checksum of the serialized message to generate the integrity check value of the message; The checksum is added to the end of the message body to form a final message data packet, and the priority level of the message is determined according to the priority identifier in the message header; The process of sorting and distributing messages is as follows: Push the message data packets to the corresponding partitions of the MQ message queue according to the priority level; The MQ message queue sorts and distributes messages according to preset priority rules. If the message priority is high, it will be processed and forwarded first. If the message priority is low, processing and forwarding will be delayed to ensure that high-priority messages are transmitted and processed in a timely manner.

7. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of determining the current link connection status in step S5 is specifically as follows: According to the protocol type of the power equipment, the adapter module is loaded to generate a heartbeat packet in a preset format, the adapter module is used to establish a connection with the power equipment, and the generated heartbeat packet is sent through the communication protocol; Obtain the response data of the power equipment to the heartbeat packet and record the time point of receiving the response data; Parse the received response data, extract the device status information and response content, and determine whether the recorded response time exceeds the preset threshold. If so, mark the link status as abnormal. If the response time is within the preset threshold, it is further determined whether the response content meets expectations. If it meets expectations, the link status is marked as normal; Generate a link status record according to the marked link status, and store the record in a system log; Count the number of link interruptions within a certain period of time. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated; Push the generated link failure warning information to the system service performance monitoring module and update the link health related indicators; The process of generating link failure warning information in step S6 is specifically as follows: Parse the response data, extract the device status information and response content, and mark the link status as abnormal if the response time exceeds the preset threshold; If the response time is within the preset threshold and the response content is as expected, the link status is marked as normal; Generate link status records based on link status and store them in system logs; Count the number of link interruptions per unit time, and determine whether the number of interruptions reaches a preset number continuously. If the number of interruptions reaches a preset number continuously, a link failure warning message is generated; Push link failure warning information to the system service performance monitoring module and update link health indicators.

8. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of collecting the key performance indicators of the server and forming time series data in step S7 is specifically as follows: According to the type and version of the server operating system, load the corresponding system performance collection API interface module; Use the system performance collection API interface module to call the function interface provided by the operating system to obtain the original data of CPU usage, memory occupancy, and disk read and write rates; By calling the system performance collection function, the real-time performance index data of the server is obtained; According to the characteristics of the original performance data, the corresponding data structure is designed, and the original data is mapped to the fields of the structure to form a standardized performance data format; Generate a unique timestamp identifier for each collected performance data and sort it in time series; Through the timestamp generation algorithm, a unique identifier is generated for each performance data record, and the performance data is sorted in ascending order according to the timestamp to construct time series data.

9. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The process of generating the real-time monitoring chart in step S8 is specifically as follows: Load the template library of chart types according to the preset visualization engine configuration; Use the parsing module of the visualization engine to identify the structure and type of performance data and determine the appropriate chart display format; Use the data mapping function of the visualization engine to bind performance data to the data source fields of the chart; According to the time series characteristics of the data, the horizontal axis of the chart is set to the time dimension, and the vertical axis is set to the performance indicator value; Using visualization engine, it generates monitoring charts including line charts and bar charts; The generated charts are displayed on the system monitoring interface through the interface update interface of the visualization engine. If the performance data is updated, the refresh mechanism of the visualization engine is triggered to regenerate the charts and update the interface display; According to the interactive events of the monitoring interface, respond to the user's zooming and filtering operations on the chart and dynamically adjust the chart content; The log recording function of the visualization engine is used to save the generation and update records of monitoring charts to form an operation history.

10. The method for collecting and processing data of electric power equipment according to claim 3, characterized in that: The electric power equipment data collection and processing method further includes: S9. Based on the data change trend in the visual chart and the preset anomaly detection algorithm, real-time monitoring and early warning are carried out; The process of real-time monitoring and early warning combined with the preset anomaly detection algorithm is specifically as follows: According to the visual chart of the system monitoring interface, the data acquisition module is used to extract the real-time change value of the performance indicator data, and the original performance data is converted into a unified format within the system according to the preset data structure; Use the preset anomaly detection algorithm to analyze the extracted performance indicator data and identify data fluctuation characteristics. If the performance indicator data fluctuation exceeds the preset threshold range, the alarm mechanism of the anomaly detection algorithm is triggered; Based on the output of the anomaly detection algorithm, a detailed description of the performance anomaly is generated, including the anomaly type and time point, and the processed performance data is stored in the system performance database, organized and indexed according to time series; Push performance anomaly information to the alarm service module through the message queue, set the alarm priority, and use the processing logic of the alarm service module to match the anomaly information with historical fault records to determine whether it is a known fault type. If it is a known fault type, call the preset fault handling process to obtain corresponding handling suggestions; if it is an unknown fault type, trigger the fault analysis module to conduct in-depth analysis of the anomaly data and extract potential fault features; Load the performance data visualization module in the monitoring system, generate a line chart of performance indicator data based on time series data, and realize real-time monitoring and early warning of system performance; According to the abnormal status of performance data, push performance alarm information to the system service performance monitoring module, update the server health index, and determine the overall operation status of the system; The entire process of anomaly detection and fault analysis is recorded in system logs to form a complete performance monitoring and early warning record.

Images