Mechanical equipment early warning method, equipment and medium

Abstract

The invention provides a mechanical equipment early warning method and device and a medium, and relates to the field of tunnel engineering, and the method comprises the steps that a data access layer obtains a mechanical equipment data list, cleans and standardizes the data of mechanical equipment, obtains intermediate data, and writes the intermediate data into a Kafka platform; intermediate data of the Kafka platform are simultaneously output to the real-time processing layer and the data storage layer; the batch processing layer obtains the time sequence data based on the data storage layer, carries out batch analysis and carries out early warning when a batch analysis result meets a preset batch processing alarm rule, and the real-time processing layer receives the intermediate data, carries out distribution on the intermediate data, obtains the preprocessed intermediate data as to-be-used real-time data and sends the to-be-used real-time data to the data storage layer. According to the invention, real-time analysis is carried out based on the to-be-used real-time data, and when the real-time analysis result meets the preset real-time alarm rule corresponding to the to-be-used real-time data, early warning is carried out, so that more comprehensive early warning is realized.

Description

Technical Field

[0001] The present invention relates to the field of tunnel engineering, and in particular to a mechanical equipment early warning method, equipment and medium. Background Art

[0002] With the rapid development of infrastructure construction, tunnel engineering projects are increasingly widely used in the fields of transportation, energy, etc. In these projects, large-scale mechanical equipment (such as three-arm rock drilling rigs, anchor injection machines, wet shotcrete rigs, secondary lining rigs, etc.) are important construction tools, and their efficient and safe operation is directly related to the progress and quality of the project. Therefore, the demand for real-time monitoring and early warning systems for these equipment has become particularly urgent.

[0003] At present, the monitoring system of large-scale mechanical equipment usually relies on the sensors built into the equipment to collect various parameters (such as temperature, pressure, vibration, etc.) during the operation of the equipment to achieve real-time monitoring of the equipment status. However, these systems have the following major shortcomings: Limited real-time data monitoring: Most existing systems can only provide basic data monitoring functions, lack the ability to conduct in-depth analysis and early warning of data, and cannot identify the risk of exceeding the standard and safety hazards of operations in combination with actual construction standards; Imperfect early warning mechanism: The early warning mechanism of the existing system is not perfect, lacks intelligent early warning functions based on comprehensive analysis, and cannot flexibly adjust the early warning rules according to the actual situation of equipment operation. Summary of the invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is: a mechanical equipment early warning method, the method comprising the following steps: The data access layer obtains a mechanical equipment data list, wherein the mechanical equipment data list includes data of a plurality of mechanical equipment, and the data of the mechanical equipment is parameter data of the mechanical equipment; Clean the data of mechanical equipment, and standardize the cleaned data according to preset standardization rules to obtain intermediate data; Write the intermediate data to the Kafka platform, and output the intermediate data of the Kafka platform to the real-time processing layer and the data storage layer at the same time; The data storage layer receives and stores the intermediate data; the batch processing layer obtains time series data based on the intermediate data stored in the data storage layer, and performs batch analysis on the time series data. When the batch analysis results meet the preset batch processing alarm rules, an early warning message is generated for early warning. The time series data is the intermediate data within a preset time period. The real-time processing layer receives the intermediate data, obtains metadata of the intermediate data, and distributes the intermediate data based on the metadata of the intermediate data, wherein the metadata of the intermediate data at least includes the type of the intermediate data; Preprocess the intermediate data after diversion according to the preprocessing rules, and obtain the preprocessed intermediate data as the real-time data to be used; Real-time analysis is performed based on the real-time data to be used, and when the real-time analysis results meet the preset real-time alarm rules, warning information is generated for warning.

[0005] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the aforementioned method.

[0006] According to yet another aspect of the present invention, an electronic device is provided, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0007] The present invention has at least the following beneficial effects: In summary, the data access layer obtains a list of mechanical equipment data, cleans the data of the mechanical equipment, and standardizes the cleaned data according to preset standardization rules, obtains intermediate data, writes the intermediate data into the Kafka platform, and outputs the intermediate data of the Kafka platform to the real-time processing layer and the data storage layer at the same time, the data storage layer receives the intermediate data and stores it; the batch processing layer obtains time series data based on the data storage layer, performs batch analysis, and generates warning information for warning when the batch analysis results meet the preset batch processing alarm rules, the real-time processing layer receives the intermediate data, obtains the metadata of the intermediate data, and diverts the intermediate data based on the metadata of the intermediate data, pre-processes the diverted intermediate data according to the pre-processing rules, obtains the pre-processed intermediate data as the real-time data to be used, performs real-time analysis based on the real-time data to be used, and when the real-time analysis results meet the preset real-time alarm rules corresponding to the real-time data to be used, generates warning information for warning, the present invention analyzes the batch processing layer and the real-time processing layer, can predict potential safety risks in the construction process in advance, and issue warnings in time, thereby achieving more comprehensive warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A flowchart of a mechanical equipment early warning method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0011] The embodiment of the present invention provides a mechanical equipment early warning method, such as Figure 1 As shown, the method comprises the following steps: S100, the data access layer obtains a mechanical equipment data list, wherein the mechanical equipment data list includes data of a plurality of mechanical equipment, and the data of the mechanical equipment is parameter data of the mechanical equipment.

[0012] Specifically, the data access layer obtains the list of mechanical equipment data and further includes: the data access layer obtains the list of mechanical equipment data through several data transmission protocols, and the data transmission protocols include the MQTT protocol and the HTTP protocol. MQTT (Message Queue Telemetry Transport Protocol): Suitable for resource-constrained devices, such as smart helmets, sensors and other IoT devices, supports low-bandwidth, high-real-time scenarios, and realizes efficient data transmission between devices and systems; HTTP / HTTPS: Suitable for devices with large data volumes, such as construction site video streams and image data.

[0013] Specifically, the data of mechanical equipment includes IoT device data, audio and video data, and construction machinery construction data. The IoT device data includes: data transmitted by smart safety helmets, temperature sensors, etc.; the audio and video data includes: data transmitted by video cameras and audio sensors from the construction site; the construction machinery construction data includes: real-time operation data of large-scale construction machinery and equipment, such as temperature, pressure, vibration and other data.

[0014] S200, cleaning the data of the mechanical equipment, and standardizing the cleaned data according to preset standardization rules to obtain intermediate data.

[0015] Specifically, cleaning the data of mechanical equipment includes filtering invalid data and abnormal data, wherein the invalid data is the data of mechanical equipment with null values, and the abnormal data is the data of mechanical equipment that does not meet the preset format requirements. For example, abnormal data is such as incorrect date format and non-numeric characters in the numeric field.

[0016] Specifically, the preset standardization rule can be determined according to actual needs; in one embodiment of the present invention, the preset standardization rule is to set the time to a 24-hour system.

[0017] S300 writes the intermediate data into the Kafka platform, and outputs the intermediate data of the Kafka platform to the real-time processing layer and the data storage layer at the same time.

[0018] Specifically, the intermediate data is written to the Kafka platform, which can transmit the intermediate data simultaneously. The real-time processing layer can continuously read the latest data from Kafka for real-time analysis, while the data storage layer can read the real-time data from Kafka for storage, ensuring efficient collaboration between the two. Moreover, Kafka, as a message middleware, allows a large amount of data to be written into the system in a short time without causing congestion in the subsequent processing layer.

[0019] S400, the data storage layer receives and stores the intermediate data; the batch processing layer obtains the time series data based on the intermediate data stored in the data storage layer, and performs batch analysis on the time series data. When the batch analysis results meet the preset batch processing alarm rules, an early warning information is generated for early warning. The time series data is the intermediate data within a preset time period.

[0020] Specifically, after generating the warning information for warning, it also includes: grading and pushing the warning information, wherein the higher the level of the warning information, the higher the priority of the push method. In one embodiment of the present invention, the warning information is divided into three levels, and the third level is the highest level. The third level warning: immediately triggers SMS notification and is highlighted on the real-time large screen. The second level warning: is pushed to the relevant person in charge through the relevant App to remind him to deal with it as soon as possible. The first level warning: is included in the daily, weekly or monthly report for subsequent review and improvement.

[0021] Specifically, the preset time period can be one day, one week, one month, etc.; the setting of the preset time period needs to balance processing efficiency and resource consumption. Different task requirements have different sensitivities to data delays. Setting an appropriate preset time period can reduce unnecessary data backlogs. For example, report data can extract a larger range of data at one time, while real-time monitoring tasks are suitable for batch updates. If the amount of data is large, it is recommended to divide the batch data into multiple smaller batches to avoid a single processing time that is too long.

[0022] S500, the real-time processing layer receives intermediate data, obtains metadata of the intermediate data, and distributes the intermediate data based on the metadata of the intermediate data, wherein the metadata of the intermediate data at least includes the type of the intermediate data.

[0023] S600, preprocessing the diverted intermediate data according to a preprocessing rule, and obtaining the preprocessed intermediate data as real-time data to be used.

[0024] Specifically, through diversion, different preprocessing algorithms are designed for each type of intermediate data to ensure that each type of intermediate data receives the most appropriate preprocessing algorithm. This ensures efficient data processing by the real-time processing layer while reducing latency and ensuring the real-time performance of the system. Different data types may contain different types of noise or errors. For example, temperature sensor data may require preprocessing to remove abnormal peaks, while video data may require preprocessing to remove blank frames.

[0025] S700, performing real-time analysis based on the real-time data to be used, and generating warning information for warning when the real-time analysis result meets the preset real-time alarm rule corresponding to the real-time data to be used.

[0026] Furthermore, real-time analysis based on the real-time data to be used also includes: using Flink's Window operator for calculations based on time windows or data volume windows, such as calculating the average temperature within one minute, or detecting abnormal vibration frequency within five minutes. For complex data pattern recognition and prediction tasks, the real-time processing layer can integrate Flink-based machine learning models, such as fault prediction models, safety behavior recognition models in video streams, etc.

[0027] For example, a high temperature alarm threshold is set for the temperature sensor data. When the real-time analysis result is greater than the high temperature alarm threshold, an early warning message is generated for warning. An abnormal vibration threshold is set for the vibration data of the mechanical equipment. When the real-time analysis result corresponding to the vibration data of the mechanical equipment is greater than the abnormal vibration threshold, an early warning message is generated for warning.

[0028] In summary, the data access layer obtains a list of mechanical equipment data, cleans the data of the mechanical equipment, and standardizes the cleaned data according to preset standardization rules, obtains intermediate data, writes the intermediate data into the Kafka platform, and outputs the intermediate data of the Kafka platform to the real-time processing layer and the data storage layer at the same time. The data storage layer receives the intermediate data and stores it; the batch processing layer obtains time series data based on the data storage layer, performs batch analysis, and generates warning information for warning when the batch analysis results meet the preset batch processing alarm rules. The real-time processing layer receives the intermediate data, obtains the metadata of the intermediate data, and diverts the intermediate data based on the metadata of the intermediate data, pre-processes the diverted intermediate data according to the pre-processing rules, obtains the pre-processed intermediate data as the real-time data to be used, performs real-time analysis based on the real-time data to be used, and generates warning information for warning when the real-time analysis results meet the preset real-time alarm rules corresponding to the real-time data to be used. The present invention analyzes the batch processing layer and the real-time processing layer, can predict potential safety risks in the construction process in advance, and issue warnings in time, thereby achieving more comprehensive warnings.

[0029] Furthermore, when the intermediate data after diversion is preprocessed according to the preprocessing rules, it also includes: Based on the data type and data frequency, the intermediate data after diversion is divided into slices to obtain slice data.

[0030] Specifically, it can be understood that different types of data may have different processing complexities. For example, the data volume of IoT sensors is usually small and the processing time is short, which is suitable for smaller shards; while streaming media data (such as video streams) has a large data volume and complex processing, which is suitable for larger shards; for example, high-frequency data with high real-time requirements can use smaller shards to ensure fast data processing; and low-frequency data can use larger shards to reduce frequent sharding operations; the intermediate data after diversion is sharded based on the data type and data frequency, so that the intermediate data after diversion can be sharded in the most reasonable way, thereby improving processing efficiency.

[0031] In another embodiment of the present invention, the data may be sliced ​​according to time windows, for example, by time windows of seconds, minutes, etc., to ensure that the system receives and processes data evenly within a fixed time interval, thereby avoiding sudden traffic shocks.

[0032] Use the consistent hashing algorithm to map the slice data to the designated node for preprocessing to achieve load balancing.

[0033] In summary, based on the data type and data frequency, the intermediate data after diversion is sharded to obtain the slice data, and the consistent hashing algorithm is used to map the slice data to the specified node for preprocessing. After the intermediate data is diverted, the sharding is continued to ensure that each node has a balanced load and the stability and efficiency of the system are guaranteed. Using the consistent hashing algorithm, even if the number of nodes changes (such as nodes joining or leaving), consistent hashing can effectively reduce the redistribution of data, which is suitable for scenarios where the system has a large amount of data, many nodes, and frequent data access.

[0034] Furthermore, the real-time processing layer and batch processing layer of the present invention use the Flink open source stream processing framework, and use the built-in checkpoint mechanism and state recovery function of Flink to ensure that the system can recover quickly in the event of node failure or data loss, and ensure the continuity and reliability of data processing. Among them, Flink's checkpoint mechanism provides the function of state preservation and recovery, ensuring that processing can be restored from the checkpoint when a failure occurs. The key process of the checkpoint mechanism is as follows: Checkpoints are triggered regularly, and the frequency is configured by the user. The configuration automatically creates checkpoints every certain period of time (such as 10 seconds). Whenever a checkpoint is triggered, Flink will also save snapshots of the state of each operator. These snapshots contain the current processing state of the operator (such as counters, window data, etc.) so that they can be restored in the event of a failure; Flink's state recovery mechanism ensures that the system can continue to process unfinished data after restarting. When restarting after a node failure, Flink will reload the state from the most recent successful checkpoint, including the processing state and data stream position of each operator, to ensure that processing continues from the point of failure. The checkpoint not only contains state data, but also saves the offset or position of each input stream. During recovery, Flink will go back to the location recorded in the checkpoint to continue processing, avoiding data loss or duplicate processing and ensuring the continuity of the data flow.

[0035] Furthermore, the batch processing layer obtains time series data based on the intermediate data stored in the data storage layer, and after batch analysis of the time series data, it also includes: Get batch analysis results. Specifically, the batch analysis results can be used to determine whether there is invalid data or noise data in the intermediate data.

[0036] Based on the batch analysis results, a statistical model is used for processing to obtain abnormal data points. The statistical model includes a mean calculation model and a variance calculation model. Further, the statistical model also includes a Z-score detection model and an ARIMA model. The abnormal data point is a data point that exceeds a preset range.

[0037] Manual analysis is performed based on abnormal data points to obtain the abnormal reasons of the abnormal data points for feedback, for example, the abnormal reasons are data exceeding a threshold, format error, etc.

[0038] Obtain batch analysis results, use statistical models for processing based on the batch analysis results, obtain abnormal data points, perform manual analysis based on the abnormal data points, obtain abnormal causes of abnormal data points for feedback, find abnormal data points through batch analysis results, and analyze the abnormal causes to improve the cleaning process.

[0039] Specifically, the batch processing layer obtains time series data based on the intermediate data stored in the data storage layer and performs batch analysis on the time series data, which also includes: When batch analysis is based on time-stamped data division, the first preset window function is used to divide the time period corresponding to the intermediate data stored in the data storage layer, and the intermediate data in the divided time period is obtained as time series data.

[0040] When batch analysis is grouped based on field intervals, the second preset window function is used to divide the field intervals corresponding to the intermediate data stored in the data storage layer, and the intermediate data in the divided field intervals are used as time series data.

[0041] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0042] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0043] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are only for illustration, not for limiting the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention.

Claims

1. A mechanical equipment early warning method, characterized in that: The method comprises the following steps: The data access layer obtains a mechanical equipment data list, wherein the mechanical equipment data list includes data of a plurality of mechanical equipment, and the data of the mechanical equipment is parameter data of the mechanical equipment; Clean the data of mechanical equipment, and standardize the cleaned data according to preset standardization rules to obtain intermediate data; Write the intermediate data to the Kafka platform, and output the intermediate data of the Kafka platform to the real-time processing layer and the data storage layer at the same time; The data storage layer receives and stores the intermediate data; the batch processing layer obtains time series data based on the intermediate data stored in the data storage layer, and performs batch analysis on the time series data. When the batch analysis results meet the preset batch processing alarm rules, an early warning message is generated for early warning. The time series data is the intermediate data within a preset time period. The real-time processing layer receives the intermediate data, obtains metadata of the intermediate data, and distributes the intermediate data based on the metadata of the intermediate data, wherein the metadata of the intermediate data at least includes the type of the intermediate data; Preprocess the intermediate data after diversion according to the preprocessing rules, and obtain the preprocessed intermediate data as the real-time data to be used; Real-time analysis is performed based on the real-time data to be used, and when the real-time analysis results meet the preset real-time alarm rules, warning information is generated for warning.

2. The mechanical equipment early warning method according to claim 1 is characterized in that: The data access layer obtains the mechanical equipment data list further includes: the data access layer obtains the mechanical equipment data list through several data transmission protocols, and the data transmission protocols include MQTT protocol and HTTP protocol.

3. The mechanical equipment early warning method according to claim 1 is characterized in that: Cleaning the data of mechanical equipment includes filtering invalid data and abnormal data.

4. The mechanical equipment early warning method according to claim 1, characterized in that: After generating the warning information for warning, it also includes: classifying and pushing the warning information, wherein the higher the level of the warning information, the higher the priority of the push method adopted.

5. The mechanical equipment early warning method according to claim 1, characterized in that: When the intermediate data after diversion is preprocessed according to the preprocessing rules, it also includes: Based on the data type and data frequency, the intermediate data after diversion is divided into slices to obtain slice data; Use the consistent hashing algorithm to map the slice data to the specified node for preprocessing.

6. The mechanical equipment early warning method according to claim 1, characterized in that: The batch processing layer obtains time series data based on the intermediate data stored in the data storage layer, and performs batch analysis on the time series data, and also includes: Get batch analysis results; Based on the batch analysis results, use statistical models to process and obtain abnormal data points; Perform manual analysis based on abnormal data points to obtain the abnormal causes of the abnormal data points.

7. The mechanical equipment early warning method according to claim 6, characterized in that: The statistical model includes a mean calculation model and a variance calculation model.

8. The mechanical equipment early warning method according to claim 1, characterized in that: The batch processing layer obtains time series data based on the intermediate data stored in the data storage layer and performs batch analysis on the time series data, including: When the batch analysis is data partitioning based on time tags, the time period corresponding to the intermediate data stored in the data storage layer is partitioned using the first preset window function, and the intermediate data in the partitioned time period is obtained as time series data; When batch analysis is grouped based on field intervals, the second preset window function is used to divide the field intervals corresponding to the intermediate data stored in the data storage layer, and the intermediate data in the divided field intervals are used as time series data.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the mechanical equipment early warning method as described in any one of claims 1-8.

10. An electronic device, characterized in that: The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.

Images