Safety early warning method and device for industrial equipment, electronic equipment and storage medium

Abstract

The invention provides a safety early warning method and device for industrial equipment, electronic equipment and a computer readable storage medium, and relates to the technical field of service intelligent fault diagnosis. The safety early warning method comprises the following steps: acquiring actual multi-modal data of all components of industrial equipment at the current moment; predicting predicted multi-modal data of all the components at the next moment based on the multi-modal collaborative prediction model and the actual multi-modal data of all the components at the current moment; and on the basis of the actual multi-modal data of all the components at the current moment and the predicted multi-modal data of all the components at the next moment, abnormal components of the industrial equipment at the next moment are determined, and early warning is given out. The problems that in the prior art, the intelligent level is low, abnormal early warning cannot be achieved, and the effect of nipping in the bud is at least solved. The method is suitable for equipment fault detection and early warning scenes.

Description

Technical Field

[0001] The present invention relates to the field of intelligent fault diagnosis technology, and in particular to a safety early warning method, device, electronic device and computer-readable storage medium for industrial equipment. Background Art

[0002] Industrial Internet is a key technology for operating the manufacturing industry. The production lines, supply chains, and manufacturing management systems involved are all very important. Once equipment fails, it may cause production interruptions and have a serious impact on factory production. In order to reduce downtime, reduce maintenance costs, improve safety, and ensure the long-term stability and healthy development of the Industrial Internet, we must be committed to achieving continuous innovation in security technology and ensuring all-round security protection for the Industrial Internet.

[0003] At present, the main security guarantee technology for industrial Internet equipment is to monitor and analyze the operating data of the equipment in real time. By monitoring the operating status of the equipment in real time, such as temperature, vibration, and current parameters, once abnormal operating data is found, the system will immediately issue a warning signal to remind relevant personnel to check and deal with it in time.

[0004] However, after the real-time monitoring mechanism in the prior art detects an abnormality and issues an early warning, it relies on the timeliness of manual inspection and fault handling. There is often a long downtime between the early warning and the completion of the fault handling, which has a serious impact on production. Therefore, the real-time monitoring mechanism in the prior art can only reduce the equipment failure rate and improve production efficiency to a certain extent, and the level of intelligence is low, and it is impossible to achieve the effect of preventing abnormal warnings before they occur. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the above-mentioned deficiencies in the prior art and to provide a safety warning method, device, electronic device and computer-readable storage medium for industrial equipment. The method can realize intelligent and rapid safety warning of industrial equipment, effectively improve the efficiency of fault detection, warning and processing, reduce dependence on manual labor, reduce equipment failure rate and improve production efficiency.

[0006] In a first aspect, the present invention provides a safety warning method for industrial equipment, comprising: collecting actual multimodal data of all components of the industrial equipment at a current moment; predicting the predicted multimodal data of all components at the next moment based on a multimodal collaborative prediction model and the actual multimodal data of all components at the current moment; determining the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, and issuing a warning.

[0007] Preferably, before predicting the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment, the safety warning method for industrial equipment also includes: constructing a time series model, a mining model based on a spatiotemporal model, and an adaptive neural network model to obtain a multimodal collaborative prediction model.

[0008] Preferably, the method of predicting the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment specifically includes: inputting the actual multimodal data of all components at the current moment into the time series model of the multimodal collaborative prediction model, the mining model and the adaptive neural network model, respectively, to obtain first predicted multimodal data, second predicted multimodal data and third predicted multimodal data, wherein the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are used to characterize the predicted multimodal data of all components at the next moment outputted by the time series model, the mining model and the adaptive neural network model respectively; respectively calculating the adaptive weights corresponding to the time series model, the mining model and the adaptive neural network model at the current moment; and calculating the predicted multimodal data of all components at the next moment based on the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, as well as the adaptive weights corresponding to the time series model, the mining model and the adaptive neural network model at the current moment.

[0009] Preferably, respectively calculating the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model, specifically comprises: calculating the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model according to formulas (1) and (2):

[0010]

[0011] Among them, y k represents the actual multimodal data of all components at the current time t, represents the predicted multimodal data of all components at the current time t output by the i-th model. The i-th model refers to one of the following: time series model, mining model based on spatiotemporal model, and adaptive neural network model. K represents the total amount of actual multimodal data of all components at the current time. represents the loss of the i-th model at the current time t, Represents the gradient of the i-th model at the current time t. The gradient is used to represent the inverse of the loss. represents the sum of the gradients of all models at the current time t, n represents the total number of models at the current time t, n∈[1,3], α i(t) represents the adaptive weight corresponding to the i-th model at the current time t.

[0012] Preferably, the method of determining the abnormal component of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment specifically includes: calculating the abnormal operation thresholds of all components based on the actual multimodal data of all components at the current moment; judging whether the predicted multimodal data of the mth component at the next moment exceeds the abnormal operation threshold of the mth component, wherein m=1, 2, ..., M, and M represents the total number of components of the industrial equipment; in response to the predicted multimodal data of the mth component at the next moment exceeding the abnormal operation threshold of the mth component, determining that the mth component is the abnormal component of the industrial equipment at the next moment.

[0013] Preferably, after determining the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, the safety warning method for industrial equipment also includes: determining the states of non-abnormal components and the abnormal components of the industrial equipment at the next moment; predicting the states of the non-abnormal components based on the alert state propagation model, the states of the abnormal components and the predicted multimodal data of all components at the next moment; judging whether the states of the non-abnormal components meet the preset alert state; in response to the states of the non-abnormal components meeting the preset alert state, determining that the non-abnormal components are the alert diffusion components of the industrial equipment at the next moment, and adjusting the states of the abnormal components and the alert diffusion components.

[0014] Preferably, predicting the state of the non-abnormal component based on the alert state propagation model, the state of the abnormal component and the predicted multimodal data of all components at the next moment specifically includes: predicting the state of the non-abnormal component according to formula (3):

[0015] Among them, F i (t) represents the state of the i-th non-abnormal component of the industrial equipment at the next time t, F j (t) represents the alert state of the jth abnormal component of the industrial equipment at the next time t, N represents the total number of abnormal components of the industrial equipment, β ij represents the coupling coefficient between the i-th non-abnormal component and the j-th abnormal component, x i (t) represents the predicted multimodal data of the i-th non-abnormal component at the next time t, x j (t) represents the predicted multimodal data of the jth abnormal component at the next time t, and λ represents the diffusion coefficient of the alert state propagation.

[0016] In a second aspect, the present invention also provides a safety warning device for industrial equipment, including an acquisition module, a prediction module and a warning module. The acquisition module is used to acquire the actual multimodal data of all components of the industrial equipment at the current moment. The prediction module is connected to the acquisition module and is used to predict the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment. The warning module is connected to the prediction module and is used to determine the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, and issue a warning.

[0017] In a third aspect, the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the safety warning method for industrial equipment provided in the first aspect.

[0018] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the safety warning method for industrial equipment provided in the first aspect above is implemented.

[0019] The present invention provides a safety warning method, device, electronic device and computer-readable storage medium for industrial equipment. By using the multimodal collaborative prediction model, the multimodal collaborative prediction model can dynamically adjust its own parameters and structure according to the changes in multimodal data, dynamically capture the characteristic relationship of the actual multimodal data of all components at each moment, accurately and quickly predict the predicted multimodal data of all components at the next moment, greatly improve the efficiency, effect, adaptability and reliability of multimodal data prediction, and provide strong data support for subsequent fault discovery, warning and processing, thereby improving the efficiency and reliability of fault discovery, warning and processing. Therefore, the present invention can realize intelligent and rapid safety warning of industrial equipment, improve the efficiency, effect, adaptability and reliability of multimodal data prediction, effectively improve the efficiency of fault discovery, warning and processing, reduce dependence on manual labor, reduce equipment failure rate, and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of a safety early warning method for industrial equipment according to Embodiment 1 of the present invention;

[0021] Figure 2 This is a structural diagram of a 5G network industrial Internet according to Embodiment 1 of the present invention;

[0022] Figure 3 This is a schematic structural diagram of a safety warning device for industrial equipment according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0024] It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention rather than to limit the present invention.

[0025] It can be understood that, in the absence of conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other.

[0026] It can be understood that, for the convenience of description, the drawings of the present invention only show the parts related to the present invention, while the parts irrelevant to the present invention are not shown in the drawings.

[0027] It can be understood that each unit and module involved in the embodiments of the present invention may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0028] It can be understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of the present invention may occur in an order different from that marked in the drawings.

[0029] It is understood that the flowcharts and block diagrams of the present invention illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present invention. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or may be implemented by a combination of hardware and computer instructions.

[0030] It can be understood that the units and modules involved in the embodiments of the present invention can be implemented by software or hardware. For example, the units and modules can be located in a processor.

[0031] Embodiment 1:

[0032] like Figure 1 As shown, this embodiment provides a safety early warning method for industrial equipment.

[0033] In this embodiment, a safety warning method for industrial equipment is implemented by deploying 5G (5th Generation Mobile Communication Technology) or 6G (6th Generation Mobile Communication Technology) network industrial Internet in a smart manufacturing workshop. For example, the 5G network industrial Internet includes but is not limited to: multiple industrial equipment, sensors, 5G private networks, MEC (Mobile Edge Computing) servers, cloud and application systems, among which industrial equipment includes digital control machine tools, automated assembly lines, and mobile robots. Figure 2 In the 5G network industrial Internet shown, industrial equipment includes video surveillance, inspection machines, smart AGVs (Automated Guided Vehicles), production equipment and unmanned forklifts. The 5G private network includes IPLOOK5GC (IPLOOK 5G Core, a core network solution based on 5G technology, designed to provide an efficient network architecture to support the delivery of various 5G applications and services), 5G base stations, data transmission equipment and mobile communication equipment. The cloud includes but is not limited to cross-enterprise industrial cloud platforms and 5G+ smart park cloud platforms. The application systems include but are not limited to: production monitoring and equipment management systems, product inspection systems, drone inspection systems and environmental inspection systems.

[0034] Safety early warning methods for industrial equipment include:

[0035] S101, collecting actual multimodal data of all components of industrial equipment at the current moment.

[0036] In this embodiment, according to the functional characteristics of industrial equipment, the components of industrial equipment can be divided into key components and structural components, wherein key components refer to components that play a decisive role in the operation and performance of industrial equipment, and structural components refer to components that support and protect key components. The components of this embodiment take key components as an example, and the components include but are not limited to: spindle, stator, rotor, motor, engine.

[0037] In this embodiment, sensors are deployed on the key components of each industrial equipment. Through the sensors, actual multimodal data of the key components of the industrial equipment are collected every second or every minute, where the actual multimodal data includes but is not limited to: temperature, pressure, and vibration. The sensors of the industrial equipment are connected to the MEC server through the 5G private network. After the sensors collect the actual multimodal data of the key components of the industrial equipment, the real-time actual multimodal data can be transmitted to the MEC server through the 5G private network for processing. Through the 5G private network, the multimodal data of the key components of the industrial equipment collected by the sensors can be quickly and stably transmitted to the MEC server for processing. For scenarios that require high-frequency collection (such as vibration monitoring and equipment noise detection), the high bandwidth of the 5G private network supports the transmission of multiple data points per second, ensuring high-precision monitoring.

[0038] It should be noted that sensors are closely integrated with 5G base stations. Therefore, 5G base stations are usually deployed in workshops or near industrial equipment to ensure that industrial equipment and sensors are connected through 5G private networks to achieve high bandwidth, low latency and high reliability of data transmission. In addition, 5G private networks provide dedicated network resources to ensure stable and fast data transmission in industrial environments without interference from the public network.

[0039] In this embodiment, sensors can also be deployed directly on each industrial equipment (such as machine tools). The sensors collect the temperature, vibration and rotation speed of several industrial equipment and send them to the MEC server for data analysis and prediction to identify abnormal industrial equipment.

[0040] S102, predicting the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment.

[0041] In this embodiment, after the sensor transmits the actual multimodal data of the key components of the industrial equipment to the MEC server through the 5G private network, the MEC server is responsible for real-time processing and analysis of the actual multimodal data, reducing the time for data transmission to the cloud and improving the response speed, including: building and deploying a multimodal collaborative prediction model on the MEC server, and the MEC server predicts the predicted multimodal data of all components of the industrial equipment at the next moment based on the multimodal collaborative prediction model.

[0042] It should be noted that while deploying a 5G private network in the factory, MEC servers are deployed in the 5G private network. Usually, the UPF (User Plane Function) network element of the 5G private network is sunk close to the user, and then the MEC function is added on the basis of UPF to provide the IT (Information Technology) services and cloud computing functions required by telecom users nearby, thereby creating a telecom-grade service environment with high performance, low latency and high bandwidth, allowing users to enjoy high-quality business experience.

[0043] Similarly, the MEC server predicts the predicted multimodal data of all components of industrial equipment at the next moment based on the multimodal collaborative prediction model. After the MEC server receives the temperature, vibration and speed of several industrial equipment sent by the sensor, it can also predict the predicted multimodal data of several industrial equipment at the next moment based on the multimodal collaborative prediction model. For example: There are 20 CNC machine tools in the intelligent manufacturing workshop for high-load production, and the temperature, vibration and power parameters of the machine tools fluctuate greatly during operation. Traditional monitoring methods cannot accurately predict the alert status of equipment, resulting in frequent shutdowns and maintenance. To this end, a multimodal collaborative prediction model based on adaptive weight adjustment is introduced for equipment status monitoring.

[0044] Optionally, before S102: predicting predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment, the safety early warning method for industrial equipment further includes:

[0045] S104, constructing a time series model, a mining model based on a spatiotemporal model, and an adaptive neural network model to obtain a multimodal collaborative prediction model.

[0046] In this embodiment, multiple models are selected, and multiple models are constructed and trained based on training data to obtain a multimodal collaborative prediction model, wherein the models include but are not limited to: time series models, mining models based on spatiotemporal models, and adaptive neural network models. Time series models include but are not limited to: statistical models based on historical data, and statistical models based on historical data include but are not limited to: linear regression (LR) model, ARIMA (Autoregressive Integrated Moving Average model, differential integrated moving average autoregressive model, also known as integrated moving average autoregressive model). Mining models based on spatiotemporal patterns include but are not limited to: spatiotemporal analysis (spatial-temporal, ST) model. Adaptive neural network models include but are not limited to: LSTM (Long Short Term Memory, long short-term memory) model.

[0047] Time series models are usually good at processing data with time dependence, trend and seasonal characteristics, mining models based on spatiotemporal models are usually good at processing data with both time and space characteristics, and adaptive neural network models are usually good at capturing data with complex nonlinear relationships and long-term dependence characteristics. Therefore, this embodiment constructs a time series model, a mining model based on a spatiotemporal model and an adaptive neural network model to form a multimodal collaborative prediction model, which can reduce the overfitting risk and deviation that may occur in a single model, making the prediction results more stable and reliable, and can cope with data with multiple characteristics, such as data with time, space and nonlinear relationships at the same time. It is suitable for more complex practical application scenarios, and combines the advantages of different prediction models to more comprehensively capture the laws in actual multimodal data and improve the accuracy of overall predictions.

[0048] Specifically, S102: based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment, predicting the predicted multimodal data of all components at the next moment, including steps S1021 to S1023:

[0049] S1021, input the actual multimodal data of all components at the current moment into the time series model, mining model and adaptive neural network model of the multimodal collaborative prediction model respectively, and obtain the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, wherein the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are used to characterize the predicted multimodal data of all components at the next moment outputted by the time series model, the mining model and the adaptive neural network model respectively.

[0050] It should be noted that, as needed, the actual multimodal data can also be selectively input into at least one of the following: time series model, mining model and adaptive neural network model. For example, it is clear that the characteristic relationship of the actual multimodal data only includes time series characteristics, and the time series model has the best performance in processing time series characteristic data. Therefore, the actual multimodal data of all components at the current moment (such as temperature) can be directly input into the time series model for prediction; similarly, if the actual multimodal data is pressure or vibration, it often contains more complex models and features, and the time series features are shorter. Therefore, the actual multimodal data of all components at the current moment can be input into the mining model and the adaptive neural network model for prediction.

[0051] In addition, the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data can also be used to characterize the predicted multimodal data of all components in the next period output by the time series model, the mining model and the adaptive neural network model respectively, and the period includes but is not limited to 10 minutes.

[0052] S1022, respectively calculating the adaptive weights corresponding to the time series model, mining model and adaptive neural network model at the current moment.

[0053] In this embodiment, while selecting multiple models and constructing and training multiple models based on training data to obtain a multimodal collaborative prediction model, this embodiment also configures an adaptive weight adjustment mechanism on the multimodal collaborative prediction model. Based on the adaptive adjustment mechanism of the multimodal collaborative prediction model, the adaptive weights of the time series model, mining model and adaptive neural network model are dynamically adjusted according to their performance at different times, so that the advantages of different models can be effectively integrated, thereby obtaining better performance in the prediction of multimodal data, realizing real-time updating of the multimodal collaborative model, enabling the multimodal collaborative model to quickly respond to changes in new data, and improving the applicability of the multimodal collaborative model in a rapidly changing environment, thereby improving the accuracy of the overall prediction.

[0054] Specifically, S1022: respectively calculating the adaptive weights corresponding to the time series model, mining model and adaptive neural network model at the current moment, including: calculating the adaptive weights corresponding to the time series model, mining model and adaptive neural network model at the current moment according to formulas (1) and (2):

[0055]

[0056] Among them, y k represents the actual multimodal data of all components at the current time t, represents the predicted multimodal data of all components at the current time t output by the i-th model. The i-th model refers to one of the following: time series model, mining model based on spatiotemporal model, and adaptive neural network model. K represents the total amount of actual multimodal data of all components at the current time. represents the loss of the i-th model at the current time t, Represents the gradient of the i-th model at the current time t. The gradient is used to represent the inverse of the loss. represents the sum of the gradients of all models at the current time t, n represents the total number of models at the current time t, n∈[1,3], α i (t) represents the adaptive weight corresponding to the i-th model at the current time t.

[0057] In this embodiment, it is assumed that the predicted multimodal data obtained by the multimodal collaborative model in executing the prediction task at the previous moment is: the predicted temperature P output by the linear regression model T,LR (t) = 72°C, predicted temperature P output by LSTM model T,LSTM (t) = 68°C, predicted temperature P output by the spatiotemporal analysis modelT,ST (t) = 71°C, and the actual multimodal data collected at the current moment is the actual temperature y true =70℃, according to formula (1), the predicted multimodal data obtained by the multimodal collaborative model to perform the prediction task at the previous moment and the actual multimodal data collected at the current moment, the loss value of the time series model, mining model and adaptive neural network model at the current moment is calculated as follows: the loss value of the linear regression model (70-72) 2 =(-2) 2 =4, loss value of LSTM model (70-68) 2 =2 2 =4, loss value of the spatiotemporal analysis model (70-71) 2 =(-1) 2 = 1. The loss values ​​of the time series model, mining model and adaptive neural network model at the current moment are taken as the performance of the time series model, mining model and adaptive neural network model at the current moment.

[0058] The loss value is used to measure the prediction accuracy of the model. The smaller the loss value, the better the prediction effect of the model. Usually, the model with a lower loss value at the current moment obtains a higher weight in the prediction task at the current moment. Therefore, the inverse of the loss value of the time series model, mining model and adaptive neural network model at the current moment is calculated: 0.25, 0.25 and 1, and the gradient of the time series model, mining model and adaptive neural network model at the current moment is obtained. According to formula (2) and the gradient of the time series model, mining model and adaptive neural network model at the current moment, the adaptive weight of the time series model, mining model and adaptive neural network model at the current moment is calculated: the adaptive weight α of the linear regression model LR (t) = 0.25 / (0.25+0.25+1) = 0.25 / 1.5 = 0.1667, adaptive weight α of LSTM model LSTM (t) = 0.25 / 1.5 = 0.1667, adaptive weight α of the spatiotemporal analysis model ST (t) = 1 / 1.5 = 0.6667. This embodiment calculates the loss value of different models of the multimodal collaborative model to perform the prediction task, which is used to adjust the weight of the model's next prediction data, and uses the loss value as a feedback signal, thereby effectively utilizing computing resources and improving the coordination effect of the multimodal collaborative model. When the prediction data of different models are subsequently fused, the prediction data of the model with better performance dominates the final prediction value of the multimodal collaborative model, thereby improving the prediction accuracy and reliability of the multimodal collaborative prediction model.

[0059] It should be noted that formula (1) refers to the mean square error (MSE) calculation formula, which is usually used for regression tasks to measure the gap between the model prediction value and the true value. This embodiment can also use different calculation methods to calculate the loss value for different models (i.e., time series models, mining models, and adaptive neural network models) or prediction tasks at different times, such as: mean absolute error (MAE), root mean square error (RMSE), cross-entropy loss, and Huber loss.

[0060] Assume that the multimodal data only includes temperature, and the total number of models at the current moment n = 1, that is, at the current moment, only the temperature is input into the time series model for prediction, and the predicted temperature of all parts of the industrial equipment in the next 10 minutes is: 75°C. After 10 minutes, the actual temperature collected by the sensor is 77°C. Using the mean square error calculation formula as the loss function, it can be calculated that the loss value of the time series model in the next 10 minutes is (77-75) 2 =4.

[0061] Assume that the multimodal data includes vibration in addition to temperature, and the total number of models at the current moment n=3, input the temperature and vibration into the time series model, mining model and adaptive neural network model for prediction, and obtain the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data for the next 30 seconds. Suppose the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are all 55Hz and 75℃. The actual vibration collected by the sensor in the next 30 seconds is 52Hz and the actual temperature is 77℃. Using the mean square error calculation formula as the loss function, it can be calculated that the loss values ​​of the time series model, mining model and adaptive neural network model in the next 30 seconds are the same, all (77-75) 2 +(55-52) 2 =13.

[0062] S1023, based on the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, and the adaptive weights corresponding to the time series model, the mining model and the adaptive neural network model at the current moment, calculate the predicted multimodal data of all components at the next moment.

[0063] In this embodiment, it is assumed that the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are respectively: the predicted temperature P output by the linear regression model ～T,LR (t) = 80°C, predicted temperature P output by LSTM model～T,LSTM (t) = 82°C, predicted temperature P output by the spatiotemporal analysis model ～T,ST (t) = 79°C, according to the formula The first predicted multimodal data, the second predicted multimodal data, the third predicted multimodal data, and the adaptive weights corresponding to the time series model, the mining model, and the adaptive neural network model at the current moment are used to calculate the predicted multimodal data P of all components at the next moment. final (t) = α LR (t)*P ～T,LR (t)+α LS1M (t)*P ～T,LSTM (t)+α ST (t)*P ～T,ST (t) = 0.1667*80 + 0.1667*82 + 0.6667*79 = 13.36 + 13.67 + 52.67 = 79.7 ° C, where the adaptive weight of the i-th model at the current time t, the i-th model represents one of the time series model, mining model and adaptive neural network model, P i (t) represents the predicted multimodal data of all components at the current time t output by the i-th model. Assuming that the i-th model refers to a time series model, then P i (t) represents the first predicted multimodal data.

[0064] S103, based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, determine the abnormal components of the industrial equipment at the next moment, and issue a warning.

[0065] Specifically, S103: based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, determining the abnormal components of the industrial equipment at the next moment, and issuing an early warning, including steps S1031 to S1033:

[0066] S1031, calculating abnormal operation thresholds of all components based on the actual multimodal data of all components at the current moment.

[0067] In this embodiment, the abnormal operation threshold θ(t) of all components at the current moment is calculated according to the formula θ(t)=μ(t)+k·σ(t) and the actual multimodal data of all components at the current moment, wherein μ(t) represents the mean of the actual multimodal data of all components within the preset period before the current moment, σ(t) represents the standard deviation of the actual multimodal data of all components within the preset period before the current moment, and k represents an adjustment factor, which is used to control the sensitivity to data fluctuations, and k includes one of the following: 1, 2, and 3, depending on the application scenario. This embodiment dynamically adjusts the threshold abnormal operation threshold according to the actual multimodal data of all components within the preset period before the current moment, and can ensure that the abnormal operation threshold is more in line with the actual situation according to the changes in real-time data and historical data, thereby reducing false alarms and missed alarms and improving the sensitivity of abnormal monitoring.

[0068] It should be noted that the preset period may be the frequency of the sensor collecting the actual multimodal data of all components before the current moment. For example, after the sensor collects the actual multimodal data of all components 10 times before the current moment, the actual multimodal data of all components collected 10 times are taken. Assuming that the mean μT(t) of the actual multimodal data of the mth component in the last 10 times is 75.2°C, the standard deviation σT(t) is 1.83°C, and k is 2, which is a relatively high sensitivity and is used to detect abnormalities early. The abnormal operation threshold θ of the mth component is calculated. m (t)=75.2+2*1.83=78.86℃.

[0069] S1032, determining whether the predicted multimodal data of the mth component at the next moment exceeds the abnormal operation threshold of the mth component, where m=1, 2, ..., M, and M represents the total number of components of the industrial equipment.

[0070] S1033: In response to the predicted multimodal data of the mth component at the next moment exceeding the abnormal operation threshold of the mth component, determine that the mth component is an abnormal component of the industrial equipment at the next moment.

[0071] In this embodiment, it is assumed that the predicted multimodal data P of the mth component at the next moment is m (t) = 79.7°C, exceeding the abnormal operation threshold θ of the mth component m (t) = 78.86°C, therefore, the mth component is determined to be an abnormal component, where P final (t) = {P m (t)}, θ(t)={θ m (t)}.

[0072] Optionally, after determining the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, the safety early warning method for the industrial equipment further includes:

[0073] S105, determining the states of the non-abnormal components and the abnormal components of the industrial equipment at the next moment.

[0074] In this embodiment, after the MEC server predicts the predicted multimodal data of all components of the industrial equipment at the next moment based on the multimodal collaborative prediction model, the MEC server also determines the status of all components of the industrial equipment at the next moment based on the predicted multimodal data of all components of the industrial equipment at the next moment, where the status includes but is not limited to: alert status; judging whether there are components of the industrial equipment in the alert state at the next moment; and issuing an early warning in response to components in the alert state at the next moment. m (t) exceeds the abnormal operation threshold θ m (t), the MEC server determines that the mth component is in an alert state, and at the same time determines that the components that do not exceed their respective abnormal operation thresholds are non-abnormal components.

[0075] For example, in an industrial motor system, the rotor and stator are key components. The predicted temperature x of the motor rotor output by the multi-modal collaborative model rotor (t) will reach 90°C, and the predicted stator temperature x output by the multi-modal collaborative model stator (t) = 70°C, and the abnormal operation thresholds of the motor rotor and stator are both 70°C. At this time, the state F rotor (r0) is recorded as 1 (indicating alert state).

[0076] S106 , predicting the state of the non-abnormal component based on the alert state propagation model, the state of the abnormal component and the predicted multimodal data of all components at the next moment.

[0077] In this embodiment, an alert status propagation model is also constructed and deployed on the MEC server, and the MEC server predicts the status of non-abnormal components based on the alert status propagation model.

[0078] Specifically, S106: predicting the state of the non-abnormal component based on the alert state propagation model, the state of the abnormal component and the predicted multimodal data of all components at the next moment, including: predicting the state of the non-abnormal component according to formula (3):

[0079] Among them, F i (t) represents the state of the oth non-abnormal component of the industrial equipment at the next time t, F j(t) represents the alert state of the jth abnormal component of the industrial equipment at the next time t, N represents the total number of abnormal components of the industrial equipment, β ij represents the coupling coefficient between the i-th non-abnormal component and the j-th abnormal component, x i (t) represents the predicted multimodal data of the i-th non-abnormal component at the next time t, x j (t) represents the predicted multimodal data of the jth abnormal component at the next time t, and λ represents the diffusion coefficient of the alert state propagation.

[0080] In this embodiment, an exponential decay function is used to simulate the gradual weakening of the influence between the various components of the industrial equipment, determine the mutual influence relationship between the various components of the industrial equipment, and generate an alert state propagation model such as formula (3). According to formula (3), it can be seen that the propagation speed of the alert state is related to the multimodal data difference and coupling strength between the components. The various components of industrial equipment usually have physical association relationships and data association relationships, where the physical association relationships include but are not limited to: physical connection, heat conduction, and mechanical movement. Different physical association relationships correspond to different coupling strengths, β ij It reflects the coupling strength between components. The stronger the physical connection or energy transfer path between components, the greater the coupling coefficient. For example, in an electric motor, the coupling between the rotor and the stator is stronger. The difference in multimodal data between components reflects the data association relationship. In addition, the speed of alert state propagation is also related to the diffusion coefficient λ of alert state propagation, which is used to describe the inherent fault propagation characteristics of industrial equipment. In practical applications, the diffusion coefficient λ is determined by experimental measurement or modeling of the physical properties of industrial equipment, rather than artificial setting. A larger λ indicates a faster propagation speed of the impact between components, and a smaller λ indicates a slower propagation. By using the alert state propagation model with real-time response capabilities, continuously obtaining the latest multimodal data of industrial equipment and simulating alert state propagation, ensuring the accuracy of alert state propagation prediction, early warnings can be issued in time to inform the situation where the possible alert state affects non-abnormal components, thereby providing a reference for equipment maintenance and repair, ensuring timely human intervention, and avoiding more serious equipment damage.

[0081] It is predicted that the motor rotor will be in an alert state, which may gradually affect the stator and other related components. Therefore, an alert state propagation model is needed to predict the impact of the motor rotor's alert state on the stator.

[0082] Since the rotor and stator are coupled through magnetic field and have a compact physical structure, the coupling coefficient β is set rotor,stator =0.8. Setting the diffusion coefficient λ = 10 indicates that the alert state spreads quickly. According to formula (3), the state of the stator is predicted

[0083] S107, determining whether the state of the non-abnormal component satisfies a preset warning state.

[0084] S108, in response to the state of the non-abnormal component satisfying the preset warning state, determining the non-abnormal component as the warning diffusion component of the industrial equipment at the next moment, and adjusting the states of the abnormal component and the warning diffusion component.

[0085] In this embodiment, F stator (t) = 0.1082 means that the rotor warning state begins to spread and affect the stator, but due to the difference in their states (20°C), the impact is attenuated and the current impact on the stator is relatively light (only 10.82% of the fault intensity). You can set the value of the preset warning state of the stator yourself. For example, when F stator (t) exceeds 0.15, the stator is determined to be a warning diffusion component. If the state of the rotor continues to deteriorate (the temperature rises further), or the temperature of the stator approaches the temperature of the rotor, the spread of the impact will accelerate and the abnormal intensity of the stator will further increase. As time goes by, if the predicted temperature of the rotor output by the subsequent multimodal collaborative model rises to 100°C, it still exceeds the abnormal operation threshold and the state of the stator needs to be re-predicted. The predicted temperature of the stator is 85°C, and the state of the stator has been raised to F stator (t) = 0.1785, the warning state of the rotor is spreading, the stator is more affected, and the preset warning state is met, which may cause problems in the stator in the short term, requiring attention and manual intervention.

[0086] It should be noted that after determining that the stator meets the preset warning state, the state of the non-abnormal components is also predicted through the stator, that is, F in formula (3) j (t) is 0.1785, not 1. The preset alert state usually corresponds to a non-abnormal component, that is, the preset alert state of each non-abnormal component is different.

[0087] In response to the state of the non-abnormal component satisfying the preset alert state, determining the non-abnormal component as the alert diffusion component of the industrial equipment at the next moment, and adjusting the states of the abnormal component and the alert diffusion component, this embodiment also realizes the collaborative work of the MEC server and the cloud through the 5G private network, specifically including: the MEC server uploads the actual multimodal data of all components at the current moment, the predicted multimodal data of all components at the next moment, the abnormal component information, the non-abnormal component information and the alert diffusion component information to the cloud; the cloud performs long-term data analysis and global optimization of model parameters, and sends the optimization results to the MEC server through the 5G private network, wherein the model parameters include model parameters of the multimodal collaborative prediction model and the alert state propagation model.

[0088] The present embodiment provides a safety warning method for industrial equipment. By making use of the ability of a multimodal collaborative prediction model to dynamically adjust its own parameters and structure according to changes in multimodal data, the method dynamically captures the characteristic relationship of the actual multimodal data of all components at each moment, and accurately and quickly predicts the predicted multimodal data of all components at the next moment, thereby greatly improving the efficiency, effect, adaptability and reliability of multimodal data prediction, providing strong data support for subsequent fault discovery, warning and processing, thereby improving the efficiency and reliability of fault discovery, warning and processing, realizing intelligent and rapid safety warning of industrial equipment, improving the efficiency, effect, adaptability and reliability of multimodal data prediction, effectively improving the efficiency of fault discovery, warning and processing, reducing dependence on manual labor, reducing equipment failure rate, and improving production efficiency.

[0089] Embodiment 2:

[0090] like Figure 3 As shown, this embodiment provides a safety warning device for industrial equipment, including an acquisition module 31, a prediction module 32 and a warning module 33. The acquisition module 31 is used to collect the actual multimodal data of all components of the industrial equipment at the current moment. The prediction module 32 is connected to the acquisition module 31 and is used to predict the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment. The warning module 33 is connected to the prediction module 32 and is used to determine the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, and issue a warning.

[0091] Optionally, the safety warning device for industrial equipment further includes: a construction module 34, which is used to construct a time series model, a mining model based on a spatiotemporal model, and an adaptive neural network model to obtain a multimodal collaborative prediction model.

[0092] Specifically, the prediction module 32 includes: a prediction unit 321, a first calculation unit 322 and a second calculation unit 323, the prediction unit 321 is used to input the actual multimodal data of all components at the current moment into the time series model, the mining model and the adaptive neural network model of the multimodal collaborative prediction model respectively, to obtain the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, wherein the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are used to characterize the predicted multimodal data of all components at the next moment output by the time series model, the mining model and the adaptive neural network model respectively, the first calculation unit 322 is used to calculate the adaptive weights corresponding to the time series model, the mining model and the adaptive neural network model at the current moment respectively, and the second calculation unit 323 is used to calculate the predicted multimodal data of all components at the next moment based on the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, as well as the adaptive weights corresponding to the time series model, the mining model and the adaptive neural network model at the current moment.

[0093] Specifically, the first calculation unit 322 includes: a calculation subunit, which is used to calculate the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model according to formulas (1) and (2):

[0094]

[0095] Among them, y k represents the actual multimodal data of all components at the current time t, represents the predicted multimodal data of all components at the current time t output by the i-th model. The i-th model refers to one of the following: time series model, mining model based on spatiotemporal model, and adaptive neural network model. K represents the total amount of actual multimodal data of all components at the current time. represents the loss of the i-th model at the current time t, Represents the gradient of the i-th model at the current time t. The gradient is used to represent the inverse of the loss. represents the sum of the gradients of all models at the current time t, n represents the total number of models at the current time t, n∈[1,3], α i (t) represents the adaptive weight corresponding to the i-th model at the current time t.

[0096] Specifically, the early warning module 33 includes: a third calculation unit 331, a judgment unit 332 and a determination unit 333. The third calculation unit 331 is used to calculate the abnormal operation thresholds of all components based on the actual multimodal data of all components at the current moment. The judgment unit 332 is used to judge whether the predicted multimodal data of the mth component at the next moment exceeds the abnormal operation threshold of the mth component, wherein m=1, 2,…, M, and M represents the total number of components of the industrial equipment. The determination unit 333 is used to determine that the mth component is an abnormal component of the industrial equipment at the next moment in response to the predicted multimodal data of the mth component at the next moment exceeding the abnormal operation threshold of the mth component.

[0097] Optionally, the safety warning device for industrial equipment also includes: a first determination module 35, a prediction module 36, a judgment module 37 and a second determination module 38, the first determination module 35 is used to determine the states of the non-abnormal components and the abnormal components of the industrial equipment at the next moment, the prediction module 36 is used to predict the states of the non-abnormal components based on the alert state propagation model, the states of the abnormal components and the predicted multimodal data of all components at the next moment, the judgment module 37 is used to judge whether the states of the non-abnormal components meet the preset alert state, and the second determination module 38 is used to determine that the non-abnormal components are the alert diffusion components of the industrial equipment at the next moment in response to the states of the non-abnormal components meeting the preset alert state, and adjust the states of the abnormal components and the alert diffusion components.

[0098] Specifically, the prediction module 36 includes: a fourth calculation unit 361, which is used to predict the state of the non-abnormal component according to formula (3):

[0099] Among them, F i (t) represents the state of the i-th non-abnormal component of the industrial equipment at the next time t, F j (t) represents the alert state of the jth abnormal component of the industrial equipment at the next time t, N represents the total number of abnormal components of the industrial equipment, β ij represents the coupling coefficient between the i-th non-abnormal component and the j-th abnormal component, x i (t) represents the predicted multimodal data of the i-th non-abnormal component at the next time t, x j (t) represents the predicted multimodal data of the jth abnormal component at the next time t, and λ represents the diffusion coefficient of the alert state propagation.

[0100] It can be understood that the safety warning device for industrial equipment provided above executes the safety warning method for industrial equipment corresponding to the embodiment 1 provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the scheme corresponding to the safety warning method for industrial equipment in the embodiment 1 above, and will not be repeated here.

[0101] Embodiment 3:

[0102] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the safety warning method for industrial equipment in the above-mentioned embodiment 1.

[0103] Embodiment 4:

[0104] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the safety early warning method for industrial equipment in the above-mentioned embodiment 1 is implemented.

[0105] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A safety early warning method for industrial equipment, characterized in that: include: Collect the actual multimodal data of all components of industrial equipment at the current moment; Based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment, predict the predicted multimodal data of all components at the next moment; Based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, the abnormal components of the industrial equipment at the next moment are determined and an early warning is issued.

2. The safety early warning method for industrial equipment according to claim 1, characterized in that: Before predicting the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment, the method further includes: A time series model, a mining model based on a spatiotemporal model and an adaptive neural network model are constructed to obtain a multimodal collaborative prediction model.

3. The safety early warning method for industrial equipment according to claim 2, characterized in that: The method of predicting the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment specifically includes: Inputting the actual multimodal data of all components at the current moment into the time series model of the multimodal collaborative prediction model, the mining model and the adaptive neural network model respectively, to obtain the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, wherein the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data are used to characterize the predicted multimodal data of all components at the next moment respectively output by the time series model, the mining model and the adaptive neural network model; Respectively calculating the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model; Based on the first predicted multimodal data, the second predicted multimodal data and the third predicted multimodal data, as well as the time series model at the current moment, the mining model and the adaptive weights corresponding to the adaptive neural network model, the predicted multimodal data of all components at the next moment are calculated.

4. The safety early warning method for industrial equipment according to claim 3, characterized in that: The step of respectively calculating the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model specifically includes: According to formulas (1) and (2), the adaptive weights corresponding to the time series model at the current moment, the mining model and the adaptive neural network model are calculated: Among them, y k represents the actual multimodal data of all components at the current time t, represents the predicted multimodal data of all components at the current time t output by the i-th model. The i-th model refers to one of the following: time series model, mining model based on spatiotemporal model, and adaptive neural network model. K represents the total amount of actual multimodal data of all components at the current time. represents the loss of the i-th model at the current time t, Represents the gradient of the i-th model at the current time t. The gradient is used to represent the inverse of the loss. represents the sum of the gradients of all models at the current time t, n represents the total number of models at the current time t, n∈[1,3], α i (t) represents the adaptive weight corresponding to the i-th model at the current time t.

5. The safety early warning method for industrial equipment according to claim 1, characterized in that: The method of determining the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment specifically includes: Based on the actual multimodal data of all components at the current moment, calculate the abnormal operation thresholds of all components; Determine whether the predicted multimodal data of the mth component at the next moment exceeds the abnormal operation threshold of the mth component, where m=1, 2, …, M, and M represents the total number of components of the industrial equipment; In response to the predicted multimodal data of the mth component at the next moment exceeding the abnormal operation threshold of the mth component, the mth component is determined to be an abnormal component of the industrial equipment at the next moment.

6. The safety early warning method for industrial equipment according to claim 1, characterized in that: After determining the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, the method further includes: Determine the status of the non-abnormal component and the abnormal component of the industrial equipment at the next moment; Predicting the state of the non-abnormal component based on the alert state propagation model, the state of the abnormal component and the predicted multimodal data of all components at the next moment; Determining whether the state of the non-abnormal component meets the preset warning state; In response to the state of the non-abnormal component satisfying a preset warning state, the non-abnormal component is determined to be a warning diffusion component of the industrial equipment at a next moment, and the states of the abnormal component and the warning diffusion component are adjusted.

7. The safety early warning method for industrial equipment according to claim 6, characterized in that: The predicting the state of the non-abnormal component based on the alert state propagation model, the state of the abnormal component and the predicted multimodal data of all components at the next moment specifically includes: According to formula (3), the state of the non-abnormal component is predicted: Among them, F i (t) represents the state of the i-th non-abnormal component of the industrial equipment at the next time t, F j (t) represents the alert state of the jth abnormal component of the industrial equipment at the next time t, N represents the total number of abnormal components of the industrial equipment, β ij represents the coupling coefficient between the i-th non-abnormal component and the j-th abnormal component, x i (t) represents the predicted multimodal data of the i-th non-abnormal component at the next time t, x j (t) represents the predicted multimodal data of the jth abnormal component at the next time t, and λ represents the diffusion coefficient of the alert state propagation.

8. A safety warning device for industrial equipment, characterized in that: Including collection module, prediction module and early warning module, The acquisition module is used to collect the actual multimodal data of all components of industrial equipment at the current moment. The prediction module is connected to the acquisition module and is used to predict the predicted multimodal data of all components at the next moment based on the multimodal collaborative prediction model and the actual multimodal data of all components at the current moment. The early warning module is connected to the prediction module and is used to determine the abnormal components of the industrial equipment at the next moment based on the actual multimodal data of all components at the current moment and the predicted multimodal data of all components at the next moment, and issue an early warning.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement a safety early warning method for industrial equipment according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the safety early warning method for industrial equipment according to any one of claims 1 to 7 is implemented.

Images