Smart factory real-time safety monitoring method and system based on edge computing

Abstract

The invention relates to the technical field of digital factories, in particular to a smart factory real-time safety monitoring method and system based on edge computing. Firstly, an edge computing device, a camera and a sensor group are deployed in each production area; then, acquiring image data through a camera; acquiring sensor data through the sensor group; and finally, constructing a safety monitoring model, and operating the safety monitoring model on the edge computing device. Inputting image data and sensor data into the safety monitoring model, and obtaining a safety monitoring result through a data preprocessing layer, a feature analysis layer, a feature fusion layer and a monitoring result prediction layer in sequence; the image data and the sensor data are finely analyzed through the safety equipment wearing feature analysis network and the dangerous behavior feature analysis network, the finally obtained safety monitoring result can accurately reflect the real-time safety state of the smart factory, and efficient and dynamic safety management is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of digital factories, and specifically to a real-time safety monitoring method and system for smart factories based on edge computing. Background Art

[0002] A smart factory is an advanced manufacturing system that uses digitalization and automation technologies to integrate various production resources. This factory architecture achieves highly optimized and intelligent management of the production process by integrating technologies such as information technology, artificial intelligence, the Internet of Things (IoT), and machine learning. Smart factories can not only improve production efficiency and product quality, but also achieve effective use of resources and minimize energy consumption, thereby achieving the goal of sustainable development.

[0003] Although smart factories have significantly improved their automation and intelligence levels, the safety risks in the production process have also increased. For example, equipment failure, operational errors, system errors, etc. may lead to serious safety accidents. Therefore, it is particularly important to implement real-time safety monitoring, which can timely detect and respond to these potential risks, effectively prevent accidents, protect factory assets and employee safety, and ensure the smooth operation of the production process.

[0004] Edge computing plays a key role in the safety monitoring of smart factories. By processing data at the location where the data is generated (i.e., the factory site), edge computing technology can significantly reduce the delay in data transmission and achieve almost real-time data processing and response. This approach not only speeds up decision-making, but also improves the reliability and efficiency of the system, especially when the network connection is unstable or the amount of data is extremely large. In addition, edge computing can enhance data security because sensitive data can be analyzed and stored locally without long-distance transmission.

[0005] Edge computing is a computing architecture whose core idea is to transfer data processing tasks from the cloud or central data center to the edge of the network, that is, close to the source of data generation. In the application of real-time monitoring of smart factories, edge computing can process data instantly at the data generation location, significantly reducing the delay in transmission to the cloud. This enables the factory management system to respond quickly to various situations, such as equipment failure warnings, safety hazard detection, etc., thereby improving production efficiency and safety. In addition, edge computing can effectively reduce bandwidth requirements and improve data security, because most sensitive data is processed locally and does not need to be transmitted over long distances to the cloud. The application of this technology makes real-time monitoring of smart factories more efficient and reliable.

[0006] Before the widespread use of edge computing technology, traditional factory safety monitoring technology mainly relied on centralized data processing systems. This system usually aggregates data collected by sensors and cameras throughout the factory to a central server for processing and analysis. Although this method can process large amounts of data, it has the problem of slow data processing and response speed, especially when the amount of data is large or needs to be processed across geographical locations. In addition, the stability and reliability of the centralized system are also constrained by the quality of the network. Once there is a problem with the central server or network, the efficiency and effectiveness of the entire monitoring system will be affected.

[0007] In the prior art, there are still certain defects in the real-time safety monitoring of smart factories. For example, the Chinese patent with publication number CN115008059B proposes a digital factory production safety intelligent monitoring and early warning system based on intelligent manufacturing. By setting the safe welding area of ​​the welding robot, analyzing the welding quality and safety, and issuing danger warnings at each welding time point, it realizes a comprehensive analysis of the safety of the welding process from multiple dimensions, and then obtains the welding quality coefficient and welding safety factor. Although this method effectively solves the limitations of the analysis results and reduces the risks of managers, it relies on data quality and has limited ability to respond to emergencies. The Chinese patent with publication number CN117273470B proposes a vehicle cargo fixer processing factory production safety intelligent monitoring and early warning system. To this end, a real-time safety monitoring method and system for smart factories based on edge computing are proposed. Through a detailed and comprehensive analysis of the mechanical production operation safety index, electrical production supply safety index, and workshop production environment safety index of each target workshop, the comprehensive production safety index of each target workshop is evaluated to screen out each abnormal production workshop and issue a production risk warning for it.

[0008] Existing technologies in the field of smart factory safety monitoring usually face data processing delays, high network load for centralized processing, insufficient preprocessing, and lack of targeted and dynamically adaptive analysis mechanisms, which limit their effectiveness in real-time and accurate safety monitoring.

[0009] To this end, a real-time safety monitoring method and system for smart factories based on edge computing are proposed. Summary of the invention

[0010] The purpose of the present invention is to provide a real-time safety monitoring method and system for smart factories based on edge computing. First, edge computing devices, cameras and sensor groups are deployed in each production area; then, image data is obtained through the camera; sensor data is obtained through the sensor group; finally, a safety monitoring model is constructed, and the safety monitoring model runs on the edge computing device; the image data and sensor data are input into the safety monitoring model, and the safety monitoring results are obtained through the data preprocessing layer, feature analysis layer, feature fusion layer and monitoring result prediction layer in sequence; the present invention uses the safety equipment wearing feature analysis network and the dangerous behavior feature analysis network to perform a fine analysis of the image data and sensor data, and the final safety monitoring results can accurately reflect the real-time safety status of the smart factory, ensuring efficient and dynamic safety management.

[0011] To achieve the above object, the present invention provides the following technical solutions:

[0012] A real-time safety monitoring method for a smart factory based on edge computing, comprising:

[0013] Deploy edge computing devices, cameras, and sensor groups in each production area; the sensor groups include sound sensors, light sensors, and temperature sensors;

[0014] Acquire image data and sensor data; the image data is acquired by the camera; the sensor data is acquired by the sensor group; the sensor data includes noise data, brightness data and temperature data;

[0015] Constructing a safety monitoring model, which runs on the edge computing device; inputting the image data and the sensor data into the safety monitoring model, and sequentially passing through a data preprocessing layer, a feature analysis layer, a feature fusion layer, and a monitoring result prediction layer to obtain a safety monitoring result; the safety monitoring result is used to reflect the safety status of the factory;

[0016] The data preprocessing layer is used to perform a preprocessing operation on the image data and the sensor data to obtain a first input data group;

[0017] The feature analysis layer is used to perform feature analysis on the first input data group to obtain a safety device wearing feature vector and a dangerous behavior feature vector;

[0018] The feature fusion layer is used to fuse the safety equipment wearing feature vector and the dangerous behavior feature vector to obtain a comprehensive safety analysis feature vector;

[0019] The monitoring result prediction layer is used to analyze the comprehensive safety analysis feature vector, obtain the probability values ​​of the three result labels of safety, warning and danger respectively, and finally obtain the safety monitoring result according to the size of the probability value.

[0020] The specific implementation process of the preprocessing operation includes:

[0021] Using image recognition technology to identify the image data, marking the areas where the safety helmet, protective glasses, safety gloves and earplugs are worn respectively;

[0022] Using the YOLO algorithm to identify the mechanical equipment and the body parts of the employee closest to the mechanical equipment in the image data, marking the mechanical equipment, the key body parts of the employee and the production stages corresponding to the mechanical equipment; the production stages include welding, assembly and chemical treatment;

[0023] Performing outlier removal, data smoothing and standardization processing on the brightness data and the temperature data;

[0024] Performing noise processing on the noise data to obtain a noise feature group;

[0025] The image data and the sensor data are synchronized and spliced ​​according to the timestamps to obtain a first input data group.

[0026] The specific process of performing noise processing on the noise data includes:

[0027] Selective denoising: Use a bandpass filter to selectively denoise the noise data to obtain denoised noise data; the bandpass filter H(f) is defined as: Among them, f L ,f H They are the lower and upper frequency limits of the filter respectively;

[0028] Volume normalization: Calculate the average sound pressure level of each sound sample in the denoised noise data and adjust it to the target sound pressure level to obtain the normalized noise data. The specific calculation formula is:

[0029]

[0030] Among them, s nor (t) is the standardized noise data, that is, the sound signal after volume normalization, s(t) is the original sound signal at time t in the denoised noise data, L tar is the target sound pressure level, L mean is the average sound pressure level of the original sound signal;

[0031] Feature extraction: extracting frequency features in the standardized noise by fast Fourier transform, including energy features, sound intensity features and frequency component features;

[0032] The calculation formula of the energy characteristic E is:

[0033]

[0034] Where s1(t1) is the amplitude of the sound signal at time t1, and N is the total number of samples;

[0035] The calculation formula of the sound intensity characteristic I is:

[0036] I=10log 10 (E);

[0037] The calculation formula of the frequency component characteristics is:

[0038]

[0039] Among them, X(k) is the converted frequency component feature at frequency k, x(ns) is the amplitude of the sound signal at time ns, N1 is the total number of samples, and i is the imaginary unit.

[0040] The feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on the first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

[0041] The safety device wearing feature analysis network includes a dynamic weight adjustment mechanism; the dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group input to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is:

[0042] w env =σ(W env ·y env +b env );

[0043] Among them, w env is the environmental factor weight, σ is the Sigmoid activation function, W env is the weight matrix used to transform the sensor data y env Converted to environmental factor weight, benv is the bias term.

[0044] The key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is:

[0045]

[0046] Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; n is the number of risk features;

[0047] The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is:

[0048]

[0049] Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

[0050] A real-time safety monitoring system for smart factories based on edge computing, including:

[0051] An equipment deployment unit, used to deploy edge computing equipment, cameras and sensor groups in each production area; the sensor group includes a sound sensor, a light sensor and a temperature sensor;

[0052] A data acquisition unit, used to acquire image data and sensor data; the image data is acquired by the camera; the sensor data is acquired by the sensor group; the sensor data includes noise data, brightness data and temperature data;

[0053] The security monitoring unit includes a security monitoring model, which runs on the edge computing device and is used to analyze the image data and the sensor data to obtain a security monitoring result.

[0054] The safety monitoring model includes a data preprocessing layer, a feature analysis layer, a feature fusion layer and a monitoring result prediction layer; the feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on a first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

[0055] The safety device wearing feature analysis network includes a dynamic weight adjustment mechanism; the dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group input to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is:

[0056] w env =σ(W env ·y env +b env );

[0057] Among them, w env is the environmental factor weight, σ is the Sigmoid activation function, W env is the weight matrix used to transform the sensor data y env Converted to environmental factor weight, b env is the bias term.

[0058] The key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is:

[0059]

[0060] Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; nn is the number of risk features;

[0061] The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is:

[0062]

[0063] Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. The present invention can realize the localization of data processing by deploying edge computing devices in each production area to run the safety monitoring model, significantly reducing data transmission delay and network load, and enhancing the real-time response capability of the system. In addition, the preprocessing of image data and sensor data is performed on the edge device, which can effectively filter and optimize the original data and improve the accuracy and efficiency of subsequent analysis. In particular, the selective denoising and feature extraction of noise data can significantly improve the data quality, thereby enhancing the monitoring system's ability to detect potential safety risks and ensuring that the safety monitoring of the factory environment is more accurate and reliable.

[0066] 2. In the feature analysis layer of the safety monitoring model, the present invention not only uses the LSTM network structure to capture the dynamic changes of time series data, but also introduces an innovative dynamic weight adjustment mechanism. This mechanism can adjust the input weight of the data according to real-time environmental factors, including brightness, temperature and noise, thereby optimizing the model's recognition accuracy of safety equipment. This dynamic adjustment not only improves the adaptability of the model, but also ensures that the use of safety equipment by staff can be accurately monitored in various operating environments, significantly improving the effectiveness of safety management.

[0067] 3. In the feature analysis layer of the safety monitoring model, the dangerous behavior feature analysis network significantly improves the effectiveness of the monitoring system by combining the LSTM network structure with the key parts attention mechanism. This network structure uses LSTM to process complex time series data to capture behavioral patterns, while the key parts attention mechanism focuses on key parts of the employee's body, including hands and head, and prioritizes the analysis of the dynamics of these parts during dangerous operations. This dual analysis strategy enables the system to more accurately identify and warn of potential high-risk behaviors, so that measures can be taken in advance to avoid accidents, enhancing the safety protection capabilities of the entire workplace. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1A flow chart of a real-time safety monitoring method for a smart factory based on edge computing provided by an embodiment of the present invention;

[0069] Figure 2 A model framework diagram of a security monitoring model provided by an embodiment of the present invention;

[0070] Figure 3 A structural diagram of a smart factory real-time safety monitoring system based on edge computing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In order to accurately monitor the real-time safety of smart factories, the present invention proposes a real-time safety monitoring method and system for smart factories based on edge computing.

[0073] Embodiment 1

[0074] This embodiment performs safety monitoring and analysis on smart factory A;

[0075] In this embodiment, the specific implementation process is implemented by a smart factory real-time safety monitoring method based on edge computing, see Figure 1 The process description of the method proposed in the present invention is as follows; the real-time safety monitoring method of a smart factory based on edge computing includes:

[0076] S10. Deploy edge computing devices, cameras and sensor groups in each production area; the sensor group includes a sound sensor, a light sensor and a temperature sensor;

[0077] S20. Acquire image data and sensor data; image data is acquired through a camera; sensor data is acquired through a sensor group;

[0078] S30. Construct a security monitoring model, which runs on the edge computing device; input image data and sensor data into the security monitoring model, and obtain security monitoring results through the data preprocessing layer, feature analysis layer, feature fusion layer and monitoring result prediction layer in sequence. Figure 2 Shown is a model framework diagram of the safety monitoring model;

[0079] The sensor data includes noise data, brightness data and temperature data. In this embodiment, some of the collected sensor data are shown in Table 1:

[0080] Table 1: Some sensor data examples

[0081] Timestamp Noise level (dB) Brightness (lux) Temperature(℃) 2023 / 7 / 18:00 82 507 22 2023 / 7 / 18:02 82 496 23 2023 / 7 / 18:04 146 511 21 2023 / 7 / 18:06 156 510 22 2023 / 7 / 18:08 186 508 24 ... ... ... ...

[0082] Further, the data preprocessing layer is used to perform a preprocessing operation on the image data and the sensor data to obtain a first input data group;

[0083] The feature analysis layer is used to perform feature analysis on the first input data group to obtain a safety device wearing feature vector and a dangerous behavior feature vector;

[0084] The feature fusion layer is used to fuse the safety equipment wearing feature vector and the dangerous behavior feature vector to obtain a comprehensive safety analysis feature vector;

[0085] The monitoring result prediction layer is used to analyze the comprehensive safety analysis feature vector, obtain the probability values ​​of the three result labels of safety, warning and danger respectively, and finally obtain the safety monitoring result according to the size of the probability value.

[0086] By deploying edge computing devices, cameras, and sensor groups in each production area, the safety monitoring model of the present invention can process images and sensor data directly on the device. This layout reduces data transmission delays and reduces dependence on central servers. The safety monitoring model processes data in turn through the data preprocessing layer, feature analysis layer, feature fusion layer, and monitoring result prediction layer, effectively improving the data processing speed and the real-time nature of safety monitoring, enabling the system to quickly respond to potential risks and implement necessary safety measures, greatly improving the efficiency and accuracy of factory safety management.

[0087] Furthermore, the specific implementation process of the preprocessing operation includes:

[0088] Using image recognition technology to identify the image data, marking the areas where the safety helmet, protective glasses, safety gloves and earplugs are worn respectively;

[0089] Using the YOLO algorithm to identify the mechanical equipment and the body parts of the employee closest to the mechanical equipment in the image data, marking the mechanical equipment, the key body parts of the employee and the production stages corresponding to the mechanical equipment; the production stages include welding, assembly and chemical treatment;

[0090] The brightness data and the temperature data are processed by removing outliers, smoothing data, and standardizing data; the noise data is processed to obtain a noise feature group; the image data and the sensor data are synchronized and spliced ​​according to timestamps to obtain a first input data group.

[0091] In this embodiment, the image data is deeply analyzed by combining image recognition technology and YOLO algorithm, which can accurately mark the wearing of employees' safety equipment including helmets, protective glasses, safety gloves and earplugs, and identify key body parts adjacent to mechanical equipment. In addition, by removing outliers, smoothing and standardizing brightness and temperature data, processing noise for noise data, and synchronizing and integrating all data into the first input data group, the system can provide a more accurate and comprehensive safety status assessment, thereby significantly improving the efficiency and effectiveness of production safety management.

[0092] Furthermore, the specific process of performing noise processing on the noise data includes:

[0093] Selective denoising: Use a bandpass filter to selectively denoise the noise data to obtain denoised noise data; the bandpass filter H(f) is defined as: Among them, f L ,f H They are the lower and upper frequency limits of the filter respectively;

[0094] Volume normalization: Calculate the average sound pressure level of each sound sample in the denoised noise data and adjust it to the target sound pressure level to obtain the normalized noise data. The specific calculation formula is:

[0095]

[0096] Among them, s nor (t) is the standardized noise data, that is, the sound signal after volume normalization, s(t) is the original sound signal at time t in the denoised noise data, L tar is the target sound pressure level, L mean is the average sound pressure level of the original sound signal;

[0097] Among them, the average sound pressure level L of the original sound signal is calculated mean It is a key step in the standardization process, which involves the following specific calculation steps:

[0098] Convert sound signal to sound pressure level: Convert the physical amplitude of the sound signal to sound pressure level, sound pressure level L p The calculation formula is:

[0099]

[0100] Where p is the instantaneous sound pressure at the measurement point, p ref As the reference sound pressure, 20 micropascals is taken in this embodiment;

[0101] Calculate the average sound pressure level of the sound signal, the average sound pressure level L meanCalculated by the following formula:

[0102]

[0103] Where N is the total number of samples, L p [n] is the sound pressure level of the nth sample;

[0104] Feature extraction: extracting frequency features in the standardized noise by fast Fourier transform, including energy features, sound intensity features and frequency component features;

[0105] The calculation formula of the energy characteristic E is:

[0106]

[0107] Where s1(t1) is the amplitude of the sound signal at time t1, and N is the total number of samples;

[0108] The calculation formula of the sound intensity characteristic I is:

[0109] I=10log 10 (E);

[0110] The calculation formula of the frequency component characteristics is:

[0111]

[0112] Where X(k) is the converted frequency component feature at frequency k, x(ns) is the amplitude of the sound signal at time ns, N1 is the total number of samples, and i is an imaginary unit;

[0113] In this embodiment, first, a bandpass filter is used for selective denoising to accurately filter out noise within the non-target frequency range and retain key sound signals, which helps to reduce misjudgment and improve the effectiveness of signal processing. Next, the volume normalization process adjusts all sound samples to the same sound pressure level to ensure the consistency of subsequent analysis and the accuracy of comparison. In addition, the frequency features extracted by fast Fourier transform, such as energy, sound intensity, and frequency components, provide the monitoring system with more detailed and in-depth acoustic features, allowing the system to more effectively identify potential safety risks, thereby achieving more accurate monitoring of noise pollution and mechanical failures in the factory environment. These steps improve the response speed and early warning accuracy of the safety monitoring system as a whole, and provide strong technical support for maintaining factory safety.

[0114] Furthermore, the feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on the first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

[0115] The feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network, both of which use LSTM network structures to capture and analyze the complex changes in time series data. The safety equipment wearing feature analysis network uses the LSTM processing flow to accurately identify whether workers are correctly wearing necessary safety equipment, such as helmets and protective glasses, and generate safety equipment wearing feature vectors. In addition, the dangerous behavior feature analysis network combines the attention mechanism of key parts, prioritizes the analysis of the dynamics of key parts including hands and head, more effectively identifies potential dangerous behaviors, and generates dangerous behavior feature vectors. This method of combining advanced sequence models and attention mechanisms significantly improves the prediction accuracy and response speed of the monitoring system, providing strong data support and decision-making basis for factory safety management.

[0116] Furthermore, the safety device wearing feature analysis network includes a dynamic weight adjustment mechanism; this mechanism dynamically adjusts the network weights in the LSTM network structure according to environmental variables including brightness, temperature and noise, as well as the complexity of the operating task. This method can help the model more accurately identify the wearing status of safety equipment in complex or changing environments.

[0117] The dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is:

[0118] w env =σ(W env ·y env +b env );

[0119] Among them, w env is the environmental factor weight, which is an output processed by the Sigmoid activation function and is used to adjust the weight of the data input to the LSTM layer. It reflects the impact of the current environmental conditions on the identification of security devices. σ is the Sigmoid activation function, which is used to convert the result of the linear combination into a value between 0 and 1 so that the output can be interpreted as a weight; W envis the weight matrix used to transform the sensor data y env Converted to environmental factor weight; y env is the sensor data, including brightness, temperature and noise; b env It is a bias term used in the linear transformation of the weight matrix to increase the flexibility and nonlinear ability of the model.

[0120] The input data input to the LSTM network structure after the dynamic weight adjustment mechanism is in the form of:

[0121] y adjusted =W env ·y tt ;

[0122] Among them, y adjusted is the adjusted input data, which is the original data y tt and the environmental weight w env This adjustment reflects the actual impact of environmental conditions on the observed data.

[0123] The hidden state update formula of the LSTM layer is:

[0124]

[0125] in, The LSTM hidden state at time step tt is used to capture and convey information in time series data; is the LSTM hidden state of the previous time step, which is used to transmit state information in the time series;

[0126] These three formulas together form a safety equipment wear feature analysis network that dynamically adapts to environmental variables. By adjusting the weight of the input data, the model can more effectively handle data variations under different environmental conditions, thereby improving the accuracy and reliability of safety monitoring.

[0127] The safety equipment wear feature analysis network introduces a dynamic weight adjustment mechanism that can dynamically adjust the weights of the LSTM network according to environmental variables, including brightness, temperature, and noise. Environmental factor weights are calculated by analyzing sensor data, and these weights are used to adjust the weights of data entering the LSTM layer after being processed by the activation function, so that the model can more accurately identify the wearing status of safety equipment under various environmental conditions. This environment-sensitive dynamic adjustment method not only improves the adaptability of the model, but also significantly enhances the accuracy and reliability of safety monitoring, ensuring that potential safety risks can be effectively identified in complex or changing working environments.

[0128] Furthermore, the key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is:

[0129]

[0130] Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; nn is the number of risk features;

[0131] In this embodiment, the risk characteristic function f is constructed ii The specific process of (ss,pp,td) includes:

[0132] Collect relevant data, including: accident history data, recording accidents and near-incidents involved in different production stages and operations, with special attention to injuries related to specific body parts; operational behavior data, collecting behavioral data during operations, such as employees' movement patterns, frequency, and interactions with operating equipment; environmental monitoring data, continuously collecting environmental parameters including brightness, temperature, and noise.

[0133] Perform feature engineering on the collected data, including: extracting action recognition features from video surveillance in the raw data and extracting environmental anomalies from sensor data; creating risk-related features, based on existing safety knowledge and accident data, creating features that can reflect the level of risk, including the position and speed of the hand during specific machine operations;

[0134] Regression analysis is used to analyze the relationship between different characteristics and the probability of accident occurrence, f ii The specific form of (ss,pp,td) is:

[0135]

[0136] Among them, e1, e2, e3, …, e v are the features extracted from the data, e1,e2,e3,…,e are obtained through data training. v The respective weights.

[0137] The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is:

[0138]

[0139] Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

[0140] The key part attention mechanism introduced in the dangerous behavior feature analysis network significantly enhances the accuracy and efficiency of safety monitoring through the risk assessment function. The risk assessment function combines the risk characteristics of production stage, body part and time, which are obtained based on historical data analysis and calculated in real time through the conversion function of sensor and operation data. Through this mechanism, the model can dynamically calculate and assign attention weights to various key parts and production stages, giving priority to those activity areas with high risks. This not only improves the response speed of the early warning system, but also makes safety measures more targeted, thereby effectively preventing accidents and ensuring workplace safety.

[0141] Embodiment 2

[0142] This embodiment performs safety monitoring and analysis on B smart factory;

[0143] As an embodiment of the present invention, refer to Figure 3 ,A smart factory real-time safety monitoring system based on edge computing includes: an equipment deployment unit, a data acquisition unit, and a safety monitoring unit;

[0144] The device deployment unit is used to deploy edge computing devices, cameras and sensor groups in each production area; the sensor group includes a sound sensor, a light sensor and a temperature sensor;

[0145] The data acquisition unit is used to acquire image data and sensor data; the image data is acquired by the camera; the sensor data is acquired by the sensor group; the sensor data includes noise data, brightness data and temperature data; in this embodiment, some of the collected sensor data are shown in Table 2:

[0146] Table 2 Example of some sensor data collected

[0147] Timestamp Noise level (dB) Brightness (lux) Temperature(℃) 2023 / 7 / 514:42 124 653 29 2023 / 7 / 514:44 126 642 29 2023 / 7 / 514:46 118 678 29 2023 / 7 / 514:48 96 686 28 2023 / 7 / 514:50 102 657 28 ... ... ... ...

[0148] The security monitoring unit includes a security monitoring model, which runs on the edge computing device and is used to analyze the image data and the sensor data to obtain a security monitoring result.

[0149] Furthermore, the safety monitoring model includes a data preprocessing layer, a feature analysis layer, a feature fusion layer and a monitoring result prediction layer; the feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on a first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

[0150] Furthermore, the safety device wear feature analysis network includes a dynamic weight adjustment mechanism; the dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group input to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is:

[0151] w env =σ(W env ·y env +b env );

[0152] Among them, w env is the environmental factor weight, σ is the Sigmoid activation function, W env is the weight matrix used to transform the sensor data y env Converted to environmental factor weight, b env is the bias term.

[0153] Furthermore, the key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is:

[0154]

[0155] Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; nn is the number of risk features;

[0156] The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is:

[0157]

[0158] Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

[0159] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time safety monitoring method for smart factories based on edge computing, characterized in that: include: Deploy edge computing devices, cameras, and sensor clusters in every production area; The sensor group includes a sound sensor, a light sensor and a temperature sensor; Acquire image data and sensor data; The image data is acquired by the camera; the sensor data is acquired by the sensor group; the sensor data includes noise data, brightness data and temperature data; Constructing a safety monitoring model, which runs on the edge computing device; inputting the image data and the sensor data into the safety monitoring model, and sequentially passing through a data preprocessing layer, a feature analysis layer, a feature fusion layer, and a monitoring result prediction layer to obtain a safety monitoring result; the safety monitoring result is used to reflect the safety status of the factory; The data preprocessing layer is used to perform a preprocessing operation on the image data and the sensor data to obtain a first input data group; The feature analysis layer is used to perform feature analysis on the first input data group to obtain a safety device wearing feature vector and a dangerous behavior feature vector; The feature fusion layer is used to fuse the safety equipment wearing feature vector and the dangerous behavior feature vector to obtain a comprehensive safety analysis feature vector; The monitoring result prediction layer is used to analyze the comprehensive safety analysis feature vector, obtain the probability values ​​of the three result labels of safety, warning and danger respectively, and finally obtain the safety monitoring result according to the size of the probability value.

2. According to claim 1, a smart factory real-time safety monitoring method based on edge computing is characterized in that: The specific implementation process of the preprocessing operation includes: Using image recognition technology to identify the image data, marking the areas where the safety helmet, protective glasses, safety gloves and earplugs are worn respectively; Using the YOLO algorithm to identify the mechanical equipment and the body parts of the employee closest to the mechanical equipment in the image data, marking the mechanical equipment, the key body parts of the employee and the production stages corresponding to the mechanical equipment; the production stages include welding, assembly and chemical treatment; Performing outlier removal, data smoothing and standardization processing on the brightness data and the temperature data; Performing noise processing on the noise data to obtain a noise feature group; The image data and the sensor data are synchronized and spliced ​​according to the timestamps to obtain a first input data group.

3. According to claim 2, a smart factory real-time safety monitoring method based on edge computing is characterized in that: The specific process of performing noise processing on the noise data includes: Selective denoising: Use a bandpass filter to selectively denoise the noise data to obtain denoised noise data; the bandpass filter H(f) is defined as: Among them, f L ,f H They are the lower and upper frequency limits of the filter respectively; Volume normalization: Calculate the average sound pressure level of each sound sample in the denoised noise data and adjust it to the target sound pressure level to obtain the normalized noise data. The specific calculation formula is: Among them, s nor (t) is the standardized noise data, that is, the sound signal after volume normalization, s(t) is the original sound signal at time t in the denoised noise data, L tar is the target sound pressure level, L mean is the average sound pressure level of the original sound signal; Feature extraction: extracting frequency features in the standardized noise by fast Fourier transform, including energy features, sound intensity features and frequency component features; The calculation formula of the energy characteristic E is: Where s1(t1) is the amplitude of the sound signal at time t1, and N is the total number of samples; The calculation formula of the sound intensity characteristic I is: I=10log 10 (E); The calculation formula of the frequency component characteristics is: Among them, X(k) is the converted frequency component feature at frequency k, x(ns) is the amplitude of the sound signal at time ns, N1 is the total number of samples, and i is the imaginary unit.

4. According to the method of real-time safety monitoring of smart factories based on edge computing in claim 1, it is characterized in that: The feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on the first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

5. A method for real-time safety monitoring of smart factories based on edge computing according to claim 4, characterized in that: The safety device wearing feature analysis network includes a dynamic weight adjustment mechanism; the dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group input to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is: w env =σ(W env ·y env +b env ); Among them, w env is the environmental factor weight, σ is the Sigmoid activation function, W env is the weight matrix used to transform the sensor data y env Converted to environmental factor weight, b env is the bias term.

6. A method for real-time safety monitoring of smart factories based on edge computing according to claim 4, characterized in that: The key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is: Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; nn is the number of risk features; The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is: Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

7. A real-time safety monitoring system for smart factories based on edge computing, characterized in that: include: Equipment deployment unit, used to deploy edge computing equipment, cameras and sensor groups in each production area; The sensor group includes a sound sensor, a light sensor and a temperature sensor; A data acquisition unit, used to acquire image data and sensor data; the image data is acquired by the camera; the sensor data is acquired by the sensor group; the sensor data includes noise data, brightness data and temperature data; The security monitoring unit includes a security monitoring model, which runs on the edge computing device and is used to analyze the image data and the sensor data to obtain a security monitoring result.

8. The real-time safety monitoring system for smart factories based on edge computing according to claim 7 is characterized in that: The safety monitoring model includes a data preprocessing layer, a feature analysis layer, a feature fusion layer and a monitoring result prediction layer; the feature analysis layer includes a safety equipment wearing feature analysis network and a dangerous behavior feature analysis network; the safety equipment wearing feature analysis network is an LSTM network structure; the dangerous behavior feature analysis network is an LSTM network structure and a key part attention mechanism; the safety equipment wearing feature analysis network is used to perform a first feature analysis on a first input data group to obtain a safety equipment wearing feature vector; the dangerous behavior feature analysis network is used to perform a second feature analysis on the first input data group to obtain a dangerous behavior feature vector.

9. The real-time safety monitoring system for smart factories based on edge computing according to claim 8 is characterized in that: The safety device wearing feature analysis network includes a dynamic weight adjustment mechanism; the dynamic weight adjustment mechanism is used to analyze the sensor data in the first input data group input to obtain the environmental factor weight; the environmental factor weight is used to adjust the data weight input to the LSTM network structure; the calculation formula of the environmental factor weight is: w env =σ(W env ·y env +b env ); Among them, w env is the environmental factor weight, σ is the Sigmoid activation function, W env is the weight matrix used to transform the sensor data y env Converted to environmental factor weight, b env is the bias term.

10. The real-time safety monitoring system for smart factories based on edge computing according to claim 8 is characterized in that: The key part attention mechanism includes a risk assessment function, and the formula of the risk assessment function R(ss, pp, td) is: Among them, f ii (ss,pp,td) is the risk characteristic function of production stage ss, body part pp and time td, which is obtained by analyzing historical data, env td is the sensor data at time td, op td represents the operation data at time td, i.e., the shortest distance between the mechanical equipment and the key body parts of the employee, g(env td ) is the conversion function of the sensor data; he(op td ) is the conversion function of the operating data, β ii ,γ,δ are model parameters; n is the number of risk features; The formula for calculating the dynamic attention weight at each time point using the risk assessment function R(ss,pp,td) is: Among them, α ss,pp (td) is the dynamic attention weight of a specific production stage ss and a specific body part pp at time td; T is the total time steps contained in the analyzed time series data; k is an iteration variable used to traverse each time point in T during the summation process.

Images