Industrial scene personnel state abnormity detection and early warning system and method based on TinyML

Abstract

The invention discloses a TinyML-based industrial scene personnel state abnormity detection and early warning system and method, and the system employs M5Stack StacC Plus2 as an edge calculation terminal, and collects the movement and physiological data of a worker; the method comprises the steps of data acquisition, feature extraction, time sequence analysis and multi-level early warning decision making, and real-time monitoring and abnormal early warning of the personnel state are achieved. The feature extraction adopts a mode of combining PCA dimensionality reduction and ANN to extract features; the time sequence analysis is carried out based on an improved bidirectional attention LSTM network; and the multi-stage early warning decision realizes a three-stage early warning mechanism according to the attitude abnormal degree and physiological index comprehensive evaluation. According to the invention, lightweight edge calculation deployment is realized, a multi-level early warning mechanism suitable for the industrial scene is set, the practicability is high, the safety guarantee level of personnel in the industrial scene is effectively improved, and the risk of safety accidents is reduced.

Description

Technical Field

[0001] The present invention relates to the field of safety monitoring technology, and in particular to a TinyML-based industrial scene personnel status abnormality detection and early warning system and method. Background Art

[0002] In the field of industrial production, especially in places like chemical plants with complex production environments and high safety risks, ensuring the safety of workers is of vital importance. There are many dangerous factors in chemical plants, such as high temperature, high pressure, and leakage of toxic and harmful gases, which may cause workers to experience physical discomfort, falls, poisoning and other abnormal conditions during the operation process, which not only threatens the life and health of workers, but also causes immeasurable damage to production equipment, production progress and the surrounding environment.

[0003] At present, chemical plants have begun to pay attention to the status monitoring of personnel; traditional monitoring methods mainly rely on manual inspections and some simple sensor alarm devices. The manual inspection method has obvious disadvantages. On the one hand, it is inefficient and difficult to achieve real-time and continuous monitoring of many workers; on the other hand, manual judgment is highly subjective, prone to misjudgment or missed judgment, and unable to detect abnormal conditions of personnel in a timely and accurate manner. Although the existing simple sensor alarm device can monitor and alarm certain specific environmental parameters (such as excessive concentration of toxic gases, etc.), it is incapable of detecting abnormal physical conditions of individual workers.

[0004] In addition, some advanced personnel monitoring technologies based on computer vision can achieve a certain degree of analysis of personnel posture and behavior, but in the harsh environment of chemical plants such as complex lighting conditions, occlusion, dust and smoke, their monitoring accuracy and reliability are greatly reduced, and the false alarm rate is high, making it difficult to actually apply them to the daily production safety monitoring of chemical plants. At the same time, these traditional monitoring technologies often need to transmit a large amount of data to a remote central server for processing and analysis, which not only places high demands on network bandwidth, but also the delay problem in the data transmission process seriously affects the timeliness of early warning, and cannot meet the actual needs of chemical plants for real-time monitoring and rapid response of personnel safety.

[0005] In summary, the existing personnel status monitoring technology has problems such as incomplete, inaccurate, untimely monitoring and poor environmental adaptability in the application scenarios of chemical plants. There is an urgent need for a technical solution that can overcome these defects and realize accurate, real-time and reliable monitoring of the status of chemical plant personnel and abnormal warning. Summary of the invention

[0006] For the above technical problems, this technical solution provides an industrial scenario personnel status anomaly detection and early warning system and method based on TinyML, and proposes a multi-level algorithm architecture of PCA dimensionality reduction + ANN feature extraction + bidirectional attention LSTM. Among them, PCA dimensionality reduction retains 95% of the variance information to ensure the integrity of data features. ANN uses a three-layer (64-32-16) structure for feature extraction. The bidirectional attention LSTM enhances the recognition ability of long-term stationary states through a residual connection mechanism. This architecture can also operate stably on resource-constrained devices such as M5Stack StickC Plus2, effectively solving the above problems.

[0007] The present invention is realized through the following technical solutions:

[0008] An industrial scenario personnel status anomaly detection and early warning method based on TinyML, comprising the steps of:

[0009] Step 1: Data collection: Collect three-axis acceleration data A(t) = [a x (t), a y (t), a z (t)] and three-axis angular velocity data G(t) = [g x (t), g y (t), g z (t)] through a portable data collection terminal, collect heart rate data H(t) through a heart rate sensor, and collect body temperature data T(t) using a temperature sensor; transmit the collected data to an edge computing processing unit for processing;

[0010] Step 2: Data preprocessing: Input the collected original data into the edge computing processing unit for denoising processing;

[0011] Step 3: Feature extraction: Process the preprocessed data, and the specific operation method is:

[0012] Step 3.1: Perform PCA dimensionality reduction on the original data. The formula for PCA dimensionality reduction is:

[0013] X PCA = XW PCA ,

[0014] In the above formula, X is the original data matrix, W PCA is the PCA transformation matrix, and X PCA is the data matrix after PCA dimensionality reduction;

[0015] Step 3.2: Use ANN for feature extraction:

[0016] Use the pre-trained ANN feature extraction network to extract features from the data after PCA dimensionality reduction. The network adopts a three-layer neural network structure, including an input layer - hidden layer 1 - hidden layer 2 - output layer; the number of nodes in the input layer is determined according to the dimension of the data after PCA dimensionality reduction, the number of nodes in hidden layer 1 is 64, the number of nodes in hidden layer 2 is 32, the number of nodes in the output layer is 16, and the ReLU function is used as the activation function:

[0017] h l =f(W l h l-1 +b l )

[0018] In the above formula, h l is the output of the l-th hidden layer, W l is the weight matrix of the l-th layer, b l is the bias vector of the l-th layer, and f is the ReLU activation function;

[0019] Step 4: Temporal analysis: Use the improved bidirectional attention LSTM network to perform temporal modeling on the data after feature extraction; through temporal analysis, the system obtains the change characteristics of the personnel status data in the time dimension, and the change characteristics will be used as an important basis for multi-level early warning decision-making;

[0020] Step 5: Comprehensive evaluation: The system comprehensively evaluates the personnel status by using multi-level early warning decision-making according to the preset early warning conditions, combining the posture characteristics and physiological indicators, so as to achieve accurate abnormal early warning;

[0021] The multi-level early warning decision-making includes:

[0022] First-level early warning condition: Slight fall is detected and physiological indicators are normal

[0023] Score 1 =w 11 F pose +w 12 F physio <θ 1 ;

[0024] Second-level early warning condition: Severe fall is detected or physiological indicators are abnormal

[0025] Score 2 =w 21 F pose +w 22 F physio <θ 2 ;

[0026] Third-level early warning condition: Fall is detected and the person is stationary for a long time or physiological crisis

[0027] Score 3= w 31 F pose + w 32 F physio <θ 3 ;

[0028] In the above formula, w 11 , w 12 are the first-level warning weight coefficients, w 21 , w 22 are the second-level warning weight coefficients, w 31 , w 32 are the third-level warning weight coefficients, F pose is the score of the abnormal posture degree, which is calculated from the posture features after normalization; F physio is the score of the abnormal physiological index degree, which is calculated from the physiological indexes after normalization; θ 1 , θ 2 , θ 3 are the first-level, second-level, and third-level warning thresholds respectively.

[0029] Furthermore, the portable data acquisition terminal adopts the LSM6DSO six-axis inertial sensor built in M5Stack StickC Plus2; the heart rate sensor adopts the heart rate sensor of model MAX30100; the temperature sensor adopts the temperature sensor of model DS18B20.

[0030] Furthermore, the acquisition frequency of the three-axis acceleration data and the three-axis angular velocity data is 50 Hz, and the acquisition frequency of the heart rate data and the body temperature data is 1 Hz.

[0031] Furthermore, the data preprocessing in step 2 is specifically operated as follows:

[0032] Step 2.1: Input the collected original data into the edge computing processing unit, and the edge computing processing unit segments the collected original data by a sliding window with a window size of 2 seconds and an overlap rate of 50%;

[0033] Step 2.2: Denoise the signal by wavelet transform, select the Daubechies wavelet basis function, and remove the noise components according to the soft threshold rule. The soft threshold is calculated based on the wavelet coefficients of the signal and the estimated value of the noise standard deviation;

[0034] Step 2.3: Perform Z-score standardization, calculate the mean and standard deviation of the data within the window, and standardize the data to a distribution with a mean of 0 and a standard deviation of 1.

[0035] Further, the PCA dimensionality reduction of the original data described in step 3.1 is performed based on the PCA dimensionality reduction sub-module; the PCA dimensionality reduction sub-module performs eigenvalue decomposition by calculating the data covariance matrix, and the expression of the data covariance matrix is:

[0036]

[0037] where n is the number of samples, x i is the i-th sample data, μ is the sample mean, and T represents the transpose of the matrix; the eigenvalue decomposition of the covariance matrix

[0038] ∑=UΛU T ,

[0039] U is the eigenvector matrix, and its column vectors are the eigenvectors of the covariance matrix ∑; Λ is a diagonal matrix, and the diagonal elements are the corresponding eigenvalues; U T is the transpose matrix of U.

[0040] Select the eigenvectors with a cumulative contribution rate reaching 95%:

[0041]

[0042] In the above formula, k is the number of selected eigenvectors, and λ i is the i-th eigenvalue;

[0043] And perform projection transformation to achieve data dimensionality reduction; the expression of the projection transformation is:

[0044] X PCA =XU k ,

[0045] where X is the original data, and U k is the matrix composed of the selected eigenvectors.

[0046] Further, for the feature extraction using ANN described in step 3.2, the optimization objective of the ANN feature extraction network is:

[0047]

[0048] where θ is the network parameter, λ is the regularization coefficient, set to 0.001, N is the total number of training samples, f θ is the neural network model function, x i is the i-th input sample, and y i is the true label value corresponding to the i-th sample;

[0049] Use the Adam optimizer for training, and its update steps include:

[0050] m t =β1 m t-1 +(1 - β 1 )g t

[0051]

[0052] where m t is the first - order momentum estimate at the current moment, v t is the second - order momentum estimate at the current moment, g t is the gradient, β 1 and β 2 are hyperparameters of the Adam optimizer, set to 0.9 and 0.999 respectively; is the first - order momentum estimate after bias correction, is the second - order momentum estimate after bias correction; θ t is the parameter update value at the current moment; α is the learning rate, set to 0.001; ∈ is a small constant to prevent division by zero, set to 1e - 8;

[0053] Through the above optimization process, the ANN feature extraction network converges to the optimal parameters, enhancing the feature extraction ability.

[0054] Furthermore, the improved bidirectional attention LSTM network in step four performs temporal modeling on the data after feature extraction. The temporal modeling includes:

[0055] The forward LSTM computational unit satisfies the condition:

[0056]

[0057] where x t is the input data, is the forward hidden state at the previous moment, which includes internally:

[0058] Input gate: i t = σ(W ii x t + W hi h t-1 + b i ),

[0059] Forget gate: f t = σ(W if x t + W hf h t-1 + b f ),

[0060] Candidate memory content: g t = tanh(W ig x t + Whg h t-1 + b g ),

[0061] Memory cell state: c t = f t ⊙ c t-1 + i t ⊙ g t ,

[0062] Output gate: o t = σ(W io x t + W ho h t-1 + b o ),

[0063] Hidden state: h t = o t ⊙ tanh(c t ),

[0064] where W ii , W if , W ig , W io , W hi , W hf , W hg , W ho are the corresponding weight matrices, b i , b f , b g , b o are the bias vectors, and σ is the sigmoid function;

[0065] The backward LSTM computational unit satisfies the condition:

[0066]

[0067] The computational process of the backward LSTM computational unit is the same as that of the forward LSTM, but the data input order is reversed; The attention mechanism computational unit calculates the attention weights, and the calculation formula is:

[0068] α t = softmax(W a [h t ; c t + b a )

[0069] where h t is the hidden state at the current moment, and the memory cell state at time t is:

[0070]

[0071] Among them, S is the sequence length, corresponding to the sliding window size, and W a is the weight matrix, and b a is the bias vector, which focuses on key timing information through the attention mechanism;

[0072] The residual connection structure of the bidirectional attention LSTM network is defined as:

[0073]

[0074] Among them, l represents the index of the LSTM layer, l ∈ {1, 2, …, L}, represents the hidden state of the l-th layer at time t, represents the hidden state of the (l - 1)-th layer at time t;

[0075] Attention weight calculation:

[0076] Calculate the attention score:

[0077] Normalize to obtain the attention weight:

[0078] Calculate the context vector: z t = ∑ s α t,s h s ;

[0079] Among them, v a ∈ R d is the attention vector parameter, used to calculate the attention score; U a ∈ R d×d is the attention weight matrix, used to encode the hidden state of the input sequence; h s ∈ R d represents the hidden state at time step s, s represents the time step of the source sequence; z t ∈ R d represents the context vector at time t; d is the hidden state dimension; α t,s represents the attention weight of time t to time step s, is the transpose of the attention vector parameter, h t-1 is the hidden state of the previous moment, e t,s is the attention score of time t to time step s, a scalar value. s′ is the time step index in the sequence, h s is the hidden state of the source sequence at time s.

[0080] An industrial scenario personnel status anomaly detection and early warning system based on TinyML is applied to the above-mentioned industrial scenario personnel status anomaly detection and early warning method based on TinyML. The anomaly detection and early warning system includes:

[0081] Portable data acquisition terminal: The terminal includes an inertial measurement unit and a main controller. The inertial measurement unit and the main controller are connected through an I2C interface. The terminal is also provided with an interface for expanding external sensors;

[0082] Edge computing processing unit: Connected to the portable data acquisition terminal; The edge computing processing unit includes: a data preprocessing module, a feature extraction module, and a timing analysis module; The data preprocessing module performs preprocessing operations on the collected raw data; The feature extraction module extracts features from the preprocessed data; The timing analysis module performs timing analysis on the data after feature extraction;

[0083] Model compression unit: Includes a weight quantization sub-module, a knowledge distillation sub-module, and a channel pruning sub-module;

[0084] Inference acceleration unit: Includes an operator fusion sub-module and a memory optimization sub-module;

[0085] Cloud monitoring platform: Communicates with the edge computing processing unit using an encrypted wireless communication protocol, has an automatic reconnection mechanism, and meets the latency and reliability requirements for real-time monitoring; The platform includes a data storage module, a deep analysis module, a warning management module, and a visualization display module, which are used to receive, store, analyze the data uploaded by the edge computing processing unit, and perform warning management and visualization display.

[0086] Furthermore, the model compression unit includes:

[0087] Weight quantization sub-module: Compresses 32-bit floating-point numbers into 8-bit fixed-point numbers;

[0088]

[0089] where, w q is the quantized 8-bit fixed-point number weight value, w is the original 32-bit floating-point number weight value, w min and w max are the minimum and maximum values of the weights respectively. By weight quantization, the model storage space and computational amount are reduced. After weight quantization, the model storage space is reduced by 75%, and the computational amount is reduced by 60%;

[0090] Knowledge distillation sub-module: Minimizes the KL divergence between the teacher model and the student model;

[0091]

[0092] where, L KD is the knowledge distillation loss function value, α is the balance coefficient, set to 0.5; T temp is the temperature parameter, set to 2.0; z t and zs They are the outputs of the teacher model and the student model respectively. CE is the cross-entropy loss function, and y s is the predicted value of the student model, and y true is the true label. Through knowledge distillation, the student model can streamline the model structure while maintaining high accuracy. After knowledge distillation, the accuracy of the student model has an error of within ±2% compared with the teacher model, and at the same time, the number of model parameters is reduced by 70%.

[0093] Channel pruning sub-module: Channel pruning of the model is performed based on the importance score of the L1 norm. The formula for the score is:

[0094]

[0095] In the above formula, score c is the importance score of channel c, and |W c | 1 is the L1 norm of the weight tensor. i, j, k are the three-dimensional indices of the weight tensor, and W i,j,k,c is the weight tensor corresponding to the channel. Channels with an importance score lower than 0.01 are removed to reduce the computational complexity of the model and improve the running efficiency of the model on edge computing devices. After channel pruning, the computational complexity of the model is reduced by 65%.

[0096] Furthermore, the inference acceleration unit includes:

[0097] Operator fusion sub-module, which merges the batch normalization layer and the convolutional layer using the following formula

[0098]

[0099]

[0100] where y is the output of the batch normalization layer, W fused is the fused weight, α fused is the fused scaling factor, b fused is the fused bias, ∈ is the numerical stability constant, W is the weight parameter of the original convolutional layer, x is the input feature map, μ bn is the batch normalization mean parameter, σ 2 bn is the batch normalization variance, γ is the scaling factor of the batch normalization layer, and β is the offset factor of the batch normalization layer. By operator fusion, the number of calculation steps is reduced and the inference speed is improved. After operator fusion, the model inference speed is increased by 200%.

[0101] Memory optimization sub-module, which adopts a shared memory pool design and performs dynamic memory allocation according to the following formula

[0102]

[0103] Among them, and are the input memory, output memory, and temporary memory requirements of the l-th layer respectively. By optimizing the memory, the memory utilization rate is improved, the memory access latency is reduced, and the inference process is accelerated. After memory optimization, the memory access latency is reduced by 40%.

[0104] Furthermore, the model evaluation metrics of the industrial scenario personnel status anomaly detection and warning system based on TinyML include:

[0105] Model performance metrics:

[0106]

[0107] Resource occupancy metrics:

[0108] Memory 用法 = ∑ l (param l + feature l );

[0109] FLOPS = ∑ l ops l ;

[0110] Latency metrics:

[0111] Latency = t 预处理 + t 推理 + t 后处理 ;

[0112] Among them, TP is the true positive, that is, the correctly identified abnormal state; FP is the false positive, that is, the wrongly identified normal state as abnormal; FN is the false negative, that is, the missed abnormal state; F1 is the score, a comprehensive indicator for evaluating the model performance, calculated by the harmonic mean of precision and recall; the closer the F1 value is to 1, the more comprehensive the system can capture potential dangerous states while reducing false alarms; param l is the number of parameters of the l-th layer, including weights and biases; feature l is the feature map size of the l-th layer, determined by the input dimension and layer configuration; ops l is the number of operations of the l-th layer; t 预处理 , t 推理 , t 后处理 are the times required for data preprocessing, inference, and postprocessing respectively.

[0113] Beneficial effects

[0114] An industrial scenario personnel status abnormal detection and early warning system and method based on TinyML proposed by the present invention has the following beneficial effects compared with the prior art:

[0115] (1) In the method of this technical solution, a multi-level algorithm architecture of PCA dimensionality reduction + ANN feature extraction + bidirectional attention LSTM is proposed. Among them, PCA dimensionality reduction retains 95% of the variance information to ensure the integrity of data features. ANN uses a three-layer (64-32-16) structure for feature extraction. The bidirectional attention LSTM enhances the recognition ability of long-term stationary states through the residual connection mechanism. This architecture can also operate stably on resource-constrained devices such as M5Stack StickC Plus2, effectively solving the technical problem of deploying complex deep learning models at the edge.

[0116] (2) In the method of this technical solution, a multi-level early warning decision-making mechanism based on pose features and physiological indicators is proposed. By comprehensively evaluating three key indicators: the abnormality degree of the fall pose (F_pose), the fluctuation of physiological indicators (F_physio), and the stationary duration (F_time), each indicator complements each other, accurately judges the severity of the abnormality, establishes a complete three-level early warning system, enhances the ability to distinguish abnormalities, and effectively improves the early warning accuracy.

[0117] (3) In the system proposed by this technical solution, 8-bit fixed-point quantization provides a basis for channel pruning and knowledge distillation, reducing resource requirements; channel pruning streamlines the model structure and improves the operation efficiency; knowledge distillation further optimizes the model while maintaining the accuracy; operator fusion reduces the calculation steps, and the memory sharing pool optimizes the memory management. Each technology promotes each other, greatly improving the efficiency and performance of the model on edge devices. The comprehensive advantage far exceeds the individual effects of each technology.

[0118] (4) In the system proposed by this technical solution, the system architecture composed of a portable data acquisition terminal, an edge computing unit, and a cloud platform operates in coordination. Each part works closely together to achieve all-round real-time monitoring and early warning of personnel status, improve the safety guarantee level of personnel in industrial scenarios, and demonstrate the overall coordination advantage of the system. Description of the Drawings

[0119] Figure 1 It is the overall system architecture diagram of the present invention.

[0120] Figure 2 It is the algorithm flow chart in the present invention.

[0121] Figure 3 It is the deep learning network structure diagram in the present invention.

[0122] Figure 4 It is the flow chart of the three-level early warning decision-making method based on multi-dimensional feature fusion in the present invention.

[0123] Figure 5 This is the data processing flowchart of the present invention. Specific embodiments

[0124] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Without departing from the design concept of the present invention, various modifications and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope of the present invention.

[0125] Embodiment 1:

[0126] As Figure 1 shown, an industrial scenario personnel status abnormal detection and early warning system based on TinyML is applied to the above-mentioned industrial scenario personnel status abnormal detection and early warning method based on TinyML. The abnormal detection and early warning system includes:

[0127] Portable data acquisition terminal: This terminal includes an inertial measurement unit and a main controller. The inertial measurement unit and the main controller are connected through an I2C interface. The terminal is also provided with an interface for expanding external sensors.

[0128] Edge computing processing unit: Connected to the portable data acquisition terminal; the edge computing processing unit includes: a data preprocessing module, a feature extraction module, and a time series analysis module; the data preprocessing module performs preprocessing operations on the collected raw data; the feature extraction module extracts features from the preprocessed data; the time series analysis module performs time series analysis on the data after feature extraction.

[0129] Model compression unit: Includes a weight quantization sub-module, a knowledge distillation sub-module, and a channel pruning sub-module.

[0130] Among them, the weight quantization sub-module compresses 32-bit floating-point numbers into 8-bit fixed-point numbers;

[0131]

[0132] In the above formula, w q is the 8-bit fixed-point number weight value after quantization, w is the original weight, w min and w max are the minimum and maximum values of the weight respectively. By weight quantization, the model storage space and calculation amount are reduced. After weight quantization, the model storage space is reduced by 75%, and the calculation amount is reduced by 60%.

[0133] The knowledge distillation sub-module includes minimizing the KL divergence between the teacher model and the student model;

[0134]

[0135] Among them, L KD is the value of the knowledge distillation loss function, α is the balance coefficient, set to 0.5; T temp is the temperature parameter, set to 2.0; z t and z s are the outputs of the teacher model and the student model respectively, CE is the cross-entropy loss function, y s is the predicted value of the student model, y true is the true label; through knowledge distillation, the student model simplifies the model structure while maintaining high accuracy. After knowledge distillation, the accuracy of the student model has an error of within ±2% compared with the teacher model, and at the same time, the number of model parameters is reduced by 70%.

[0136] The channel pruning sub-module prunes the model based on the importance score of the L1 norm. The formula for the score is:

[0137]

[0138] In the above formula, score c is the importance score of channel c, |W c | 1 is the L1 norm of the weight tensor, i, j, k are the three-dimensional indices of the weight tensor, W i,j,k,c is the weight tensor corresponding to the channel. Channels with an importance score lower than 0.01 are removed to reduce the model's computational complexity and improve the model's running efficiency on edge computing devices. After channel pruning, the model's computational complexity is reduced by 65%.

[0139] Inference acceleration unit: includes an operator fusion sub-module and a memory optimization sub-module.

[0140] The operator fusion sub-module combines the batch normalization layer and the convolutional layer using the following formula

[0141]

[0142] Among them, y is the output of the batch normalization layer, W fused is the fused weight, α fused is the fused scaling factor, b fused is the fused bias, ∈ is the numerical stability constant, W is the weight parameter of the original convolutional layer, x is the input data, μ bn is the batch normalization mean parameter, σ 2 bnis the batch normalization variance, γ is the scaling factor of the batch normalization layer, and β is the offset factor of the batch normalization layer; by fusing operators, the number of calculation steps is reduced and the inference speed is improved. After operator fusion, the model inference speed is increased by 200%.

[0143] The memory optimization sub-module adopts a shared memory pool design and dynamically allocates memory according to the following formula:

[0144]

[0145] where and are the input memory, output memory, and temporary memory requirements of the l-th layer respectively. By optimizing the memory, the memory utilization rate is improved, the memory access latency is reduced, and the inference process is accelerated. After memory optimization, the memory access latency is reduced by 40%.

[0146] Cloud monitoring platform: Communicates with the edge computing processing unit using an encrypted wireless communication protocol and has an automatic reconnection mechanism to meet the latency and reliability requirements for real-time monitoring; the platform includes a data storage module, a deep analysis module, a warning management module, and a visualization display module for receiving, storing, analyzing the data uploaded by the edge computing processing unit, and performing warning management and visualization display.

[0147] The model evaluation metrics of the industrial scenario personnel status anomaly detection and warning system based on TinyML include:

[0148] Model performance metrics:

[0149]

[0150] Resource occupancy metrics:

[0151] Memory 用法 = ∑ l (param l + feature l );

[0152] FLOPS = ∑ l ops l ;

[0153] Latency metrics:

[0154] Latency = t 预处理 + t 推理 + t 后处理 ;

[0155] Among them, TP is the true positive, FP is the false positive, FN is the false negative, and F1 is the F1 score, which is a comprehensive indicator for evaluating the model performance and is calculated by the harmonic mean of precision and recall. The closer the F1 value is to 1, the more comprehensively the system can capture potential dangerous states while reducing false alarms; param l is the number of parameters in the l-th layer, including weights and biases; feature l is the size of the feature map in the l-th layer, which is determined by the input dimension and layer configuration; ops l is the number of operations in the l-th layer; t 预处理 、t 推理 、t 后处理 are the times required for data preprocessing, inference, and postprocessing respectively.

[0156] Example 2:

[0157] As Figure 2 shown, an industrial scenario personnel status anomaly detection and warning method based on TinyML includes the steps:

[0158] Step 1: Data collection: Collect triaxial acceleration data and triaxial angular velocity data through a portable data collection terminal; use the LSM6DSO six-axis inertial sensor built in M5Stack StickC Plus2 to collect triaxial acceleration data A(t) = [a x (t), a y (t), a z (t)] and triaxial angular velocity data G(t) = [g x (t), g y (t), g z (t)]; the collection frequency of triaxial acceleration data and triaxial angular velocity data is 50Hz. Collect heart rate data H(y) through a MAX30100 heart rate sensor, and collect body temperature data T(t) using a DS18B20 temperature sensor; the collection frequency of heart rate data and body temperature data is 1Hz. Transmit the collected data to the edge computing processing unit for processing; to ensure the effectiveness and analyzability of the data.

[0159] Step 2: Data preprocessing: Input the collected raw data into the edge computing processing unit for denoising processing; the specific operation method is:

[0160] Step 2.1: Input the collected raw data into the edge computing processing unit, and the edge computing processing unit performs sliding window segmentation on the collected raw data, with a window size of 2 seconds and an overlap rate of 50%;

[0161] Step 2.2: Denoise the signal using wavelet transform. Select the Daubechies wavelet basis function and remove the noise components according to the soft threshold rule. The soft threshold is calculated based on the wavelet coefficients of the signal and the estimated value of the noise standard deviation;

[0162] Step 2.3: Perform Z-score standardization. Calculate the mean and standard deviation of the data within the window and standardize the data to a distribution with a mean of 0 and a standard deviation of 1.

[0163] Step Three: Feature extraction: Process the preprocessed data. The specific operation method is as follows:

[0164] Step 3.1: Perform PCA dimensionality reduction on the original data. The formula for PCA dimensionality reduction is:

[0165] X PCA = XW PCA ,

[0166] In the above formula, X is the original data matrix, W PCA is the PCA transformation matrix, and X PCA is the data matrix after PCA dimensionality reduction.

[0167] Performing PCA dimensionality reduction on the original data is based on the PCA dimensionality reduction sub-module; the PCA dimensionality reduction sub-module performs eigenvalue decomposition by calculating the data covariance matrix. The expression of the data covariance matrix is:

[0168]

[0169] where n is the number of samples, x i is the i-th sample data, μ is the sample mean, and T represents the transpose of the matrix; the eigenvalue decomposition of the covariance matrix

[0170] Σ = UΛU T ,

[0171] U is the eigenvector matrix, and its column vectors are the eigenvectors of the covariance matrix Σ; Λ is a diagonal matrix, and the diagonal elements are the corresponding eigenvalues; U T is the transpose matrix of U.

[0172] Select the eigenvectors with a cumulative contribution rate reaching 95%:

[0173]

[0174] In the above formula, k is the number of selected eigenvectors;

[0175] And perform projection transformation to achieve data dimensionality reduction; the expression of the projection transformation is:

[0176] X PCA = XUk ,

[0177] Among them, X is the original data, and U k is the matrix composed of the selected feature vectors.

[0178] Step 3.2: Use ANN for feature extraction:

[0179] Use the pre-trained ANN feature extraction network to extract features from the data after PCA dimensionality reduction. This network adopts a three-layer neural network structure, including an input layer - hidden layer 1 - hidden layer 2 - output layer; the number of nodes in the input layer is determined according to the dimension of the data after PCA dimensionality reduction, the number of nodes in hidden layer 1 is 64, the number of nodes in hidden layer 2 is 32, and the number of nodes in the output layer is 16. The activation function uses the ReLU function:

[0180] h l = f(W l h l-1 + b l )

[0181] In the above formula, h l is the output of the l-th hidden layer, W l is the weight matrix of the l-th layer, b l is the bias vector of the l-th layer, and f is the ReLU activation function;

[0182] When using ANN for feature extraction, the optimization objective of the ANN feature extraction network is:

[0183]

[0184] Among them, θ is the network parameter, λ is the regularization coefficient, set to 0.001, N is the total number of training samples, f θ is the neural network model function, x i is the i-th input sample, and y i is the true label value corresponding to the i-th sample;

[0185] Use the Adam optimizer for training, and its update steps include:

[0186] m t = β 1 m t-1 + (1 - β 1 )g t

[0187]

[0188] Among them, m t is the first-order momentum estimation value at the current moment, v t is the second-order momentum estimation value at the current moment, and g tis the gradient, β 1 and β 2 are hyperparameters of the Adam optimizer, which are set to 0.9 and 0.999 respectively; is the first-order momentum estimate after bias correction, is the second-order momentum estimate after bias correction; θ t is the parameter update value at the current moment; α is the learning rate, set to 0.001; ∈ is a small constant to prevent division by zero, set to 1e - 8; Through the above optimization process, the ANN feature extraction network converges to the optimal parameters, enhancing the feature extraction ability.

[0189] Step 4: Temporal analysis: Use the improved bidirectional attention LSTM network to perform temporal modeling on the data after feature extraction; Through temporal analysis, the system obtains the change characteristics of the personnel status data in the time dimension, and the change characteristics will be used as an important basis for multi-level early warning decisions.

[0190] The improved bidirectional attention LSTM network performs temporal modeling on the data after feature extraction, as Figure 3 shown, the temporal modeling includes:

[0191] Forward LSTM computational unit, satisfying the condition:

[0192]

[0193] where, x t is the input data, is the forward hidden state at the previous moment, which internally includes:

[0194] Input gate: i t = σ(W ii x t + W hi h t-1 + b i ),

[0195] Forget gate: f t = σ(W if x t + W hf h t-1 + b f ),

[0196] Candidate memory content: g t = tanh(W ig x t + W hg h t-1 + b g ),

[0197] Memory cell state: c t = ft ⊙c t-1 +i t ⊙g t ,

[0198] Output gate: o t = σ(W io x t +W ho h t-1 +b o ),

[0199] Hidden state: h t = o t ⊙tanh(c t ),

[0200] where W ii , W if , W ig , W io , W hi , W hf , W hg , W ho are the corresponding weight matrices, b i , b f , b g , b o are the bias vectors, and σ is the sigmoid function;

[0201] Backward LSTM computational unit, satisfying the condition:

[0202]

[0203] The computational process of the backward LSTM computational unit is the same as that of the forward LSTM, but the data input order is reversed; The attention mechanism computational unit calculates the attention weights, and the calculation formula is:

[0204] α t = softmax(W a [h t ; c t +b a )

[0205] where h t is the hidden state at the current moment, and the memory cell state at time t is:

[0206]

[0207] where T is the sequence length, corresponding to the sliding window size, W a is the weight matrix, b a is the bias vector, and the key temporal information is focused through the attention mechanism.

[0208] The residual connection structure of the bidirectional attention LSTM network is defined as:

[0209]

[0210] where l represents the index of the LSTM layer, l ∈ {1, 2, …, L}, represents the hidden state of the l-th layer at time t, represents the hidden state of the (l - 1)-th layer at time t;

[0211] Attention weight calculation:

[0212] Calculate the attention score:

[0213] Normalize to obtain the attention weight:

[0214] Calculate the context vector: z t = ∑ s α t,s h s ;

[0215] where v a ∈ R d is the attention vector parameter for calculating the attention score; U a ∈ R d×d is the attention weight matrix for encoding the hidden state of the input sequence; h s ∈ R d represents the hidden state at time step s, s represents the time step of the source sequence; z t ∈ R d represents the context vector at time t; d is the dimension of the hidden state; α t,s represents the attention weight of time t to time step s.

[0216] Step Five: Comprehensive evaluation: The system comprehensively evaluates the personnel status according to the preset warning conditions, combining the posture features and physiological indicators, and adopts a multi-level warning decision to achieve accurate abnormal warning; as Figure 4 shown, the multi-level warning decision includes:

[0217] First-level warning condition: Slight fall detected and physiological indicators are normal

[0218] Score 1 = w 11 F pose + w 12 F physio < θ 1 ;

[0219] Second-level warning condition: Relatively serious fall detected or physiological indicators are abnormal

[0220] Score 2 = w 21 F pose + w 22 F physio <θ 2 ;

[0221] Level 3 early warning condition: Fall detected and long - term stillness or physiological crisis

[0222] Score 3 = w 31 F pose + w 32 F physio <θ 3 ;

[0223] In the above formula, w 11 , w 12 are the weight coefficients of the first - level early warning, w 21 , w 22 are the weight coefficients of the second - level early warning, w 31 , w 32 are the weight coefficients of the third - level early warning, F pose is the score of the abnormal posture degree, calculated from the posture features after normalization; F physio is the score of the abnormal physiological index degree, calculated from the physiological indexes after normalization; θ 1 , θ 2 , θ 3 are the first - level, second - level and third - level early warning thresholds respectively.

[0224] Case study: Monitoring of personnel status in chemical production scenarios

[0225] 1. Application scenario setting:

[0226] In this embodiment, a typical chemical production workshop is used as the application scenario. There are complex process flows, high - temperature and high - pressure equipment, and potential risks of leakage of toxic and harmful gases in the workshop. The workshop area is about 3000 square meters, divided into different functional areas such as reaction area, storage area and operation control area. The staff perform operations such as equipment inspection, material addition, and process parameter monitoring in each area. The working environment temperature usually fluctuates between 25 - 40 °C, and the relative humidity is between 40 - 70%. There are certain electromagnetic interference sources in the environment, such as the electromagnetic fields generated by the operation of large motors, transformers and other equipment.

[0227] The workshop operates on a three - shift system, with about 15 staff members in each shift. They need to work continuously in the workshop for 8 hours, facing potential risks such as physical fatigue, slipping, and physical discomfort caused by inhaling trace amounts of harmful gases.

[0228] 2. Hardware System Configuration

[0229] Portable data acquisition terminal: The M5Stack StickC Plus2 device is adopted. The built-in acceleration sensor has a measurement range of ±2g and an accuracy of 0.01g. The measurement range of the gyroscope sensor is ±200° / s and the accuracy is 0.1° / s. The heart rate monitoring module adopts the photoplethysmogram (PPG) technology, and the measurement error is within ±3 beats per minute. The measurement accuracy of the body temperature sensor is ±0.2°C. The main controller of the terminal is ESP32-S3, which conducts high-speed data transmission with the inertial measurement unit through the I2C interface, and the data transmission rate can reach 400kHz. At the same time, the terminal is equipped with an interface for expanding external sensors, such as an external gas sensor can be connected to detect the concentration of toxic and harmful gases in the environment, so as to more comprehensively monitor the safety status of the working environment of the staff.

[0230] Edge computing processing unit: An embedded computing board based on the ARM Cortex-A72 architecture is selected as the edge computing device, which has a 4-core processor with a main frequency as high as 2.0GHz and a memory of 8GB DDR4. This device has powerful computing capabilities, can meet the real-time processing requirements of complex algorithms, and can operate efficiently through optimized models and algorithms under resource-constrained conditions. The edge computing processing unit conducts wireless data transmission with the portable data acquisition terminal through Bluetooth 5.0, and the transmission distance can reach 50 meters, ensuring stable and fast data reception within the workshop range. At the same time, it has a data verification and retransmission mechanism to ensure the accuracy and integrity of data transmission.

[0231] Cloud monitoring platform: It is built on the private cloud server within the enterprise, adopts a distributed storage architecture, and has a storage capacity of 50TB, which can meet the long-term storage requirements of personnel status data. The server is equipped with high-performance CPUs and GPUs for in-depth analysis and processing of data, and the data processing speed can reach 10GB / s. The cloud platform communicates with the edge computing processing unit through the 4G / 5G network, and adopts the SSL / TLS encryption protocol to ensure the security of data transmission. The average network communication delay is within 50 milliseconds, and the data transmission packet loss rate is less than 0.1%, ensuring the timeliness and reliability of real-time monitoring.

[0232] 3. Data Processing Flow

[0233] Data collection: The portable data collection terminal collects the triaxial acceleration and triaxial angular velocity data of the staff at a frequency of 50 Hz, and collects the heart rate and body temperature data at a frequency of 1 Hz. During the collection process, first, the original sensor data is preliminarily filtered. The Kalman filter algorithm is used to remove noise interference and improve the stability and accuracy of the data. Then, the collected data is cached in real-time. When the cached data volume reaches a certain threshold (such as 100 data points), the data is sent to the edge computing processing unit through the wireless transmission module.

[0234] Data preprocessing: After the edge computing processing unit receives the data, the data preprocessing module performs sliding window segmentation on it. The window size is 2 seconds and the overlap rate is 50%. The window segmentation operation is implemented by setting a counter and a data buffer. The counter triggers a window segmentation every 100 data points (corresponding to 2 seconds, sampling rate 50 Hz), and the data in the data buffer is processed subsequently.

[0235] Wavelet transform is used to denoise the signal. The Daubechies wavelet basis function (such as db4) is selected. By decomposing the signal into multiple layers, the noise components are removed according to the soft threshold rule. The calculation of the soft threshold is based on the wavelet coefficients of the signal and the estimated value of the noise standard deviation, effectively improving the signal-to-noise ratio of the signal.

[0236] Z-score standardization is performed. The mean and standard deviation of the data within the window are calculated. The mean is obtained by accumulating the data points and dividing by the number of data points. The standard deviation is calculated according to the variance formula (the variance is the sum of the squares of the differences between each data point and the mean divided by the number of data points). Then, the data is standardized to a distribution with a mean of 0 and a standard deviation of 1, which is convenient for subsequent feature extraction and model processing.

[0237] Feature extraction: The PCA dimensionality reduction sub-module processes the preprocessed data, calculates the data covariance matrix, which is obtained by summing the products of the deviations of the sample data from the mean and dividing by the sample size minus 1. Then, eigenvalue decomposition is performed to obtain the eigenvalues and eigenvectors. The eigenvectors with a cumulative contribution rate reaching 95% are selected. The number of selected eigenvectors is determined by calculating the sum of the eigenvalue ratios. Finally, projection transformation is performed to achieve data dimensionality reduction, mapping the original high-dimensional data to a low-dimensional space and reducing the data complexity.

[0238] The ANN feature extraction sub-module adopts a three-layer neural network structure (input layer - hidden layer 1 - hidden layer 2 - output layer). The number of nodes in the input layer is determined according to the data dimension after PCA dimensionality reduction. The number of nodes in hidden layer 1 is 64, the number of nodes in hidden layer 2 is 32, and the number of nodes in the output layer is set to 16 according to the requirements of subsequent tasks. The ReLU function is used as the activation function, and the output of each layer is calculated through forward propagation. The weight matrix and bias vector are updated according to the optimization objective during the training process. The optimization objective is to minimize the mean square error between the predicted value and the true value plus the regularization term. The Adam optimizer is used for training, and its hyperparameters are set as learning rate 0.001, β 1 = 0.9, β 2 = 0.999, ∈ = 1e - 8, and the regularization coefficient λ = 0.001.

[0239] Time series analysis: The improved bidirectional attention LSTM network performs time series modeling on the data after feature extraction. The forward LSTM computational unit calculates the hidden state at the current moment based on the input data and the hidden state at the previous moment. The calculations of the input gate, forget gate, candidate memory content, and output gate are obtained by multiplying with the corresponding weight matrix and bias vector and then passing through the sigmoid or tanh activation function. The memory cell state and hidden state are updated according to the gating mechanism. The calculation process of the backward LSTM computational unit is similar, except that the data input order is reversed.

[0240] The attention mechanism computational unit combines the hidden state of the bidirectional LSTM with the context vector by calculating the attention weights. The weight matrix and bias vector are learned during the training process, and the attention weights are obtained through the softmax function, thereby focusing on key time series information and improving the model's ability to capture important information in long time series data.

[0241] Multi-level early warning decision-making:

[0242] The first-level early warning condition is set as detecting a minor fall and normal physiological indicators. For example, through the analysis of human posture data, when it is detected that the body tilt angle of a person exceeds a certain threshold (such as 30°) for a short duration (such as 1 - 2 seconds) and physiological indicators such as heart rate and body temperature are within the normal range, the early warning score Score is calculated 1 = w 1 F pose + w 2 F physio <θ 1 where F pose is calculated according to the degree of posture abnormality (such as by calculating the displacement and angle change of human key points), F physio is calculated according to the degree of deviation of physiological indicators from the normal range, w 1 = 0.7, w 2= 0.3 is the weight coefficient. When Score 1 <θ 1 (θ 1 is set to 0.5 according to historical data and practical experience), a first-level warning is issued.

[0243] The second-level warning condition is detecting a relatively serious fall or abnormal physiological indicators. For example, when the fall angle of a person is relatively large (exceeding 60°) and the duration is relatively long (exceeding 3 seconds), or the heart rate exceeds the normal range (such as exceeding 100 beats per minute or less than 60 beats per minute), the body temperature rises abnormally (exceeding 38°C), etc. Calculate Score 2 = w 1 F pose + w 2 F physio , w 1 = 0.6, w 2 = 0.4. When Score 2 <θ 2 (θ 2 is set to 0.3), a second-level warning is issued.

[0244] The third-level warning condition is detecting a fall and remaining stationary for a long time or a physiological crisis. For example, when there is no sign of recovery within 5 minutes after a person falls, or the heart rate drops suddenly (less than 50 beats per minute), the body temperature is too high (exceeding 40°C) and other serious physiological abnormalities. Calculate Score 3 = w 1 F pose + w 2 F physio , w 1 = 0.5, w 2 = 0.5. When Score 3 <θ 3 (θ 3 is set to 0.1), a third-level warning is issued.

[0245] 4. Model Optimization and Inference Acceleration

[0246] Model Compression:

[0247] The weight quantization unit compresses the 32-bit floating-point parameters in the model into 8-bit fixed-point numbers. By calculating the minimum and maximum values of the weights and mapping the weights to the 8-bit integer range according to the formula, after weight quantization, the model storage space is reduced by 75%, and the calculation amount is reduced by 60%, effectively reducing the storage requirements and computing resource consumption of the model on edge computing devices.

[0248] The knowledge distillation unit uses a pre-trained large teacher model (such as a complex deep learning model trained on a large-scale personnel status dataset) to guide the training of the lightweight student model. By minimizing the KL divergence and cross-entropy loss function between the outputs of the teacher model and the student model, as well as the temperature parameter and balance coefficient, the student model can streamline its model structure while maintaining high accuracy. After knowledge distillation, the accuracy of the student model has an error of within ±2% compared to the teacher model, and at the same time, the number of model parameters is reduced by 70%.

[0249] The channel pruning unit prunes the model based on the importance score of the L1 norm. It calculates the L1 norm of each channel weight tensor as the importance score and removes channels with an importance score lower than 0.01, reducing the model's computational complexity and improving the running efficiency of the model on edge computing devices. After channel pruning, the computational complexity of the model is reduced by 65%.

[0250] Inference acceleration:

[0251] The operator fusion unit merges the batch normalization layer and the convolutional layer, calculates the merged parameters according to the formula, reduces the number of calculation steps, and improves the inference speed. After operator fusion, the inference speed of the model is increased by 200%.

[0252] The memory optimization unit adopts a shared memory pool design, dynamically allocates memory according to the input memory, output memory, and temporary memory requirements of each layer, improves the memory utilization rate, reduces the memory access latency, and accelerates the inference process. After memory optimization, the memory access latency is reduced by 40%.

[0253] 5. System operation effect evaluation

[0254] Model performance metrics: By testing on a personnel status dataset (including normal status and various abnormal status data) collected in a simulated chemical production scenario, the accuracy (Precision), recall (Recall), and F1 value of the model are calculated. For example, in the test set, there are 1000 samples, among which the true positives (TP) are 800, the false positives (FP) are 50, and the false negatives (FN) are 100. Then the accuracy Recall rate F1 value

[0255] Resource Occupancy Metrics: When the system runs on edge computing devices, monitor the memory usage and computational volume (FLOPS) of the system. For example, when the system runs, the occupied memory is approximately 2GB, and the computational volume is 1 billion floating-point operations per second (1 GFLOPS). Compared with the unoptimized model, the resource occupancy is significantly reduced, indicating that the model optimization measures are effective.

[0256] Latency Metrics: Measure the total latency time of the system from data acquisition to warning output, including data preprocessing, inference, and postprocessing times. In actual tests, the average latency time is 200 milliseconds, which meets the real-time requirements for detecting abnormal personnel status in chemical production scenarios, can issue warnings in a timely manner, and ensure the safety of staff.

[0257] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. Through the above embodiments, the specific application methods, data processing processes, model optimization effects, and system operation performance evaluations of the present invention in chemical production scenarios are demonstrated, reflecting the effectiveness and practicality of the invention in detecting and warning abnormal personnel status in complex industrial environments. Although not actually deployed, its feasibility and advantages are expounded from the perspectives of theory and simulation data. Any equivalent transformation or modification made according to the spirit and essence of the present invention should be covered by the present invention.

Claims

1. A TinyML-based industrial scene personnel status abnormality detection and early warning method, characterized in that: Includes steps: Step 1: Data collection: Collect triaxial acceleration data A(t)=[a x (t),a y (t),a x (t)] and the three-axis angular velocity data G(t) = [g x (t),g y (t),g z (t)], collect heart rate data H(t) through the heart rate sensor, and collect body temperature data T(t) using the temperature sensor; The collected data is transmitted to the edge computing processing unit for processing; Step 2: Data preprocessing: Input the collected raw data into the edge computing processing unit for denoising; Step 3: Feature extraction: Process the preprocessed data. The specific operation method is as follows: Step 3.1: Perform PCA dimensionality reduction on the original data. The formula for PCA dimensionality reduction is: X PCA =XW PCA , In the above formula, X is the original data matrix, W PCA is the PCA transformation matrix, X PCA is the data matrix after PCA dimensionality reduction; Step 3.2: Feature extraction using ANN: A pre-trained ANN feature extraction network is used to extract features from the data after PCA dimension reduction. The network adopts a three-layer neural network structure, including input layer-hidden layer 1-hidden layer 2-output layer; the number of nodes in the input layer is determined according to the data dimension after PCA dimension reduction, the number of nodes in hidden layer 1 is 64, the number of nodes in hidden layer 2 is 32, and the number of nodes in the output layer is 16. The activation function adopts the ReLU function: h l =f(W l h l-1 +b l ) In the above formula, h l is the output of the lth hidden layer, W l is the weight matrix of the lth layer, b l is the bias vector of the lth layer, and f is the ReLU activation function; Step 4: Time series analysis: Use the improved bidirectional attention LSTM network to perform time series modeling on the data after feature extraction; through time series analysis, the system obtains the change characteristics of personnel status data in the time dimension, and the change characteristics will serve as an important basis for multi-level warning decision-making; Step 5: Comprehensive evaluation: The system uses multi-level warning decision-making to comprehensively evaluate the personnel status based on the preset warning conditions, combined with posture characteristics and physiological indicators, to achieve accurate abnormal warning; The multi-level early warning decision-making includes: Level 1 warning condition: A minor fall is detected and physiological indicators are normal Score1=w 11 F pose +w 12 F ph ysio <θ1; Secondary warning conditions: severe falls or abnormal physiological indicators detected Score2=w 21 F pose +w 22 F ph ysio <θ2; Level 3 warning condition: Fall detected and prolonged inactivity or physiological crisis Score3=w 31 F pose +w 32 F ph ysio <θ3; In the above formula, w 11 、w 12 is the first-level warning weight coefficient, w 21 、w 22 is the secondary warning weight coefficient, w 31 、w 32 is the weight coefficient of the third-level warning, F pose is the posture abnormality score, which is calculated by the normalized posture features; F ph ysio is the abnormality score of physiological indicators, which is calculated through normalized physiological indicators; θ1, θ2, and θ3 are the first, second, and third level warning thresholds, respectively.

2. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: The portable data acquisition terminal adopts the LSM6DSO six-axis inertial sensor built into the M5Stack StickC Plus2; the heart rate sensor adopts the MAX30100 heart rate sensor; and the temperature sensor adopts the DS18B20 temperature sensor.

3. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: The acquisition frequency of the three-axis acceleration data and the three-axis angular velocity data is 50 Hz, and the acquisition frequency of the heart rate data and the body temperature data is 1 Hz.

4. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: The data preprocessing described in step 2 is specifically performed as follows: Step 2.1: Input the collected raw data into the edge computing processing unit, which performs sliding window segmentation on the collected raw data, with a window size of 2 seconds and an overlap rate of 50%; Step 2.2: Use wavelet transform to denoise the signal, select Daubechies wavelet basis function, and remove noise components according to the soft threshold rule. The soft threshold is calculated based on the wavelet coefficient of the signal and the estimated value of the noise standard deviation; Step 2.3: Perform Z-score standardization, calculate the mean and standard deviation of the data in the window, and standardize the data to a distribution with a mean of 0 and a standard deviation of 1.

5. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: The PCA dimensionality reduction of the original data described in step 3.1 is operated based on the PCA dimensionality reduction submodule; the PCA dimensionality reduction submodule performs eigenvalue decomposition by calculating the data covariance matrix, and the expression of the data covariance matrix is: Where n is the number of samples, x i is the i-th sample data, μ is the sample mean, and T represents the transpose of the matrix; Eigenvalue decomposition of the covariance matrix Σ=UWU T , U is the eigenvector matrix, whose column vectors are the eigenvectors of the covariance matrix ∑; Λ is a diagonal matrix, and the diagonal elements are the corresponding eigenvalues; U T is the transposed matrix of U; Select the eigenvectors whose cumulative contribution reaches 95%: In the above formula, k is the number of selected eigenvectors, λ i is the i-th eigenvalue; And perform projection transformation to achieve data dimensionality reduction; the expression of projection transformation is: X PCA =XU k , Among them, X is the original data, U k is the matrix consisting of the selected eigenvectors.

6. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: In step 3.2, ANN is used for feature extraction. The optimization goal of the ANN feature extraction network is: Among them, θ is the network parameter, λ is the regularization coefficient, which is set to 0.001, N is the total number of training samples, and f θ is the neural network model function, x i is the i-th input sample, y i is the true label value corresponding to the i-th sample; The Adam optimizer is used for training, and its update steps include: m t =β1m t-1 +(1-β1)g t Among them, m t is the first-order momentum estimate at the current moment, v t is the second-order momentum estimate at the current moment, g t is the gradient, β1 and β2 are the hyperparameters of the Adam optimizer, which are set to 0.9 and 0.999 respectively; is the bias-corrected first-order momentum estimate, is the bias-corrected second-order momentum estimate; θ t is the parameter update value at the current moment; α is the learning rate, set to 0.001; ∈ is a small constant to prevent division by zero, set to 1e-8; Through the above optimization process, the ANN feature extraction network converges to the optimal parameters and enhances the feature extraction capability.

7. According to claim 1, a TinyML-based industrial scene personnel status abnormality detection and early warning method is characterized by: The improved bidirectional attention LSTM network described in step 4 performs time series modeling on the feature-extracted data. The time series modeling includes: Forward LSTM computing unit, meeting the conditions: Among them, x t For input data, is the forward hidden state of the previous moment, which includes: Input gate: i t =σ(W ii x t +W hi h t-1 +b i ), Forget gate: f t =σ(W if x t +W hf h t-1 +b f ), Candidate memory content: g t =tanh(W ig x t +W hg h t-1 +b g ), Memory cell status: c t =f t ⊙c t-1 +i t ⊙g t , Output gate: o t =σ(W io x t +W ho h t-1 +b o ), Hidden state: h t =o t ⊙tanh(c t ), Among them, W ii ,W if ,W ig ,W io is the input-related weight matrix, W hi ,W hf ,W hg ,W ho is the weight matrix associated with the hidden state, b i ,b f ,b g ,b o is the corresponding bias vector, σ is the sigmoid function; Backward LSTM computing unit, meeting the conditions: The calculation process of the backward LSTM calculation unit is the same as that of the forward LSTM, and the data input order is reversed; the attention mechanism calculation unit obtains the attention weight through calculation, and the calculation formula is: a t =softmax(W a [h t ;c t ]+b a ) Among them, h t is the hidden state at the current moment, and the memory unit state at time t is: Among them, S is the sequence length, corresponding to the sliding window size, W a is the weight matrix, b a For the bias vector, focus on key timing information through the attention mechanism; The residual connection structure of the bidirectional attention LSTM network is defined as: Where l represents the index of the LSTM layer, l∈{1,2,…,L}, represents the hidden state of layer l at time t, represents the hidden state of the l-1th layer at time t; Attention weight calculation: Calculate the attention score: Normalize to get the attention weight: Calculate the context vector: z t =∑ s α t,s h s ; Among them, v a ∈R d is the attention vector parameter, used to calculate the attention score; U a ∈R d×d is the attention weight matrix, which is used to encode the hidden state of the input sequence; h s ∈R d represents the hidden state at time s, where s represents the time step of the source sequence; z t ∈R d represents the context vector at time t; d is the hidden state dimension; α t,s represents the attention weight of time t to time s, is the transpose of the attention vector parameters, h t-1 is the hidden state at the previous moment, e t,s is the attention score at time t to time s, a scalar value; s′ is the time step index in the sequence, h s is the hidden state of the source sequence at time s.

8. A TinyML-based industrial scene personnel status abnormality detection and early warning system, characterized in that: The method for detecting and warning abnormal status of personnel in industrial scenes based on TinyML as described in any one of claims 1 to 7, wherein the abnormal detection and warning system comprises: Portable data acquisition terminal: The terminal includes an inertial measurement unit and a main controller. The inertial measurement unit and the main controller are connected through an I2C interface. The terminal also has an interface for expanding external sensors. Edge computing processing unit: connected to the portable data acquisition terminal; the edge computing processing unit includes: a data preprocessing module, a feature extraction module and a timing analysis module; the data preprocessing module performs preprocessing operations on the collected raw data; the feature extraction module performs feature extraction on the preprocessed data; the timing analysis module performs timing analysis on the feature extracted data; Model compression unit: including weight quantization submodule, knowledge distillation submodule and channel pruning submodule; Inference acceleration unit: includes operator fusion submodule and memory optimization submodule; Cloud monitoring platform: It communicates with the edge computing processing unit using an encrypted wireless communication protocol and has an automatic reconnection mechanism; the platform includes a data storage module, a deep analysis module, an early warning management module, and a visualization module, which are used to receive, store, and analyze data uploaded by the edge computing processing unit, and perform early warning management and visualization.

9. The TinyML-based industrial scene personnel status abnormality detection and early warning system according to claim 8, characterized in that: The model compression unit comprises: Weight quantization submodule: compresses 32-bit floating-point numbers into 8-bit fixed-point numbers; Among them, w q is the quantized 8-bit fixed-point weight value, w is the original 32-bit floating-point weight value, w min and w max are the minimum and maximum values ​​of the weight, respectively. Knowledge distillation submodule: minimize the KL divergence between the teacher model and the student model; Among them, L KD is the value of the knowledge distillation loss function, α is the balance coefficient, which is set to 0.5; T temp is the temperature parameter, set to 2.0; z t and z s are the outputs of the teacher model and the student model respectively, CE is the cross entropy loss function, y s is the predicted value of the student model, y true is the true label; Channel pruning submodule: The model performs channel pruning based on the importance score of the L1 norm. The scoring formula is: In the above formula, score c Score the importance of channel c, |W c |1 is the L1 norm of the weight tensor, i, j, k are the three-dimensional indexes of the weight tensor, W i,j,k,c The weight tensor corresponding to the channel is used to remove channels with an importance score lower than 0.01, reduce the computational complexity of the model, and improve the running efficiency of the model on edge computing devices. After channel pruning, the computational complexity of the model is reduced by 65%.

10. The TinyML-based industrial scene personnel status abnormality detection and early warning system according to claim 9, characterized in that: The reasoning acceleration unit comprises: The operator fusion submodule uses the following formula to merge the batch normalization layer with the convolution layer: Among them, y is the output of the batch normalization layer, W fused is the weight after fusion, α fused is the scaling factor after fusion, b fused is the fused bias, ∈ is the numerical stability constant, W is the weight parameter of the original convolutional layer, x is the input feature map, μ bn is the batch normalization mean parameter, σ 2 bn is the batch normalization variance, γ is the scaling factor of the batch normalization layer, and β is the offset factor of the batch normalization layer; The memory optimization submodule adopts a shared memory pool design and dynamically allocates memory according to the following formula: in, and are the input memory, output memory, and temporary memory requirements of the lth layer, respectively.

Images