Abnormal behavior monitoring method and device based on industrial Internet of Things, terminal and medium

Abstract

The invention discloses an abnormal behavior monitoring method and device based on the industrial Internet of Things, a terminal and a medium, and the method comprises the steps: obtaining first behavior associated data based on the historical behavior data of a worker, the historical data of equipment operation, and the historical abnormal event data; obtaining second behavior associated data based on environmental state historical data, personnel position historical data and the historical abnormal event data; according to the first behavior associated data and the second behavior associated data, predicting the behavior of the worker to obtain an abnormal prediction result; and obtaining a target intervention scheme according to the anomaly prediction result, the real-time behavior data corresponding to the worker, the environment real-time data corresponding to the factory and the equipment state real-time data corresponding to the equipment. According to the invention, behaviors of workers in a factory can be accurately monitored, and whether the workers will carry out abnormal behaviors can be pre-judged, so that man-made safety production accidents can be avoided.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, device, terminal and medium for monitoring abnormal behavior based on the Industrial Internet of Things. Background Technology

[0002] The core pain point of factories (especially high-risk industries such as machinery manufacturing, chemical industry, and metallurgy) is that abnormal personnel behavior can easily lead to safety accidents and efficiency losses. More than 80% of safety accidents in factories are directly related to abnormal personnel behavior. For example, employees in machinery factories "not wearing safety helmets / not fastening safety ropes", employees in chemical factories "illegally touching dangerous valves / smoking", and employees in electronics factories "illegally performing welding operations" may cause equipment damage, personal injury or death, or even explosions and fires. Therefore, it is necessary to identify and intervene in a timely manner through monitoring.

[0003] However, current technologies for monitoring abnormal human behavior have the following shortcomings: most factories still rely on "computer vision + camera" as their main monitoring method, but this approach is greatly affected by the scene environment, and the accuracy of recognition is difficult to guarantee; video surveillance can only capture visually visible behaviors and cannot detect some non-visual anomalies (such as employees touching high-voltage equipment without permission, failing to clock in after being away from their posts for too long, or not operating buttons according to procedures), resulting in blind spots in the monitoring range; at the same time, some "rare abnormal behaviors" in factories (such as employees dismantling safety protection devices without permission, or suddenly fainting) have insufficient recognition rates or cannot be predicted through real-time images, thus failing to form effective early warnings. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, terminal and medium for abnormal behavior monitoring based on the Industrial Internet of Things, which aims to accurately monitor the behavior of workers in the factory, predict whether they will engage in abnormal behavior, and thus avoid human-caused safety accidents.

[0005] To achieve the above objectives, this application provides a method for monitoring abnormal behavior based on the Industrial Internet of Things (IIoT), the method comprising: Based on staff historical behavior data, equipment operation historical data, and historical abnormal event data, the first row of related data is obtained; Based on historical environmental status data, historical personnel location data, and the aforementioned historical abnormal event data, the second row of associated data is obtained; By using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data, and anomaly prediction results are obtained. A model is generated by pre-setting an intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0006] Specifically, the first row of associated data, derived from historical employee behavior data, historical equipment operation data, and historical abnormal event data, includes: Based on the historical behavior data of the staff, determine the staff behavior weighting coefficient; Based on the device's historical operation data, determine the device interaction weight coefficient; Based on the aforementioned historical abnormal event data, the impact weighting coefficient of the abnormal events is determined; Based on the weight coefficients of the personnel behavior, the weight coefficients of the device interaction, and the weight coefficients of the impact of the abnormal event, the first behavior-related data is obtained.

[0007] Specifically, obtaining the first behavior-related data based on the personnel behavior weight coefficient, the device interaction weight coefficient, and the abnormal event impact weight coefficient includes: Based on the personnel behavior weight coefficient and the cosine similarity between the vector corresponding to the staff's historical behavior data and the vector corresponding to the historical abnormal event data, the first calculation result is obtained. The second calculation result is obtained based on the device interaction weight coefficient, the cosine similarity between the vector corresponding to the device operation history data and the vector corresponding to the historical abnormal event data; The sum of the first calculation result and the second calculation result is calculated to obtain the third calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the third calculation result, the first behavior-related data is obtained.

[0008] Specifically, the second row of associated data, derived from historical environmental state data, historical personnel location data, and historical abnormal event data, includes: Based on the historical environmental data, the weighting coefficients of environmental factors are determined; Based on the historical location data of the personnel, determine the spatial weighting coefficient; Based on the aforementioned historical abnormal event data, the impact weighting coefficient of the abnormal events is determined; Based on the environmental factor weight coefficient, the spatial weight coefficient, and the abnormal event impact weight coefficient, the second row of associated data is obtained.

[0009] Specifically, the step of obtaining the second row of associated data based on the environmental factor weight coefficient, the spatial weight coefficient, and the abnormal event impact weight coefficient includes: Based on the environmental factor weight coefficients and the DTW distance between the time series corresponding to historical environmental state data and the time series corresponding to historical abnormal event data, the fourth calculation result is obtained. Based on the spatial weighting coefficients and the DTW distance between the time series corresponding to the historical personnel location data and the time series corresponding to the historical abnormal event data, the fifth calculation result is obtained; The sum of the fourth calculation result and the fifth calculation result is calculated to obtain the sixth calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the sixth calculation result, the second row of associated data is obtained.

[0010] Specifically, the preset anomaly prediction model includes a first input layer, a behavior feature processing layer, a first fusion layer, a deep feature extraction layer, and a first output layer; By using a pre-set anomaly prediction model, based on the first behavior association data and the second behavior association data, the behavior of staff is predicted to obtain anomaly prediction results, including: Through the first input layer, based on the first row of associated data and the second row of associated data, the first row of associated data input vector and the second row of associated data input vector are obtained; Through the behavior feature processing layer, the first behavior-related data feature vector and the second behavior-related data feature vector are obtained based on the first behavior-related data input vector and the second behavior-related data input vector. The first fusion layer obtains a first fusion feature vector based on the first row-related data feature vector and the second row-related data feature vector. The deep feature extraction layer obtains the deep feature extraction vector based on the first fused feature vector. The anomaly prediction result is obtained by extracting the vector based on the deep feature through the first output layer. The anomaly prediction result includes the predicted anomaly classification and the predicted anomaly probability corresponding to the predicted anomaly classification.

[0011] Specifically, the preset intervention scheme generation model includes a second input layer, a prediction result feature processing layer, a real-time data feature processing layer, a second fusion layer, and a second output layer; A model is generated through a pre-set intervention plan. Based on the anomaly prediction results, real-time behavioral data of the staff, real-time environmental data of the factory, and real-time equipment status data, a target intervention plan is obtained, including: Through the second input layer, based on the predicted anomaly classification, the predicted anomaly probability, the real-time behavior data, the real-time environmental data, and the real-time device status data, the predicted anomaly classification label vector, the predicted anomaly probability scalar, the real-time behavior data tensor, the real-time environmental data tensor, and the real-time device status data tensor are obtained. The prediction result feature vector is obtained through the prediction result feature processing layer based on the prediction anomaly classification label vector and the prediction anomaly probability scalar. Through the real-time data feature processing layer, real-time behavior data feature vectors, real-time environment data feature vectors, and real-time device status data feature vectors are obtained based on the real-time behavior data tensor, the real-time environment data tensor, and the real-time device status data tensor. The second fusion layer obtains a second fusion feature vector based on the prediction result feature vector, the real-time behavior data feature vector, the real-time environmental data feature vector, and the real-time device status data feature vector. The target intervention scheme is obtained through the second output layer based on the second fused feature vector.

[0012] To achieve the above objectives, this application also provides an abnormal behavior monitoring device based on the Industrial Internet of Things, the device comprising: The first unit is used to obtain the first row of related data based on the historical behavior data of staff, the historical operation data of equipment, and the historical abnormal event data. The second unit is used to obtain the second row of associated data based on historical environmental status data, historical personnel location data, and the historical abnormal event data. The third unit is used to predict the behavior of staff members based on the first behavior association data and the second behavior association data by using a preset anomaly prediction model, and to obtain anomaly prediction results. The fourth unit is used to generate a model through a preset intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0013] To achieve the above objectives, this application also provides a terminal, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute the steps in any of the methods provided in this application.

[0014] To achieve the above objectives, this application also provides a medium storing a plurality of instructions adapted for loading by a processor to execute the steps in any of the methods provided in this application.

[0015] This application provides a method, device, terminal, and medium for abnormal behavior monitoring based on the Industrial Internet of Things (IIoT). First, based on historical employee behavior data, equipment operation history data, and historical abnormal event data, first behavior-related data is obtained. Second behavior-related data is obtained based on historical environmental status data, historical personnel location data, and the aforementioned historical abnormal event data. A preset anomaly prediction model is used to predict employee behavior based on the first and second behavior-related data, yielding anomaly prediction results. Finally, a preset intervention scheme generation model is used to generate a target intervention scheme based on the anomaly prediction results, real-time employee behavior data, real-time factory environmental data, and real-time equipment status data. This allows for precise monitoring of employee behavior within the factory, predicting whether abnormal behavior will occur, and thus preventing human-caused safety accidents. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the method provided in the embodiments of this application; Figure 2 A schematic diagram illustrating the specific application implementation process of the preset anomaly prediction model provided in the embodiments of this application; Figure 3 A schematic diagram illustrating the specific application implementation process of the preset intervention scheme generation model provided in this application embodiment; Figure 4 This is a schematic diagram of the device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the terminal structure provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Current technologies for monitoring abnormal human behavior have the following shortcomings: most factories still rely on "computer vision + camera" as their primary monitoring method, but this approach is greatly affected by the environment, making it difficult to guarantee accuracy; video surveillance can only capture visually visible behaviors and cannot detect some non-visual anomalies (such as employees touching high-voltage equipment without permission, failing to clock in after being away from their posts for an extended period, or not following the operating procedures), resulting in blind spots in the monitoring range; at the same time, some "rare abnormal behaviors" in factories (such as employees improperly dismantling safety protection devices or suddenly fainting) have insufficient recognition rates or cannot be predicted through real-time footage, thus failing to generate effective early warnings.

[0019] Therefore, this application provides a method, device, terminal, and medium for monitoring abnormal behavior based on the Industrial Internet of Things to solve practical technical problems.

[0020] In some embodiments, the device may be integrated into an electronic device, such as a terminal or server.

[0021] In some embodiments, the server may also be implemented as a terminal.

[0022] The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0023] The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0024] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0025] This application provides an abnormal behavior monitoring method based on the Industrial Internet of Things. The method can accurately monitor the behavior of workers in the factory, predict whether they will engage in abnormal behavior, and thus avoid human-caused safety accidents.

[0026] In some embodiments, the method can be implemented through a sensor network management system, which includes a management platform, an object platform, and a sensor network platform. The object platform includes sensors, a communication module, and a database. The management platform and the object platform are communicatively connected through the sensor network platform.

[0027] In some embodiments, the management platform is responsible for the management and control of the entire system, including task scheduling and data processing.

[0028] The object platform can refer to an object-oriented design architecture used to manage and process various data objects within the system. The object platform provides a structured way to organize, store, and manipulate data, making the system more flexible and scalable, and storing data in a database.

[0029] The database is used to store historical and real-time data, providing data support for the management platform's decision-making.

[0030] The sensor network platform can serve as an intermediate layer of the sensor network management system, used to collect data from the object platform and to transmit the operator's input instructions from the management platform to the object platform.

[0031] In some embodiments, the Internet of Things (IoT) is the core supporting technology for achieving "all-round perception, real-time interconnection, and data-driven" operation, and its application spans the entire process of data collection, transmission, processing, and intervention. The sensing devices in the Internet of Things (various sensors and smart terminals) are the "nerve endings" of data collection, responsible for capturing multi-dimensional information related to human behavior in real time; The communication technology of the Internet of Things ensures that the collected, scattered data can be transmitted to the processing center in real time and reliably to support subsequent analysis; The IoT platform (edge ​​gateway, cloud platform) is responsible for data integration and preprocessing, providing efficient support for anomaly detection; The interconnectivity of IoT devices extends "anomaly detection" to "proactive intervention," achieving a closed loop of monitoring and response.

[0032] The Internet of Things (IoT) provides full-chain support through "data collection by sensing devices - network transmission and interconnection - platform integration and processing - equipment linkage intervention," enabling the monitoring of abnormal behavior of factory personnel to be upgraded from passive recording to "proactive perception, real-time analysis, and intelligent response." It is the core technological foundation for achieving precise and intelligent safety management.

[0033] A car parts manufacturing workshop needs to monitor workers' equipment operation behavior in real time to prevent safety accidents caused by improper operation, abnormal environment, or abnormal equipment status. The workshop is equipped with personnel behavior recorders, equipment sensors, and environmental monitoring devices (temperature, humidity, dust concentration, etc.), and has accumulated three months of historical data. The workshop needs to use the methods described above to predict and intervene in abnormal behaviors.

[0034] like Figure 1 The specific process of the method can be as follows: S110. Based on staff historical behavior data, equipment operation historical data, and historical abnormal event data, the first row of related data is obtained.

[0035] In some embodiments, obtaining the first row of associated data based on staff historical behavior data, equipment operation history data, and historical abnormal event data includes the steps S111 to S114 as shown below: S111. Based on the historical behavior data of the staff, determine the staff behavior weight coefficient.

[0036] Specifically, the historical behavior data of the staff may include, for example, operation time, number of times the operation steps were compliant, and number of times the operation was erroneous.

[0037] In some embodiments, the "compliance operation rate" (number of compliant operations / total number of operations) and "impact of misoperation" (downtime of equipment due to misoperation / total number of misoperations) of each employee can be calculated. The two indicators are weighted using the Analytic Hierarchy Process (AHP) to obtain the behavior score of each employee. Then, the personnel behavior weight coefficient is determined by normalization (range 0-1). For example, if the average compliance operation rate of workshop employees is 92%, the overall personnel behavior weight coefficient is calculated. The higher the value of the personnel behavior weight coefficient, the greater the impact of personnel behavior on abnormal events.

[0038] S112. Based on the device operation history data, determine the device interaction weight coefficient.

[0039] Specifically, the equipment operation history data may include equipment start / stop frequency, parameter adjustment records, number of fault triggers, etc.

[0040] In some embodiments, the "operational sensitivity" (correlation between parameter adjustment range and failure probability) and "failure correlation" (number of times equipment failure is associated with personnel operation / total number of failures) of the statistical equipment are calculated. The weights of the two indicators are calculated using the entropy method, and then normalized after merging to obtain the equipment interaction weight coefficient. For example, since the operation sensitivity of key welding equipment is high, the equipment interaction weight coefficient is 0.45.

[0041] S113. Based on the historical abnormal event data, determine the impact weight coefficient of the abnormal event.

[0042] In some embodiments, historical abnormal event data may include accident records caused by equipment overload or personnel misoperation, including event type, scope of impact, degree of loss, etc.

[0043] Specifically, events can be categorized into levels (1-5) based on the severity of the loss (economic loss, downtime), with higher levels having greater weight. A weighted average of historical event levels is calculated and normalized to obtain the abnormal event impact weight coefficient. For example, if level 3 or higher accounts for 60% of historical abnormal events in the workshop, the abnormal event impact weight coefficient would be 0.2.

[0044] S114. Based on the personnel behavior weight coefficient, the device interaction weight coefficient, and the abnormal event impact weight coefficient, the first behavior-related data is obtained.

[0045] In some embodiments, obtaining the first behavior-related data based on the personnel behavior weight coefficient, the device interaction weight coefficient, and the abnormal event impact weight coefficient includes: Based on the personnel behavior weight coefficient and the cosine similarity between the vector corresponding to the staff's historical behavior data and the vector corresponding to the historical abnormal event data, the first calculation result is obtained. The second calculation result is obtained based on the device interaction weight coefficient, the cosine similarity between the vector corresponding to the device operation history data and the vector corresponding to the historical abnormal event data; The sum of the first calculation result and the second calculation result is calculated to obtain the third calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the third calculation result, the first behavior-related data is obtained. In some embodiments, the preset first calculation formula is as follows:

[0046] in, This indicates that the first row represents associated data. This represents the total number of historical data points. This indicates the weighting coefficient of the impact of the abnormal event. Indicates a point in time abnormal state, Indicates a point in time An anomaly occurred. Indicates a point in time No abnormalities were found. This represents the weighting coefficient of the personnel behavior. This represents the device interaction weight coefficient. Indicates a point in time Vectors and time points corresponding to historical behavior data of staff at that time The cosine similarity between vectors corresponding to historical anomalous event data at a given time. Indicates a point in time Vectors and time points corresponding to historical device operation data at that time The cosine similarity between vectors corresponding to historical anomalous event data at a given time.

[0047] Specifically, by substituting into the preset first calculation formula, the result is obtained. The higher the value of the first row of related data, the stronger the correlation between personnel-equipment behavior and abnormal events.

[0048] S120. Based on historical environmental status data, historical personnel location data, and the historical abnormal event data, the second row of associated data is obtained.

[0049] In some embodiments, obtaining the second row of associated data based on historical environmental state data, historical personnel location data, and historical abnormal event data includes the steps S121 to S124 as shown below: S121. Based on the historical environmental data, determine the weighting coefficients of environmental factors.

[0050] In some embodiments, the historical environmental data may include temperature, humidity, dust concentration, lighting intensity, etc.

[0051] Specifically, the correlation between the number of times environmental parameters exceed the standard and abnormal events can be analyzed (such as the proportion of equipment failures under high temperature conditions); the correlation strength can be calculated using the Pearson correlation coefficient, and the environmental factor weight coefficient can be obtained after normalization. For example, if the temperature exceeds the standard and the abnormal events are most correlated, then the environmental factor weight coefficient = 0.3.

[0052] S122. Based on the historical location data of the personnel, determine the spatial weighting coefficient.

[0053] In some embodiments, personnel location history data may include the duration of an employee's stay in different areas of the workshop, the distance from hazardous equipment, etc.

[0054] Specifically, the correlation between the frequency of staff staying in "high-risk areas" (such as equipment operation areas) and abnormal events can be statistically analyzed; the spatial influence weight is calculated based on the distance decay model (the closer the distance, the higher the weight), and the spatial weight coefficient is obtained after normalization. For example, if the stay of staff in high-risk areas is strongly correlated with abnormal events, the spatial weight coefficient is 0.5.

[0055] S123. Based on the historical abnormal event data, determine the impact weight coefficient of the abnormal event.

[0056] In some embodiments, the following calculation formula can be used directly: Impact weighting coefficient of abnormal events = 1 - (Environmental factor weighting coefficient + Spatial weighting coefficient) The impact weighting coefficient of the abnormal event was found to be 0.2.

[0057] S124. Based on the environmental factor weight coefficient, the spatial weight coefficient, and the abnormal event impact weight coefficient, the second row of associated data is obtained.

[0058] In some embodiments, obtaining the second row of associated data based on the environmental factor weighting coefficient, the spatial weighting coefficient, and the abnormal event impact weighting coefficient includes the following specific implementation process: Based on the environmental factor weight coefficients and the DTW distance between the time series corresponding to historical environmental state data and the time series corresponding to historical abnormal event data, the fourth calculation result is obtained. Based on the spatial weighting coefficients and the DTW distance between the time series corresponding to the historical personnel location data and the time series corresponding to the historical abnormal event data, the fifth calculation result is obtained; The sum of the fourth calculation result and the fifth calculation result is calculated to obtain the sixth calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the sixth calculation result, the second row of related data is obtained. In some embodiments, the preset second calculation formula is as follows:

[0059] in, This indicates that the second row contains related data. This represents the total number of historical data points. This indicates the weighting coefficient of the impact of the abnormal event. Indicates a point in time abnormal state, Indicates a point in time An anomaly occurred. Indicates a point in time No abnormalities were found. This represents the weighting coefficient of the environmental factors. This represents the spatial weighting coefficient. Indicates a point in time Time series and time points corresponding to historical environmental status data The DTW distance between the time series corresponding to historical anomalous event data. Indicates a point in time Historical data on personnel location corresponding to time series and time points The DTW distance between the time series corresponding to historical anomalous event data. This indicates the preset normalization function.

[0060] In some embodiments, for each time point ,calculate For example, the DTW distance between the environmental temperature fluctuation sequence and the abnormal event occurrence sequence is 5; calculate For example, the DTW distance between the personnel location movement sequence and the abnormal event sequence is 3.

[0061] The normalized values ​​are calculated as Norm(1 / 5) = 0.2 and Norm(1 / 3) = 0.33. If an anomaly occurs in that hour... Then, substituting into the preset second calculation formula, we get... =38.7, the higher the value of the second row of related data, the stronger the correlation between environment-location and abnormal events.

[0062] S130. Using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data to obtain anomaly prediction results.

[0063] In some embodiments, such as Figure 2 The preset anomaly prediction model includes a first input layer, a behavior feature processing layer, a first fusion layer, a deep feature extraction layer, and a first output layer.

[0064] Specifically, by using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data to obtain anomaly prediction results, including the steps S131 to S135 as shown below: S131. Through the first input layer, based on the first row of associated data and the second row of associated data, a first row of associated data input vector and a second row of associated data input vector are obtained.

[0065] In some embodiments, the first behavior-related data and the second behavior-related data are converted into their respective corresponding input vectors.

[0066] S132. Through the behavior feature processing layer, the first behavior-related data feature vector and the second behavior-related data feature vector are obtained based on the first behavior-related data input vector and the second behavior-related data input vector.

[0067] In some embodiments, the behavioral feature processing layer can be a fully connected layer, which maps the input vector into a vector of high-dimensional features through the fully connected layer (with ReLU activation function), i.e.: The first row is the feature vector of the associated data: [0.62, 0.35, 0.81, ...] (128 dimensions); The second row is the feature vector of the associated data: [0.45, 0.72, 0.29, ...] (128 dimensions).

[0068] S133. Through the first fusion layer, a first fusion feature vector is obtained based on the first row-related data feature vector and the second row-related data feature vector.

[0069] In some embodiments, attention weights corresponding to the two feature vectors can be calculated (e.g., the feature weight corresponding to the first row of associated data feature vectors is 0.6, and the feature weight corresponding to the second row of associated data feature vectors is 0.4), and the weights are summed to obtain the first fused feature vector (128 dimensions).

[0070] S134. The deep feature extraction layer obtains the deep feature extraction vector based on the first fused feature vector.

[0071] In some embodiments, the deep feature extraction layer can be an LSTM network layer, which captures the dynamic patterns of the time series in the first fused feature vector and outputs a deep feature extraction vector (64 dimensions).

[0072] S135. The anomaly prediction result is obtained by extracting the vector based on the deep feature through the first output layer, wherein the anomaly prediction result includes the predicted anomaly classification and the predicted anomaly probability corresponding to the predicted anomaly classification.

[0073] In some embodiments, the predicted anomaly can be classified as "device parameter misadjustment", with a predicted anomaly probability of 82%.

[0074] S140. A model is generated by pre-set intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0075] In some embodiments, such as Figure 3 The preset intervention scheme generation model includes a second input layer, a prediction result feature processing layer, a real-time data feature processing layer, a second fusion layer, and a second output layer.

[0076] Specifically, a model is generated through a preset intervention plan. Based on the anomaly prediction results, real-time behavioral data of the staff, real-time environmental data of the factory, and real-time equipment status data of the equipment, a target intervention plan is obtained, including the steps corresponding to S141 to S145 as shown below: S141. Through the second input layer, based on the predicted anomaly classification, the predicted anomaly probability, the real-time behavior data, the real-time environmental data, and the real-time device status data, the predicted anomaly classification label vector, the predicted anomaly probability scalar, the real-time behavior data tensor, the real-time environmental data tensor, and the real-time device status data tensor are obtained.

[0077] In some embodiments, the output corresponding to the second input layer can be: Predicted anomaly classification label vector: [0,1,0,...], (one-hot encoded, corresponding to "device parameter misadjustment"); Predicted anomaly probability scalar: 0.82; Real-time behavioral data tensor: [[Operation speed = 1.2 times / minute, Step compliance = 0.7], ...] (time series tensor); Real-time environmental data tensor: [[Temperature=32℃, Humidity=65%],...]; Real-time equipment status data tensor: [[Current=15A, Pressure=0.8MPa],...].

[0078] S142. Through the prediction result feature processing layer, the prediction result feature vector is obtained based on the predicted anomaly classification label vector and the predicted anomaly probability scalar.

[0079] In some embodiments, the predicted anomaly classification label vector and the predicted anomaly probability scalar are converted into a predicted result feature vector: [0.82,0,1,0,...] (64 dimensions) through the predicted result feature processing layer.

[0080] S143. Through the real-time data feature processing layer, real-time behavior data feature vector, real-time environment data feature vector, and real-time device status data feature vector are obtained based on the real-time behavior data tensor, the real-time environment data tensor, and the real-time device status data tensor.

[0081] In some embodiments, the real-time data feature processing layer performs convolution operations on the three types of real-time data, i.e., extracts local features, to obtain: Real-time behavioral data feature vector (32-dimensional); Real-time environmental data feature vector (32-dimensional); Real-time device status data feature vector (32-dimensional).

[0082] S144. Through the second fusion layer, a second fusion feature vector is obtained based on the prediction result feature vector, the real-time behavior data feature vector, the real-time environmental data feature vector, and the real-time device status data feature vector.

[0083] In some embodiments, the second fusion layer can be used to fuse four types of feature vectors in a splicing manner to obtain a second fused feature vector (64+32+32+32=160 dimensions).

[0084] S145. The target intervention scheme is obtained through the second output layer based on the second fusion feature vector.

[0085] In some embodiments, the targeted intervention program may include: Abnormal behavior warning level: Level 2 (medium risk); Intervention measures: push "equipment parameter specification values" to employee terminals in real time; notify on-site safety personnel to supervise; automatically limit the range of parameter adjustments to a safe range.

[0086] In some embodiments, the second output layer may also adopt a multi-task output structure (corresponding to different dimensions of the intervention plan). For example, the second output layer may include different contents related to the target intervention plan, such as: To handle the warning level: 1 neuron (sigmoid activation, output [0,1] mapped to "low / medium / high" levels); To handle the push notification targets: 3 neurons (softmax activation, corresponding to the probability of "team leader / safety officer / emergency team"); To address intervention measures: 4 neurons (sigmoid activation, corresponding to the activation probability of "voice reminder / on-site interception / equipment shutdown / emergency rescue").

[0087] Therefore, the structured results corresponding to the target intervention plan can include: Abnormal behavior warning level (e.g., "high"); Warning information push recipients and priorities (e.g., "safety officer (0.9) > team leader (0.6)"); On-site intervention measures (e.g., "equipment shutdown (0.95) + safety officer arrival (0.9)"); Subsequent monitoring focus (such as "continuously tracking the person's actions and equipment operating parameters").

[0088] In summary, this application provides an abnormal behavior monitoring method based on the Industrial Internet of Things, which accurately monitors the behavior of workers in the factory, predicts whether they will engage in abnormal behavior, and thus avoids human-caused safety accidents.

[0089] To better implement the above methods, this application also provides an abnormal behavior monitoring device based on the Industrial Internet of Things (IIoT). This device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0090] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the integration of an abnormal behavior monitoring device based on the Industrial Internet of Things into a terminal as an example.

[0091] For example, such as Figure 4 As shown, the abnormal behavior monitoring device 400 based on the Industrial Internet of Things may include a first unit 401, a second unit 402, a third unit 403 and a fourth unit 404. The abnormal behavior monitoring device 500 based on the Industrial Internet of Things includes: Unit 401 is used to obtain the first row of associated data based on staff historical behavior data, equipment operation historical data and historical abnormal event data; The second unit 402 is used to obtain the second row of associated data based on historical environmental status data, historical personnel location data, and the historical abnormal event data. The third unit 403 is used to predict the behavior of staff members based on the first behavior association data and the second behavior association data through a preset anomaly prediction model, and obtain anomaly prediction results. Unit 404 is used to generate a model through a preset intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0092] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0093] As can be seen from the above, the embodiments of this application can accurately monitor the behavior of workers in the factory and predict whether they will engage in abnormal behavior, thereby avoiding human-caused safety accidents.

[0094] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0095] In some embodiments, the product processing device may also be integrated into multiple electronic devices, such as multiple servers, with the multiple servers implementing the industrial Internet of Things-based abnormal behavior monitoring method of this application.

[0096] In this embodiment, the electronic device will be described in detail as a terminal, for example, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the terminal 500 involved in an embodiment of this application. Specifically: The terminal 500 may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more media, a power supply 503, an input module 504, and a communication module 505. Those skilled in the art will understand that... Figure 5 The terminal 500 structure shown does not constitute a limitation on the terminal 500, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 501 is the control center of the terminal 500. It connects various parts of the terminal 500 via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502, it performs various functions of the terminal 500 and processes data, thereby providing overall monitoring of the terminal 500. In some embodiments, the processor 501 may include one or more processing cores; in some embodiments, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 501.

[0097] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal 500, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0098] The terminal 500 also includes a power supply 503 that supplies power to the various components. In some embodiments, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0099] The terminal 500 may also include an input module 504, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0100] The terminal 500 may also include a communication module 505. In some embodiments, the communication module 505 may include a wireless module. The terminal 500 can perform short-range wireless transmission through the wireless module of the communication module 505, thereby providing users with wireless broadband Internet access. For example, the communication module 505 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0101] Although not shown, terminal 500 may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, processor 501 in terminal 500 loads the executable files corresponding to the processes of one or more applications into memory 502 according to the following instructions, and processor 501 runs the applications stored in memory 502 to realize various functions, as follows: Based on staff historical behavior data, equipment operation historical data, and historical abnormal event data, the first row of related data is obtained; Based on historical environmental status data, historical personnel location data, and the aforementioned historical abnormal event data, the second row of associated data is obtained; By using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data, and anomaly prediction results are obtained. A model is generated by pre-setting an intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0102] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0103] As can be seen from the above, the embodiments of this application can accurately monitor the behavior of workers in the factory and predict whether they will engage in abnormal behavior, thereby avoiding human-caused safety accidents.

[0104] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions, or by instructions controlling related hardware. These instructions can be stored in a medium and loaded and executed by a processor.

[0105] To this end, embodiments of this application provide a medium storing multiple instructions that can be loaded by a processor to execute steps in any of the abnormal behavior monitoring methods based on the Industrial Internet of Things provided in embodiments of this application. For example, the instructions can execute the following steps: Based on staff historical behavior data, equipment operation historical data, and historical abnormal event data, the first row of related data is obtained; Based on historical environmental status data, historical personnel location data, and the aforementioned historical abnormal event data, the second row of associated data is obtained; By using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data, and anomaly prediction results are obtained. A model is generated by pre-setting an intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

[0106] The medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0107] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a medium. A processor of a computer device reads the computer instructions from the medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0108] Since the instructions stored in the medium can execute the steps in any of the abnormal behavior monitoring methods based on the Industrial Internet of Things provided in the embodiments of this application, the beneficial effects that any of the abnormal behavior monitoring methods based on the Industrial Internet of Things provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0109] The above provides a detailed description of an abnormal behavior monitoring method, device, terminal, and medium based on the Industrial Internet of Things (IIoT) provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for monitoring abnormal behavior based on the Industrial Internet of Things, characterized in that, The method includes: Based on staff historical behavior data, equipment operation historical data, and historical abnormal event data, the first row of related data is obtained; Based on historical environmental status data, historical personnel location data, and the aforementioned historical abnormal event data, the second row of associated data is obtained; By using a preset anomaly prediction model, the behavior of staff is predicted based on the first behavior association data and the second behavior association data, and anomaly prediction results are obtained. A model is generated by pre-setting an intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

2. The method as described in claim 1, characterized in that, The first row of associated data, derived from historical employee behavior data, historical equipment operation data, and historical abnormal event data, includes: Based on the historical behavior data of the staff, determine the staff behavior weighting coefficient; Based on the device's historical operation data, determine the device interaction weight coefficient; Based on the aforementioned historical abnormal event data, the impact weighting coefficient of the abnormal events is determined; Based on the weight coefficients of the personnel behavior, the weight coefficients of the device interaction, and the weight coefficients of the impact of the abnormal event, the first behavior-related data is obtained.

3. The method as described in claim 2, characterized in that, The first behavior-related data is obtained based on the personnel behavior weight coefficient, the device interaction weight coefficient, and the abnormal event impact weight coefficient, including: Based on the personnel behavior weight coefficient and the cosine similarity between the vector corresponding to the staff's historical behavior data and the vector corresponding to the historical abnormal event data, the first calculation result is obtained. The second calculation result is obtained based on the device interaction weight coefficient, the cosine similarity between the vector corresponding to the device operation history data and the vector corresponding to the historical abnormal event data; The sum of the first calculation result and the second calculation result is calculated to obtain the third calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the third calculation result, the first behavior-related data is obtained.

4. The method as described in claim 1, characterized in that, The second row of associated data, derived from historical environmental status data, historical personnel location data, and historical abnormal event data, includes: Based on the historical environmental data, the weighting coefficients of environmental factors are determined; Based on the historical location data of the personnel, determine the spatial weighting coefficient; Based on the aforementioned historical abnormal event data, the impact weighting coefficient of the abnormal events is determined; Based on the environmental factor weight coefficient, the spatial weight coefficient, and the abnormal event impact weight coefficient, the second row of associated data is obtained.

5. The method as described in claim 4, characterized in that, The second row of associated data is obtained based on the environmental factor weight coefficient, the spatial weight coefficient, and the abnormal event impact weight coefficient, including: Based on the environmental factor weight coefficients and the DTW distance between the time series corresponding to historical environmental state data and the time series corresponding to historical abnormal event data, the fourth calculation result is obtained. Based on the spatial weighting coefficients and the DTW distance between the time series corresponding to the historical personnel location data and the time series corresponding to the historical abnormal event data, the fifth calculation result is obtained; The sum of the fourth calculation result and the fifth calculation result is calculated to obtain the sixth calculation result; Based on the impact weight coefficient of the abnormal event, the abnormal state corresponding to each time point in the historical abnormal event data, and the sixth calculation result, the second row of associated data is obtained.

6. The method as described in claim 1, characterized in that, The preset anomaly prediction model includes a first input layer, a behavior feature processing layer, a first fusion layer, a deep feature extraction layer, and a first output layer; By using a pre-set anomaly prediction model, based on the first behavior association data and the second behavior association data, the behavior of staff is predicted to obtain anomaly prediction results, including: Through the first input layer, based on the first row of associated data and the second row of associated data, the first row of associated data input vector and the second row of associated data input vector are obtained; Through the behavior feature processing layer, the first behavior-related data feature vector and the second behavior-related data feature vector are obtained based on the first behavior-related data input vector and the second behavior-related data input vector. The first fusion layer obtains a first fusion feature vector based on the first row-related data feature vector and the second row-related data feature vector. The deep feature extraction layer obtains the deep feature extraction vector based on the first fused feature vector. The anomaly prediction result is obtained by extracting the vector based on the deep feature through the first output layer. The anomaly prediction result includes the predicted anomaly classification and the predicted anomaly probability corresponding to the predicted anomaly classification.

7. The method as described in claim 6, characterized in that, The preset intervention scheme generation model includes a second input layer, a prediction result feature processing layer, a real-time data feature processing layer, a second fusion layer, and a second output layer; A model is generated through a pre-set intervention plan. Based on the anomaly prediction results, real-time behavioral data of the staff, real-time environmental data of the factory, and real-time equipment status data, a target intervention plan is obtained, including: Through the second input layer, based on the predicted anomaly classification, the predicted anomaly probability, the real-time behavior data, the real-time environmental data, and the real-time device status data, the predicted anomaly classification label vector, the predicted anomaly probability scalar, the real-time behavior data tensor, the real-time environmental data tensor, and the real-time device status data tensor are obtained. The prediction result feature vector is obtained through the prediction result feature processing layer based on the prediction anomaly classification label vector and the prediction anomaly probability scalar. Through the real-time data feature processing layer, real-time behavior data feature vectors, real-time environment data feature vectors, and real-time device status data feature vectors are obtained based on the real-time behavior data tensor, the real-time environment data tensor, and the real-time device status data tensor. The second fusion layer obtains a second fusion feature vector based on the prediction result feature vector, the real-time behavior data feature vector, the real-time environmental data feature vector, and the real-time device status data feature vector. The target intervention scheme is obtained through the second output layer based on the second fused feature vector.

8. An abnormal behavior monitoring device based on the Industrial Internet of Things, characterized in that, The device includes: The first unit is used to obtain the first row of related data based on the historical behavior data of staff, the historical operation data of equipment, and the historical abnormal event data. The second unit is used to obtain the second row of associated data based on historical environmental status data, historical personnel location data, and the historical abnormal event data. The third unit is used to predict the behavior of staff members based on the first behavior association data and the second behavior association data by using a preset anomaly prediction model, and to obtain anomaly prediction results. The fourth unit is used to generate a model through a preset intervention plan. Based on the anomaly prediction results, the real-time behavior data of the staff, the real-time environmental data of the factory, and the real-time equipment status data of the equipment, a target intervention plan is obtained. The target intervention plan includes an abnormal behavior warning level and an intervention measure plan.

9. A terminal, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A medium, characterized in that, The medium stores a plurality of instructions adapted for loading by a processor to execute the steps of the method according to any one of claims 1 to 7.

Images