Industrial operation compliance monitoring method and system

Abstract

The invention discloses an industrial operation compliance monitoring method and system, and belongs to the technical field of intelligent manufacturing and industrial automation. The method comprises the following steps: acquiring industrial operation data of employees by using a multi-modal sensor; constructing a digital twinborn model in a layered manner, and mapping the industrial operation data to the digital twinborn model; comparing the industrial operation data with standard operation data by adopting a dynamic time warping algorithm so as to detect industrial operation deviation; and constructing a behavior intention reasoning network based on a Transform architecture, fusing operation time sequence characteristics, environment state parameters and historical behavior data, and distinguishing the industrial operation deviations through multi-task learning so as to identify malicious operations. According to the method, the problems that traditional manual inspection is low in efficiency and malicious operation is difficult to identify are solved, and intelligent closed-loop management of industrial operation compliance can be realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing and industrial automation technology, and in particular to a method and system for monitoring industrial operation compliance. Background Art

[0002] At a time when industry and intelligent manufacturing are deeply integrated, industrial operation compliance monitoring and analysis has become a core supporting technology to ensure production safety risk prevention and control, process quality consistency, and manufacturing efficiency optimization. It can effectively prevent errors and identify risks, providing critical closed-loop guarantees for industrial digital transformation.

[0003] However, current industrial operation compliance monitoring technologies / solutions still have the following problems:

[0004] 1) Lack of quantitative analysis of motion trajectories: Traditional video surveillance systems can only record two-dimensional images and lack millimeter-level precision quantitative analysis of three-dimensional operation trajectories. Existing key point detection-based algorithms, such as the Open Pose algorithm, have high errors in joint positioning in complex industrial scenarios and are unable to analyze the semantics of tool postures, such as wrench tightening angle deviation.

[0005] 2) Weak intention discrimination capability: Existing operation anomaly detection algorithms are mostly based on rule engines or isolation forest models. They can only identify explicit deviations, such as action timeouts, but are not accurate enough in distinguishing between erroneous operations and intentional violations. For example, malicious operations are often disguised as normal action sequences, such as deliberately delaying the execution time of key steps. Existing technologies find it difficult to capture their hidden characteristics.

[0006] 3) Lagging real-time correction mechanism: Feedback on violations relies on manual review afterwards, and the average response delay is long. In addition, traditional AR projection technology has problems such as large spatial registration errors and poor lighting adaptability, which leads to misalignment between guidance information and physical scenes.

[0007] 4) Inefficient data fusion: Data synchronization accuracy is insufficient for multi-source sensors such as IMUs, vision sensors, and force sensors, leading to a dynamic mismatch between the digital twin model and actual operations. Furthermore, existing data fusion algorithms, such as the Kalman filter, experience a 40% reduction in filter convergence speed in unstructured scenarios, impacting data real-time performance. Summary of the Invention

[0008] The present application provides a method and system for monitoring industrial operation compliance, which are used to solve at least one of the above technical problems.

[0009] This application adopts the following technical solutions:

[0010] On the one hand, the present application provides a method for monitoring industrial operation compliance, which includes: using multimodal sensors to collect employees' industrial operation data; hierarchically constructing a digital twin model and mapping the industrial operation data to the digital twin model; using a dynamic time warping algorithm to compare the industrial operation data with standard operation data to detect industrial operation deviations; constructing a behavioral intention inference network based on the Transformer architecture, integrating operation timing characteristics, environmental state parameters and historical behavior data, and distinguishing the industrial operation deviations through multi-task learning to identify malicious operations.

[0011] In one possible implementation of the present application, the multimodal sensor includes at least one or more of a millimeter-wave radar array, a stereo depth camera system, and a flexible force sensing glove; the multimodal sensor is used to collect industrial operation data of employees, including: using the millimeter-wave radar array to collect 3D point cloud data to obtain the employee's hand micro-movements and / or hand movement trajectories in real time; using the stereo depth camera system to collect the employee's hand depth map; and using the flexible force sensing glove to measure the employee's tool grip force and torque.

[0012] In a possible implementation of the present application, the industrial operation data of employees is collected using a multimodal sensor, which also includes: constructing a 26-degree-of-freedom hand motion model based on the employee's hand depth map and combined with a hand skeleton key point detection algorithm.

[0013] In one possible implementation of the present application, a digital twin model is constructed in layers, including: a physical layer, which builds a high-fidelity tool model based on the Unity3D engine, imports industrial operation data, and calculates the operating torque deviation through rigid body dynamics simulation; a behavioral layer, which uses a temporal graph convolutional network to parse the employee's hand motion sequence, decomposes the industrial operation into atomic action units, and associates each unit with a standard time threshold and spatial trajectory constraint; a semantic layer, which builds an industrial operation association model through a knowledge graph and maps industrial operation steps to quality system clauses.

[0014] In one possible implementation of the present application, a behavioral intention inference network based on the Transformer architecture is constructed, including: optimizing the dual-stream Transformer architecture, including: timing stream, using time-sliced ​​LSTM to extract employee hand motion features, inputting a multi-head self-attention mechanism to calculate cross-step dependencies, and capturing abnormal operations; environmental stream, accessing PLC real-time data, extracting device status features through 1D-CNN, cross-attention fusion with operation timing features, and identifying environment-induced deviation operations.

[0015] In one possible implementation of the present application, constructing a behavior intention inference network based on the Transformer architecture also includes: injecting three types of adversarial samples, namely timing perturbations, spatial offsets, and tool changes, into the optimized dual-stream Transformer architecture to perform adversarial training on the behavior intention inference network.

[0016] In one possible implementation of the present application, the industrial operation deviations are distinguished through multi-task learning to identify malicious operations, including: constructing a malicious operation feature library, which includes at least multiple malicious operation feature indicators; using a multi-task loss function to jointly optimize the malicious operation classification and error operation correction regression tasks, and output the malicious operation probability.

[0017] In one possible implementation of the present application, after identifying a malicious operation, the method further includes: projecting a three-dimensional operation guidance animation through an AR real-time projection system, and optimizing the projection position and content based on a reinforcement learning algorithm.

[0018] In one possible implementation of the present application, the projection position and content are optimized, including: dynamically adjusting the projection position based on eye tracking data through an offset compensation formula; adaptively adjusting the projection transparency according to the ambient light intensity; and superimposing a red flashing warning box for malicious operations and / or erroneous operations that exceed a preset level.

[0019] On the other hand, the present application also provides an industrial operation compliance monitoring system, which includes: a multimodal data acquisition module, which uses multimodal sensors to collect employees' industrial operation data; a data twin model construction module, which hierarchically constructs a digital twin model and maps the industrial operation data to the digital twin model; a behavioral intention reasoning module, which uses a dynamic time warping algorithm to compare the industrial operation data with the standard operation data to detect industrial operation deviations; the behavioral intention reasoning module constructs a behavioral intention reasoning network based on the Transformer architecture, integrates operation timing characteristics, environmental state parameters and historical behavior data, and distinguishes the industrial operation deviations through multi-task learning to identify malicious operations.

[0020] The present application provides a method and system for monitoring industrial operation compliance, which has the following beneficial effects:

[0021] 1) Improved performance of high-precision operation trajectory analysis: Motion capture accuracy is improved, key point coordinate errors are reduced, trajectory comparison efficiency is improved, time consumption is reduced, and the false alarm rate is greatly reduced.

[0022] 2) Breakthrough in malicious operation identification capabilities: The accuracy of malicious intent identification is greatly improved compared to traditional models. Time series characteristics and data correlation analysis greatly improve early warning capabilities and significantly enhance anti-interference capabilities.

[0023] 3) AR real-time guidance performance optimization: projection accuracy is improved, response delay is reduced, environmental adaptability is enhanced, and the human-computer real-time interaction experience is greatly improved.

[0024] 4) Comprehensive improvement in production efficiency: effectively reduce the rate of illegal operations, cleverly save training costs through AR real-time guidance, and enhance quality traceability capabilities.

[0025] 5) System robustness and scalability: High adaptability to multiple scenarios, optimized resource usage, meeting the low resource constraint requirements of industrial sites, and reducing maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in this application or the prior art, the following briefly introduces the drawings required for the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments described in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0027] Figure 1 A flow chart of a method for monitoring compliance of industrial operations provided for this application;

[0028] Figure 2 This application provides an industrial operation compliance monitoring system architecture diagram. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solutions of this application, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0030] The present application proposes a method and system for monitoring industrial operation compliance, which belongs to the field of intelligent manufacturing and industrial automation technology. Employee operation data is collected through multimodal sensors and synchronously mapped to a digital twin model. A dynamic time warping algorithm is used to compare standard operation trajectories to achieve millimeter-level motion deviation detection. An innovative behavior intention inference network based on the Transformer architecture is constructed, integrating operation timing characteristics, environmental state parameters, and historical behavior data, and distinguishing between erroneous operations and malicious operations through multi-task learning. When a violation is detected, that is, when an erroneous operation and / or malicious operation is monitored, a three-dimensional operation guidance animation is projected in real time through devices such as AR glasses, and the projection position and content are dynamically optimized based on reinforcement learning. This solves the problem of low efficiency of traditional manual inspections and difficulty in identifying malicious operations, and helps to achieve intelligent closed-loop management of operation compliance.

[0031] The method in this application is described in detail below with reference to the accompanying drawings.

[0032] Figure 1 A flow chart of a method for monitoring compliance of industrial operations provided for this application, such as Figure 1 As shown, the monitoring method in this application includes at least the following execution steps:

[0033] Step 101: Collect employees' industrial operation data using multimodal sensors.

[0034] In one possible implementation of the present application, the multimodal sensor includes at least one or more of a millimeter-wave radar array, a stereo depth camera system, and a flexible force sensing glove to collect industrial operation data, specifically,

[0035] Millimeter-wave radar array: Deploys four sets of 60GHz FMCW radars, using MIMO antennas to form a 3D point cloud with a resolution of 0.5° × 0.5°. This allows real-time capture of employees' hand micro-movements and / or hand movement trajectories. The frequency range is 0.1-10Hz, and the anti-occlusion capability is 80% higher than that of pure vision solutions.

[0036] Stereo depth camera system: This system uses a binocular TOF camera with VGA resolution and a frame rate of 120fps, working in conjunction with a structured light projector to generate a high-precision depth map with an error of ±0.3mm. Combined with a hand skeleton key point detection algorithm, such as the improved HRNet algorithm, it reconstructs a 26-DOF hand motion model and obtains the hand motion trajectory.

[0037] Flexible force sensing gloves: integrated PVDF piezoelectric film sensor with a sensitivity of 50mV / N, measuring tool grip force in the range of 0-100N, and measuring torque with an accuracy of ±5Nm. The data is transmitted to the edge computing unit via Bluetooth 5.3.

[0038] Through the above multiple or multiple types of sensors, multi-modal data is collected to form industrial operation data.

[0039] Step 102: construct a digital twin model in layers and map the industrial operation data to the digital twin model.

[0040] After collecting industrial operation data, a digital twin model is constructed and the collected industrial operation data is mapped to the digital twin model.

[0041] In one possible implementation of the present application, the constructed digital twin model has three layers, so it is preferably constructed in layers, specifically:

[0042] Physical layer: Build a high-fidelity tool model based on the Unity3D engine, import industrial operation data, preferably import CAD parametric data, such as wrench tooth pitch, torque threshold, etc., and calculate the operating torque deviation through rigid body dynamics simulation with an error rate of <2%.

[0043] Behavior layer: A temporal graph convolutional network, or ST-GCN, is used to analyze employee hand motion sequences and decompose operations into atomic action units, such as "grasp-position-tighten." Each atomic action unit is associated with a standard time threshold, such as μ±3σ, and a spatial trajectory constraint, such as a Hausdorff distance of <0.1.

[0044] Semantic layer: Build an industrial operation association model through the knowledge graph, map the operation steps to the ISO 9001 quality system clauses, and trigger an L2 alarm if step A violates clause 4.3.

[0045] Those skilled in the art can understand that the specific mapping process of mapping industrial operation data into the digital twin model can be implemented through existing mapping methods or mapping algorithms, which will not be elaborated in this application.

[0046] Step 103: Use a dynamic time warping algorithm to compare the industrial operation data with the standard operation data to detect industrial operation deviations.

[0047] Step 104: Build a behavioral intention inference network based on the Transformer architecture, integrate operation timing characteristics, environmental state parameters, and historical behavior data, and distinguish industrial operation deviations through multi-task learning to identify malicious operations.

[0048] In one possible implementation of the present application, after the industrial operation data is successfully mapped to the digital twin model, malicious operations can be identified through the behavioral intention reasoning network.

[0049] Specifically, the two-stream Transformer architecture is first optimized. The optimization process mainly includes: first, the timing stream: using time-sliced ​​LSTM, preferably with a time window of 1s and a step size of 0.2s, to extract the employee's hand movement features, input the multi-head self-attention mechanism to calculate the cross-step dependencies, and capture abnormal operations such as periodic pauses; second, the environmental stream: access PLC real-time data, such as pressure, temperature, current and other data, extract the equipment status features through 1D-CNN, and perform cross-attention fusion with the operation timing features to identify environment-induced misoperations, such as high temperature causing tool slippage; third, adversarial training strategy: inject three types of adversarial samples - timing perturbation (randomly inserting a 0.5s delay), spatial perturbation (tool position offset ±5cm), and semantic perturbation (replacing non-standard tools), in order to perform adversarial training on the behavior intention reasoning network and improve the robustness of the behavior intention reasoning network.

[0050] Furthermore, after the optimization is completed, a malicious operation discrimination engine is introduced. First, a malicious operation feature library is constructed, and 12 types of malicious features are defined, including abnormal tool switching frequency (exceeding the process specification by 3σ), abnormal shortening of the key process dwell time (<50% of the standard value), sudden increase in the Hausdorff distance of the operation path (Δ>0.15), tool applied force / torque (>20% of the standard value for 3 seconds), sudden increase in the operator's heart rate variability (HRV) by >30%, number of errors in the same process (≥3 times / hour), environmental induced anomalies (operating parameter-equipment state deviation correlation coefficient <-0.7), periodic peak of the action interval autocorrelation function >0.8, tool inclination deviation (>5° within the safety range for 2 seconds), number of unauthorized equipment accesses (≥2 times / shift), sensor data mutation gradient (second-order derivative >3σ), multi-employee action synchronization error (>300ms standard timing), etc. Then, a multi-task loss function is used to enable the behavioral intention inference network to learn, jointly optimize malicious operation classification (cross entropy loss) and error operation regression (mean square error), and prevent inter-task interference through the gradient mask mechanism. Finally, the malicious probability P_malice∈[0,1] is output to identify malicious operations.

[0051] Furthermore, when violations such as malicious operations are detected, the present application can also project three-dimensional operation guidance animations in real time through devices such as AR glasses, and dynamically optimize the projection position and content based on reinforcement learning. In one example, the process can be configured as an AR real-time projection system. In the system, the hardware configuration includes: optical waveguide AR glasses equipped with a Micro-OLED display with a resolution of 2560×2560 and a brightness of 1000nit. It adopts a BirdBath optical solution, with a field of view of preferably 50°, supporting outdoor strong light environments, such as clear display in environments with light intensity >10,000 lux; an eye tracking unit with an integrated Tobii eye tracker, a sampling rate of 120Hz, real-time tracking of gaze point coordinates, and a tracking accuracy of ±0.5°, which is used to dynamically adjust the guidance projection position. The system's software algorithms include: 3D registration optimization, using the ORB-SLAM3 algorithm to build an environmental map, combining IMU data for pose estimation, with an estimation error of less than 2mm, and generating keyframe animations of projected content through Bezier curve interpolation at a frame rate of 60fps; an adaptive rendering engine, dynamically adjusting the projection transparency from 10% to 90% based on the ambient light intensity of 0-100,000 lux, using YOLOv5 for real-time detection of obstructions such as palms, and automatically switching the projection plane, such as switching from the tool surface to the projection of a neighboring device; a reinforcement learning strategy, building a DQN model to optimize the projection strategy, with the state space including the operator's skill level (historical error rate), environmental complexity (number of devices), and attention concentration (eye divergence). The reward function comprehensively guides effectiveness and interference, where the author's skill level is represented by the historical error rate, the environmental complexity is represented by the number of devices, and the attention concentration is represented by eye divergence; and effectiveness is represented by the error correction time, and interference is represented by the duration of line of sight deviation.

[0052] In addition, to protect data security, this application also proposes a trusted data governance framework. Based on data security, it builds a dual-channel mechanism for employee privacy protection to achieve a balance between compliant data utilization and privacy protection. Specifically, this framework mainly includes:

[0053] Biometric desensitization processing:

[0054] ① Dynamic pseudonymization: When collecting biometric data such as eye movement tracks and electromyographic signals, a temporary identifier is generated through AES-256 encryption with a validity period of ≤8 hours. When storing data, a virtual ID is associated with the employee rather than the employee's real identity.

[0055] ② Differential privacy injection: During the behavioral intention inference network training phase, Laplace noise (ε=0.3, δ=1e-6) is added to the original biometric data to ensure the untraceability of individual data (k-anonymity ≥ 5).

[0056] ③ Collaboration with the behavioral intention inference network: Desensitized biometric data is input into the two-stream Transformer model through a secure channel, ensuring that the accuracy rate of malicious operation identification drops by ≤1.2% (benchmark test data).

[0057] Federated learning collaborative modeling:

[0058] ① Localized model training: Employee operation data is retained in the edge computing node, and only encrypted model gradient parameters are uploaded. The Paillier homomorphic encryption algorithm is used, and the central server aggregates and updates the global malicious operation identification model.

[0059] ② Gradient obfuscation mechanism: A random mask matrix is ​​superimposed on the uploaded gradient parameters, and the mask value range is preferably [-0.1, 0.1] to prevent the reconstruction of the original data through gradient inversion attacks.

[0060] ③ Linkage with AR projection system: Under the federated learning framework, personalized AR guidance parameters, such as projection response speed, are adjusted only based on local encryption features to avoid the leakage of private data.

[0061] Data lifecycle control:

[0062] ① Intelligent Forgetting Protocol: Set automatic data clearing rules, such as triggering SHA-3 hash erasure after 30 days of operation record retention, and setting the storage period of key privacy fields, such as biometric fields, to ≤24 hours.

[0063] ② Minimize data collection principle: Through feature selection algorithms, such as the L1 regularization algorithm, dynamic shielding of non-essential data collection items is achieved. For example, eye movement data is not collected when AR guidance is not enabled.

[0064] Compliance Audit Module:

[0065] ① Blockchain evidence storage and traceability: Operation records are uploaded to the chain in a Merkle tree structure (Hyperledger Fabric framework). Each record contains a data hash value, a collection timestamp, and a processing permission identifier, such as an RBAC role code.

[0066] ② Privacy Impact Assessment: Deploy a lightweight PIA engine to monitor the privacy leakage risk level in the data flow path in real time. For example, it can quantify the score based on the CVSS 3.1 standard. When the risk value is greater than 7, the circuit breaker mechanism will be triggered to suspend data output.

[0067] Through this trusted data governance framework, data security and privacy are guaranteed, making data credibility high, audit efficiency improved, privacy leakage risks reduced, and privacy protection strength and data security improved.

[0068] Based on the same inventive concept, the present application also provides a monitoring system for industrial operation compliance, the architecture of which is as follows: Figure 2 shown.

[0069] Figure 2 This is an architectural diagram of an industrial operation compliance monitoring system provided in this application. Figure 2 As shown, the industrial operation compliance monitoring system 20 in this application specifically includes:

[0070] A multimodal data acquisition module 201 collects industrial operation data of employees using multimodal sensors;

[0071] A data twin model construction module 202 constructs a digital twin model in layers and maps the industrial operation data to the digital twin model;

[0072] The behavioral intention inference module 203 uses a dynamic time warping algorithm to compare the industrial operation data with standard operation data to detect industrial operation deviations;

[0073] The behavior intention reasoning module 203 constructs a behavior intention reasoning network based on the Transformer architecture, integrates operation timing characteristics, environmental state parameters and historical behavior data, and distinguishes the industrial operation deviations through multi-task learning to identify malicious operations.

[0074] Those skilled in the art will appreciate that the industrial operation compliance monitoring system in this application can also be configured to include:

[0075] Multimodal sensing unit: This includes a millimeter-wave radar array, a binocular TOF depth camera, and a flexible force sensing glove, which is used to simultaneously collect hand motion trajectory, tool posture, and operating torque data;

[0076] Digital twin modeling module: Builds a three-layer model consisting of the physical layer (rigid body dynamics simulation), the behavioral layer (analyzing atomic action units using a time-series graph convolutional network), and the semantic layer (process knowledge graph).

[0077] Behavioral Intention Inference Network: Based on a dual-stream Transformer architecture, it integrates operation timing features with device state parameters to output the probability of malicious operations.

[0078] AR real-time projection system: equipped with optical waveguide AR glasses and eye tracking module, dynamically generates three-dimensional operation instructions;

[0079] Data security module: realizes privacy protection under the blockchain evidence storage and federated learning framework.

[0080] Based on any of the above implementations, the core goal of this application is to build a full-link intelligent closed-loop control mechanism of "perception-reasoning-intervention", specifically:

[0081] 1) Establish an ultra-precise motion mapping system: Through multimodal sensor spatiotemporal calibration (time synchronization error <1ms) and skeleton-tool joint modeling (accuracy ±0.5mm), submillimeter dynamic mapping of operating actions and digital twins is achieved, solving the positioning drift problem of traditional visual SLAM in tool occlusion scenarios.

[0082] 2) Develop a malicious operation feature engine: Integrate temporal behavior patterns (DTW dynamic time warping), environmental state associations (PLC device parameters), and biometric features (eye tracking, electromyography) to build a multimodal malicious operation fingerprint library to achieve early warning of hidden violations.

[0083] 3) Design an adaptive AR projection system: Optimize projection parameters (brightness, position, angle, content) based on reinforcement learning, achieve a projection positioning error of less than 2mm in a dynamic environment (using vision-IMU tight coupling calibration), and support multimodal interaction (gesture-triggered operation playback, voice command call standard procedures, etc.).

[0084] 4) Build a trusted data governance framework: Use blockchain technology to store operational data on-chain (SHA-256 hash encryption), combine it with federated learning to protect employee privacy, and ensure the audit traceability and data security of the compliance analysis process.

[0085] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0086] The system and method provided in this application correspond one to one, so the system also has similar beneficial technical effects to its corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system will not be repeated here.

[0087] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0088] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0089] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present application.

Claims

1. A method for monitoring compliance of industrial operations, characterized in that: The method comprises: Use multimodal sensors to collect employees' industrial operation data; constructing a digital twin model in layers, and mapping the industrial operation data to the digital twin model; Comparing the industrial operation data with standard operation data using a dynamic time warping algorithm to detect industrial operation deviations; A behavioral intention inference network based on the Transformer architecture is constructed, which integrates operation timing characteristics, environmental state parameters and historical behavior data. Through multi-task learning, the industrial operation deviations are distinguished to identify malicious operations.

2. The method for monitoring compliance of industrial operations according to claim 1, characterized in that: The multimodal sensor includes at least one or more of a millimeter wave radar array, a stereo depth camera system, and a flexible force sensing glove; Multimodal sensors are used to collect industrial operation data from employees, including: Utilizing the millimeter-wave radar array to collect 3D point cloud data to obtain the employee's hand micro-movements and / or hand movement trajectories in real time; Using the stereo depth camera system to collect a depth map of the employee's hand; The flexible force sensing gloves are used to measure the tool gripping force and torque of employees.

3. The method for monitoring compliance of industrial operations according to claim 2, characterized in that: Multimodal sensors are used to collect industrial operation data from employees, including: Based on the employee's hand depth map and combined with the hand skeleton key point detection algorithm, a 26-degree-of-freedom hand motion model is constructed.

4. The method for monitoring compliance of industrial operations according to claim 1, characterized in that: The digital twin model is constructed in layers, including: At the physical layer, a high-fidelity tool model is built based on the Unity3D engine, industrial operation data is imported, and the operating torque deviation is calculated through rigid body dynamics simulation; The behavioral layer uses a temporal graph convolutional network to analyze employee hand motion sequences and decompose industrial operations into atomic action units. Each unit is associated with a standard time threshold and spatial trajectory constraints. The semantic layer builds an industrial operation association model through the knowledge graph and maps industrial operation steps to quality system clauses.

5. The method for monitoring compliance of industrial operations according to claim 1, characterized in that: Build a behavioral intention inference network based on the Transformer architecture, including: Optimize the two-stream Transformer architecture, including: Time series flow, using time-slicing LSTM to extract employee hand movement features, input multi-head self-attention mechanism to calculate cross-step dependencies and capture abnormal operations; The environmental flow is connected to the real-time PLC data, and the device status features are extracted through 1D-CNN. The cross-attention fusion is performed with the operation timing features to identify the environment-induced deviation operations.

6. The method for monitoring compliance of industrial operations according to claim 5, characterized in that: Building a behavioral intention inference network based on the Transformer architecture also includes: In the optimized two-stream Transformer architecture, three types of adversarial samples, namely timing perturbation, spatial offset and tool replacement, are injected to perform adversarial training on the behavioral intention inference network.

7. The method for monitoring compliance of industrial operations according to claim 6, characterized in that: Multi-task learning is used to distinguish industrial operation deviations to identify malicious operations, including: Constructing a malicious operation feature library, wherein the malicious operation feature library includes at least a plurality of malicious operation feature indicators; A multi-task loss function is used to jointly optimize the malicious operation classification and error operation correction regression tasks, and output the malicious operation probability.

8. The method for monitoring compliance of industrial operations according to claim 1, characterized in that: After identifying the malicious operation, the method further includes: The three-dimensional operation guidance animation is projected through the AR real-time projection system, and the projection position and content are optimized based on the reinforcement learning algorithm.

9. The method for monitoring compliance of industrial operations according to claim 8, characterized in that: Optimize projection location and content, including: Dynamically adjust the projection position based on eye tracking data through an offset compensation formula; Adaptively adjust the projection transparency according to the ambient light intensity; A red flashing warning box will be superimposed for malicious operations and / or erroneous operations that exceed the preset level.

10. A system for monitoring compliance of industrial operations, characterized in that: The system comprises: Multimodal data acquisition module, which uses multimodal sensors to collect employees' industrial operation data; A data twin model construction module is used to construct a digital twin model in layers and map the industrial operation data to the digital twin model; a behavioral intention inference module that uses a dynamic time warping algorithm to compare the industrial operation data with standard operation data to detect industrial operation deviations; The behavioral intention reasoning module constructs a behavioral intention reasoning network based on the Transformer architecture, integrates operation timing characteristics, environmental state parameters and historical behavior data, and distinguishes the industrial operation deviations through multi-task learning to identify malicious operations.

Images