A worker management method and system based on multimodal perception and intelligent collaboration
By constructing a worker management system based on multimodal perception and intelligent collaboration, the problems of difficult personnel mobility, passive safety management, and data silos in traditional worker management have been solved. It has achieved reliable data storage, risk warning, efficient training, and business collaboration, thereby improving management efficiency and security.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA RAILWAY NO 9 BUREAU GRP NO 1 CONSTR CO LTD
- Filing Date
- 2026-01-28
- Publication Date
- 2026-06-02
AI Technical Summary
In labor-intensive industries, traditional worker management models suffer from difficulties in managing personnel mobility, passive safety management, insufficient data credibility, and data silos, resulting in low efficiency in matching workers, numerous safety hazards, frequent labor disputes, and low management efficiency.
We will build a worker management system based on multimodal perception and intelligent collaboration, and adopt consortium blockchain notarization, multimodal data fusion, VR training and dynamic digital twins to achieve reliable data notarization, intelligent risk early warning, precise and efficient training and business collaboration.
It has enabled the reliable storage of worker data, reduced security risks, improved management efficiency and training effectiveness, broken down data silos, and achieved intelligent collaborative decision-making across systems.
Smart Images
Figure CN122133960A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial worker management technology, specifically to a worker management method and system based on multimodal perception and intelligent collaboration. Background Technology
[0002] In labor-intensive industries such as construction and infrastructure, the management of industrial workers has long faced multiple industry-specific pain points, and with the increasing trend of project scaling and flexible employment, the limitations of traditional management models are becoming more and more prominent.
[0003] First, the management challenges brought about by the mobility of personnel are particularly prominent. Workers often move between projects and regions, and their work trajectories are scattered. Traditional paper files and decentralized electronic records have problems such as untimely updates and easy loss, making it difficult for employers to quickly verify key information such as workers' skills and qualifications, safety training history, and past work performance. This not only affects the efficiency of labor matching, but may also create safety hazards for project construction by hiring personnel who do not meet the skill standards or have poor safety records.
[0004] Secondly, safety management suffers from a significant "passive" shortcoming. Construction sites are complex environments with a high concentration of high-risk scenarios such as high-altitude operations, machinery operation, and chemical use. Traditional safety management relies on manual inspections, which not only have limited coverage and are susceptible to human factors leading to missed inspections, but also lag behind in identifying potential hazards such as worker violations (e.g., not wearing safety protective equipment, unauthorized hot work), fatigue (continuous overtime work, abnormal physiological state), and environmental risks (toxic gas leaks, excessive dust, abnormal temperature and humidity). Often, rectification can only be carried out after an accident occurs, making it difficult to achieve pre-accident warnings and in-process intervention, resulting in a persistently high accident rate.
[0005] Furthermore, the credibility and security of core data are insufficient. Attendance records, payroll calculations, and other data are directly related to the rights and interests of both employers and employees. Traditional management often uses paper sign-in, regular clocking in, or manual statistics, which are prone to loopholes such as proxy clocking in, false attendance, and data tampering. This leads to inaccurate payroll calculations and frequent labor disputes, which not only harms the legitimate rights and interests of workers but also affects the reputation of enterprises.
[0006] Finally, the data silo phenomenon severely restricts management efficiency. Existing worker management involves multiple business modules such as employment registration, safety supervision, training and assessment, attendance and salary, and equipment operation and maintenance. However, each module uses independently developed systems, resulting in inconsistent data standards and incompatible interfaces, which creates information barriers.
[0007] In summary, traditional worker management models are no longer adequate to meet the demands for refined, intelligent, and trustworthy management in the context of digital transformation. There is an urgent need for a one-stop management solution that can achieve reliable data storage and evidence preservation, intelligent risk early warning, precise and efficient training, and collaborative business operations, thereby addressing long-standing management pain points in the industry. Therefore, this paper proposes a worker management method and system based on multimodal perception and intelligent collaboration. Summary of the Invention
[0008] The purpose of this invention is to provide a worker management method and system based on multimodal perception and intelligent collaboration to solve the problems mentioned in the background art.
[0009] To achieve the above objectives, the present invention provides the following technical solution: a worker management method based on multimodal perception and intelligent collaboration, comprising the following steps:
[0010] S1. Construct a consortium blockchain network composed of multiple parties, and use a reputation-weighted practical Byzantine fault-tolerant consensus algorithm to store key worker operation data on the blockchain and add timestamps to establish a trusted archive.
[0011] S2. Deploy a unified access gateway under the microservice architecture, and use the device gateway to perform protocol adaptive access to heterogeneous hardware devices on site, and uniformly collect multimodal perception data.
[0012] S3: Deploy cameras, wearable devices, and sensor arrays to collect multi-source data. Use deep learning algorithms to perform real-time fusion analysis of multimodal perception data. After the system identifies high-risk behaviors, it automatically triggers the SOAR mechanism to execute a graded response.
[0013] S4 integrates BIM models, IoT and location data to construct a dynamic digital twin that maps to the physical construction site in a 1:1 manner. Combined with workers' historical data, it generates personalized VR training programs and provides immersive training and competency assessments for workers.
[0014] S5. Establish a unified data center to aggregate all business data and display key indicators such as risk index and training completion rate in real time through visual charts, so as to realize data linkage and closed-loop decision-making between various business modules.
[0015] Preferred: In step S1, when a worker completes key operations such as entering onboarding information, signing an electronic commitment letter, confirming monthly attendance, or confirming salary payment, the platform will automatically trigger the corresponding smart contract. The system will automatically package the core data of these operations into a transaction data block that conforms to the consortium blockchain standard. This transaction data block will be broadcast to all nodes in the consortium blockchain network. After being verified by the reputation-weighted practical Byzantine fault-tolerant consensus algorithm, it will be permanently recorded on the blockchain and stamped with a precise timestamp.
[0016] In the reputation-weighted practical Byzantine fault-tolerant consensus algorithm, nodes are assigned reputation weights based on their performance in historical transactions. Nodes with higher reputation have greater say in the consensus process, thus improving consensus efficiency while ensuring security.
[0017] Preferably, in step S2, the unified access gateway has a built-in software adapter library for multiple industrial protocols. When a new device is connected, the gateway sniffs the data packets sent by the device and automatically identifies its communication protocol. If the identification is successful, the gateway automatically loads the corresponding protocol adapter. If it is a new protocol or a private protocol, the administrator can define the parsing rules of the data packets and generate a new adapter through the graphical configuration interface.
[0018] All raw data uploaded by devices is uniformly converted into standardized JSON format by the gateway and pushed to the corresponding microservices for processing via message queues, thus achieving complete decoupling of the device management module from other business modules.
[0019] Preferably, in step S3, high-definition cameras are deployed at the construction site to collect video data, and convolutional neural networks are used to analyze the video stream in real time to identify violations.
[0020] Wearable devices are used to collect workers' physiological data and location information. Long short-term memory networks are then used to analyze the workers' data to identify fatigue states caused by excessive continuous working hours and abnormal physiological indicators.
[0021] Deploy gas, temperature, humidity, dust, and noise sensors to collect environmental parameters for risk assessment;
[0022] The system integrates the above analysis results to construct a dynamic risk profile of workers, thereby achieving precise quantification of risks.
[0023] The SOAR linkage mechanism is as follows:
[0024] Level 1 Response: The system immediately alerts the violating worker through on-site audible and visual alarms and generates vibration reminders through their wearable devices;
[0025] Level 2 Response: The system pushes alarm information, including risk type, precise location, on-site screenshots, and short videos, to on-site safety management personnel via APP or SMS;
[0026] Level 3 Response: In extreme cases, the system can be linked to the access control system to restrict personnel from entering dangerous areas, or automatically cut off the power to related equipment.
[0027] Preferably, in step S4, by integrating the project's BIM model, IoT sensor data, and platform personnel positioning data, a dynamic, real-time virtual digital twin that is mapped 1:1 to the physical construction site is constructed. The IoT sensor and personnel positioning data are injected into the twin in real time, so that it truly reflects the equipment operating status, environmental changes, and personnel locations of the physical construction site.
[0028] The system generates highly customized VR training programs for workers based on their job type, historical safety accident records, and training assessment results.
[0029] The system supports multiple workers and managers to conduct collaborative emergency drills in the same digital twin VR scene. All operations and behavioral trajectories are fully recorded by the system for post-event review and evaluation.
[0030] By capturing physiological data such as workers' reaction time, operational standardization, and heart rate changes through VR headsets and controllers, and combining this data with existing quiz records and scores on the platform, a multi-dimensional competency assessment report is generated to accurately pinpoint skill gaps.
[0031] Preferably, in step S5, when an increased risk is detected in a certain area, the work plan for that area is dynamically adjusted, and all key operations are stored on the blockchain to ensure the data credibility and security of the entire closed loop.
[0032] The workbench uses visual charts to display key metrics from various systems in real time, providing decision support for managers.
[0033] A system for implementing the control method described in any of the above claims, comprising:
[0034] The consortium blockchain evidence storage module is used to build a multi-party consortium blockchain network and uses a reputation-weighted practical Byzantine fault-tolerant consensus algorithm to achieve trusted data storage.
[0035] The device access gateway module has built-in multiple industrial protocol adapters for protocol adaptive access and data standardization conversion of heterogeneous hardware devices in the field;
[0036] The intelligent early warning module is used to integrate multimodal perception data from video, physiology, and environment, and to use AI algorithms to identify risks and trigger SOAR linkage response.
[0037] The immersive training module is used to build dynamic digital twins, generate personalized VR training programs, and record workers' operational trajectories and physiological data.
[0038] A unified data center and workbench are used to aggregate all data and enable visualization and cross-system collaborative decision-making.
[0039] Compared with the prior art, the beneficial effects of the present invention are:
[0040] 1. Data trustworthiness and tamper-proof: Through consortium blockchain and reputation-weighted consensus mechanism, key data is automatically uploaded to the blockchain for notarization, solving the problems of easy data tampering, frequent labor disputes and difficulty in traceability in traditional management.
[0041] 2. Intelligent early warning and proactive intervention: Through multimodal perception data fusion and AI algorithm analysis, it realizes the transformation from "post-event traceability" to "pre-event early warning and in-event intervention", which can proactively identify violations and respond in a graded manner, significantly reducing security risks.
[0042] 3. Highly efficient and immersive training experience: Utilizing dynamic digital twins and VR simulation technology, it provides low-cost, repeatable, and risk-free immersive training, and can customize training content based on individual profiles, effectively solving the problems of traditional training methods being monotonous and lacking relevance.
[0043] 4. Business Collaboration and Data Closed Loop: By using a unified data center and microservice architecture, the data barriers between various business modules are broken down, enabling intelligent cross-system collaboration and data-driven decision-making, which greatly improves overall management efficiency. Attached Figure Description
[0044] Figure 1 This is a flowchart of the intelligent access control and file creation process of the present invention;
[0045] Figure 2 This is a flowchart illustrating the adaptive unified management protocol of the present invention.
[0046] Figure 3 This is a flowchart of the intelligent monitoring and proactive early warning system of the present invention;
[0047] Figure 4 This is a flowchart of the safety training and competency assessment process for this invention.
[0048] Figure 5 This is a flowchart of the system collaboration mechanism of the present invention. Detailed Implementation
[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
[0050] Please see Figure 1-5 This invention provides a technical solution: a worker management method and system based on multimodal perception and intelligent collaboration. The core lies in constructing an intelligent closed loop centered on the full lifecycle management of workers. This closed loop is realized through the following mutually collaborative core business processes and technical systems:
[0051] I. Intelligent Access Control and File Establishment
[0052] The first step in worker management is efficient and reliable onboarding and record-keeping. This platform uses deployed real-name identity information input devices (such as ID card readers) to read worker identity information with one click and automatically generate an electronic file for each worker, including basic information, ID photo, and employment relationship, laying a data foundation for subsequent management processes.
[0053] To achieve the above business processes and ensure the authenticity, completeness, and immutability of worker files, while efficiently connecting with various on-site hardware devices, this platform adopts the following key technologies:
[0054] 1. A trusted evidence storage method for consortium blockchains based on multi-party consensus
[0055] To address the trust challenges in traditional management, such as the vulnerability of core data to tampering, frequent labor disputes, and poor data traceability, this method constructs a consortium blockchain network specifically designed for construction site scenarios, providing immutable legal protection for key operations throughout the entire lifecycle of workers, from onboarding to offboarding.
[0056] 1.1 Building a Multi-Party Consortium Blockchain Network: The platform constructs a consortium blockchain network composed of project parties, labor companies, banks, regulatory agencies, and other parties. Each party acts as a consortium node, jointly participating in data consensus and verification to ensure the fairness and authority of the data.
[0057] 1.2 Lightweight Consensus Mechanism Design: Addressing the unstable network environment and significant differences in computing power among participants in construction site networks, this invention designs a Practical Byzantine Fault-Tolerant (PBFT) consensus algorithm based on reputation weighting. Nodes acquire reputation weights based on their performance in historical transactions; nodes with higher reputation have greater influence in the consensus process, thereby improving consensus efficiency while ensuring security.
[0058] 1.3 Key operation data on the blockchain: When workers complete key operations such as entering onboarding information, signing electronic commitment letters, confirming monthly attendance or confirming salary payment, the platform will automatically trigger the corresponding smart contracts.
[0059] Data Packaging: The system automatically packages the core data of these operations (such as personnel ID, operation time, operation type, data hash value, participant digital signature, etc.) into a transaction data block that conforms to the consortium blockchain standard.
[0060] Consensus and Recording: The transaction data block is broadcast to all nodes in the consortium blockchain network. After being verified by the reputation-weighted PBFT consensus algorithm, it is permanently recorded on the blockchain and stamped with a precise timestamp.
[0061] 1.4 Cross-Project Resume Trustworthy Verification: Workers possess private keys authorizing access to their own data (such as skill certificates and past project experience). When a worker moves to a new project, they can authorize the new company administrator to verify the authenticity of their past resume on the blockchain by calling a smart contract, thus achieving secure and trustworthy data transfer.
[0062] II. Protocol-Adaptive Device Management Method
[0063] To achieve the aforementioned automated access control process, it is necessary to interface with and manage a wide variety of on-site hardware devices (such as turnstiles, medical examination instruments, and VR devices). This method provides a unified, flexible, and scalable technical foundation for the access and collaboration of all hardware devices.
[0064] 1. Microservice architecture design: The system breaks down various business functions such as personnel management, attendance, physical examination, and training into independent service units, achieving high cohesion and low coupling, which allows the equipment management module to be developed, deployed and extended independently.
[0065] 2. Adaptive Device Protocol Access: The core of the system is a unified device access gateway, which has built-in software adapter libraries for various industrial protocols (such as Modbus, OPC-UA, MQTT, TCP / IP).
[0066] 2.1 Protocol sniffing: When a new device is connected, the gateway first sniffs the data packets sent by the device and automatically identifies its communication protocol.
[0067] 2.2 Automatic / Manual Adaptation: If the identification is successful, the gateway will automatically load the corresponding protocol adapter; if it is a new protocol or a proprietary protocol, the administrator can define the parsing rules of the data packets and generate a new adapter through the graphical configuration interface.
[0068] 3. Data Standardization and Routing: The raw data uploaded by all devices is uniformly converted into a standardized JSON format by the gateway and pushed to the corresponding microservices for processing through a message queue (such as Kafka), thus achieving complete decoupling between the device management module and other business modules.
[0069] III. Intelligent Monitoring and Proactive Safety Early Warning of On-Duty Status
[0070] To upgrade safety management from "post-event traceability" to "pre-event early warning and in-event intervention," this system achieves proactive safety early warning through the following three synergistic technical methods:
[0071] 1. Data acquisition method based on multimodal sensing
[0072] To address the issue of insufficient data dimensionality from a single sensor, this method constructs a multimodal sensing device array for comprehensive data acquisition:
[0073] High-definition cameras are deployed at the construction site to collect video data for behavioral analysis.
[0074] Wearable devices such as smart bracelets and safety helmets are used to collect physiological data such as heart rate, blood pressure, body temperature, and location for status monitoring;
[0075] Deploying gas, temperature, humidity, dust, and noise sensors to collect environmental parameters for risk assessment provides a rich data foundation for subsequent analysis.
[0076] 2. A method for constructing dynamic risk profiles based on real-time fusion analysis of multi-source heterogeneous data
[0077] To address the challenge of processing massive amounts of heterogeneous data in a unified manner, this method utilizes deep learning algorithms to perform real-time fusion analysis of collected multi-source heterogeneous data.
[0078] Convolutional neural networks (CNNs) are used to analyze video streams in real time to identify violations such as not wearing safety helmets and crossing boundaries.
[0079] Long Short-Term Memory (LSTM) networks were used to analyze time-series data of workers' heart rate, body temperature, etc., to identify fatigue states such as excessive continuous working time and abnormal physiological indicators.
[0080] The system integrates the above analysis results to construct a dynamic risk profile of workers, thereby achieving precise quantification of risks.
[0081] 3. A hierarchical response method based on SOAR linkage
[0082] To address the issues of untimely early warning responses and limited processing methods in traditional approaches, this method employs a SOAR (Security Orchestration, Automation, and Response) linkage mechanism:
[0083] Level 1 Response (Immediate Alarm): The system immediately alerts the violating worker through on-site audible and visual alarms and generates a vibration reminder through their wearable device;
[0084] Level 2 Response (Management Intervention): The system pushes alarm information containing risk type, precise location, on-site screenshots and short videos to on-site safety management personnel via APP or SMS;
[0085] Level 3 Response (Emergency Response): In extreme situations (such as the detection of a toxic gas leak), the system can be linked to the access control system to restrict personnel from entering the dangerous area, or automatically cut off the power to the relevant equipment.
[0086] IV. Immersive Safety Training and Precise Competency Assessment
[0087] To overcome the shortcomings of traditional safety training methods, such as their limited scope, high cost, and difficulty in reproducing complex accidents, the system provides an efficient and reliable training method.
[0088] 1. Immersive Security Training Method Based on Digital Twin and VR Simulation
[0089] This method provides workers with a highly customized, repeatable, and risk-free immersive training experience by constructing a virtual digital twin that maps to the physical construction site on a 1:1 scale.
[0090] 1.1 Constructing a dynamic digital twin: The system integrates the project's BIM model, IoT sensor data (such as real-time data from tower cranes, elevators, deep foundation pit monitoring instruments, and environmental monitoring instruments) and personnel positioning data from the platform to construct a dynamic, real-time virtual digital twin that is mapped 1:1 to the physical construction site.
[0091] Geometric and physical modeling: Establishing an accurate geometric model of the construction site based on the BIM model.
[0092] Real-time data-driven: IoT sensor and personnel location data are injected into the twin in real time, enabling it to accurately reflect the equipment operating status, environmental changes, and personnel locations at the physical construction site.
[0093] Extreme working condition simulation: This twin can not only reflect the real state, but also simulate various extreme working conditions (such as rainstorms, strong winds, foundation pit collapses, fires, electric shocks, etc.), providing rich scenarios for training.
[0094] 1.2 Personalized VR Training Program Generation: Based on the worker's job type, historical safety accident records, and training assessment results, the system uses machine learning algorithms (such as decision trees or collaborative filtering algorithms) to generate highly customized VR training programs. For example, for workers who frequently violate regulations while working at heights, the system will automatically push more simulated high-altitude fall scenarios.
[0095] 2. Collaborative emergency drills and capability assessments
[0096] The system supports multiple workers and managers to conduct collaborative emergency drills in the same digital twin VR scene.
[0097] 2.1 Operation trajectory recording: All operations and behavior trajectories are fully recorded by the system for post-event review and evaluation.
[0098] 2.2 Multi-dimensional ability assessment: By capturing physiological data such as workers' reaction time, operational standardization, and heart rate changes through VR headsets and controllers, and combining them with the platform's existing answer records and scores, a multi-dimensional ability assessment report is generated to accurately identify skill deficiencies.
[0099] V. Data-Driven Decision Making and Trustworthy Closed Loop
[0100] The data generated from all the above steps are ultimately aggregated into a unified data center, forming a complete business loop and providing support for management decisions.
[0101] 1. System collaboration mechanism
[0102] The aforementioned systems do not operate in isolation, but rather collaborate closely within a unified data center, collectively forming a data-driven intelligent decision-making closed loop.
[0103] 1.1 Cross-system data linkage: For example, when the safety early warning system identifies an increase in risk in a certain area, it can push the information to the production optimization system to dynamically adjust the work plan for that area; the competency assessment report of the training system can update the worker's personal file (stored through blockchain) and affect their points (linked with the points management module).
[0104] 1.2 Unified Data Support: All critical operations are stored on the blockchain to ensure the credibility and security of data throughout the closed loop.
[0105] 1.3 Visualized Decision Support: The workbench (large screen) displays key indicators from various systems in real time (such as safety risk index, training completion rate, resource utilization rate, etc.) through visual charts, providing managers with "one-picture" decision support.
[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A worker management method based on multimodal perception and intelligent collaboration, characterized in that, Includes the following steps: S1. Construct a consortium blockchain network composed of multiple parties, and use a reputation-weighted practical Byzantine fault-tolerant consensus algorithm to store key worker operation data on the blockchain and add timestamps to establish a trusted archive. S2. Deploy a unified access gateway under the microservice architecture, and use the device gateway to perform protocol adaptive access to heterogeneous hardware devices on site, and uniformly collect multimodal perception data. S3: Deploy cameras, wearable devices, and sensor arrays to collect multi-source data. Use deep learning algorithms to perform real-time fusion analysis of multimodal perception data. After the system identifies high-risk behaviors, it automatically triggers the SOAR mechanism to execute a graded response. S4 integrates BIM models, IoT and location data to construct a dynamic digital twin that maps to the physical construction site in a 1:1 manner. Combined with workers' historical data, it generates personalized VR training programs and provides immersive training and competency assessments for workers. S5. Establish a unified data center to aggregate all business data and display key indicators such as risk index and training completion rate in real time through visual charts, so as to realize data linkage and closed-loop decision-making between various business modules.
2. The worker management method based on multimodal perception and intelligent collaboration according to claim 1, characterized in that: In step S1, when a worker completes key operations such as entering onboarding information, signing an electronic commitment letter, confirming monthly attendance, or confirming salary payment, the platform will automatically trigger the corresponding smart contract. The system will automatically package the core data of these operations into a transaction data block that conforms to the consortium blockchain standard. This transaction data block will be broadcast to all nodes in the consortium blockchain network. After being verified by the reputation-weighted practical Byzantine fault-tolerant consensus algorithm, it will be permanently recorded on the blockchain and stamped with a precise timestamp. In the reputation-weighted practical Byzantine fault-tolerant consensus algorithm, nodes are assigned reputation weights based on their performance in historical transactions. Nodes with higher reputation have greater say in the consensus process, thus improving consensus efficiency while ensuring security.
3. The worker management method based on multimodal perception and intelligent collaboration according to claim 1, characterized in that: In step S2, the unified access gateway has a built-in software adapter library for various industrial protocols. When a new device is connected, the gateway sniffs the data packets sent by the device and automatically identifies its communication protocol. If the identification is successful, the gateway automatically loads the corresponding protocol adapter. If it is a new protocol or a private protocol, the administrator can define the parsing rules of the data packets and generate a new adapter through the graphical configuration interface. All raw data uploaded by devices is uniformly converted into standardized JSON format by the gateway and pushed to the corresponding microservices for processing via message queues, thus achieving complete decoupling of the device management module from other business modules.
4. The worker management method based on multimodal perception and intelligent collaboration according to claim 1, characterized in that: In step S3, high-definition cameras are deployed at the construction site to collect video data, and convolutional neural networks are used to analyze the video stream in real time to identify violations. Wearable devices are used to collect workers' physiological data and location information. Long short-term memory networks are then used to analyze the workers' data to identify fatigue states caused by excessive continuous working hours and abnormal physiological indicators. Deploy gas, temperature, humidity, dust, and noise sensors to collect environmental parameters for risk assessment; The system integrates the above analysis results to construct a dynamic risk profile of workers, thereby achieving precise quantification of risks.
5. The worker management method based on multimodal perception and intelligent collaboration according to claim 4, characterized in that: The SOAR linkage mechanism is as follows: Level 1 Response: The system immediately alerts the violating worker through on-site audible and visual alarms and generates vibration reminders through their wearable devices; Level 2 Response: The system pushes alarm information, including risk type, precise location, on-site screenshots, and short videos, to on-site safety management personnel via APP or SMS; Level 3 Response: In extreme cases, the system can be linked to the access control system to restrict personnel from entering dangerous areas, or automatically cut off the power to related equipment.
6. The worker management method based on multimodal perception and intelligent collaboration according to claim 1, characterized in that: In step S4, by integrating the project's BIM model, IoT sensor data, and platform personnel positioning data, a dynamic, real-time virtual digital twin that is mapped 1:1 to the physical construction site is constructed. The IoT sensor and personnel positioning data are injected into the twin in real time, so that it truly reflects the equipment operating status, environmental changes, and personnel locations of the physical construction site. The system generates highly customized VR training programs for workers based on their job type, historical safety accident records, and training assessment results.
7. The worker management method based on multimodal perception and intelligent collaboration according to claim 6, characterized in that: The system supports multiple workers and managers to conduct collaborative emergency drills in the same digital twin VR scene. All operations and behavioral trajectories are fully recorded by the system for post-event review and evaluation. By capturing physiological data such as workers' reaction time, operational standardization, and heart rate changes through VR headsets and controllers, and combining this data with existing quiz records and scores on the platform, a multi-dimensional competency assessment report is generated to accurately pinpoint skill gaps.
8. The worker management method based on multimodal perception and intelligent collaboration according to claim 1, characterized in that: In step S5, when an increased risk is detected in a certain area, the work plan for that area is dynamically adjusted. All key operations are stored on the blockchain to ensure the data credibility and security of the entire closed loop. The workbench uses visual charts to display key metrics from various systems in real time, providing decision support for managers.
9. A system for implementing the control method according to any one of claims 1-8, characterized in that, include: The consortium blockchain evidence storage module is used to build a multi-party consortium blockchain network and uses a reputation-weighted practical Byzantine fault-tolerant consensus algorithm to achieve trusted data storage. The device access gateway module has built-in multiple industrial protocol adapters for protocol adaptive access and data standardization conversion of heterogeneous hardware devices in the field; The intelligent early warning module is used to integrate multimodal perception data from video, physiology, and environment, and to use AI algorithms to identify risks and trigger SOAR linkage response. The immersive training module is used to build dynamic digital twins, generate personalized VR training programs, and record workers' operational trajectories and physiological data. A unified data center and workbench are used to aggregate all data and enable visualization and cross-system collaborative decision-making.