Intelligent Control Method and System for Preventing Misoperation of Power Equipment Based on Multimodal Data

Abstract

This invention relates to the field of power equipment anti-misoperation technology, and in particular to an intelligent control method and system for power equipment anti-misoperation based on multimodal data. The method includes: collecting multimodal data of the power equipment to be maintained and storing the collected multimodal data in a blockchain; preprocessing the collected multimodal data to obtain current state vectors for three attributes; calculating the consistency score for each of the three attributes based on the target state vectors and the current state vectors; calculating the overall consistency score based on the consistency scores of the three attributes; obtaining a fusion score based on the verification scores of the three attributes; if the fusion score is less than a set threshold, activating an abnormal state blocking mechanism and a self-locking control mechanism for the power equipment; if it is greater than the set threshold, activating a remote intervention mechanism and a dynamic unlocking mechanism to achieve real-time identification of misoperation.

Description

Technical Field

[0001] This invention relates to the field of power equipment anti-misoperation technology, and in particular to a method and system for intelligent control of power equipment anti-misoperation based on multimodal data. Background Technology

[0002] In the field operation, maintenance, and dispatching of power systems, QR codes have been widely used as digital identity carriers in key areas such as equipment identification, task binding, and authorization verification. Existing QR code-based error prevention operating systems typically employ static encoding mechanisms, directly embedding the device ID, task number, or link address into the QR code pattern, which is then parsed and used by the operating terminal after scanning. However, this approach has significant drawbacks in real-world high-risk operating environments: firstly, QR codes lack the ability to bind contextual elements such as operator, spatial location, and task status, making them highly susceptible to misscanning, incorrect scanning, substitution, or forgery; secondly, after scanning, most systems only perform static matching based on the device ID, lacking semantic understanding and dynamic judgment of the behavior itself, making it difficult to effectively determine whether the identified behavior is within a "reasonable spatial location," "legitimate operator," or "authorized operation phase."

[0003] According to industry accident statistics, "QR code misidentification," caused by factors such as dense equipment environments, proximity of QR code placement, and complex lighting conditions, has become one of the most frequent causes of on-site operational errors. During task execution, operators may mistakenly identify adjacent equipment QR codes as target equipment due to positioning deviations or lighting obstructions, thus bypassing anti-misoperation logic and issuing incorrect commands. This can lead to serious operational consequences such as incorrect separation, incorrect connection, or incorrect tripping, and may even cause equipment short circuits, bus power outages, or personal injury. Current mainstream anti-misoperation strategies mostly rely on static blacklists and whitelists, manual review, or post-event audits, lacking integrated prevention and control capabilities that combine multi-source sensing, real-time judgment, and proactive response. This makes it difficult to support the high real-time, high safety, and high reliability requirements of next-generation power operations in preventing misoperations. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent management and control method and system for preventing misoperation of power equipment based on multimodal data. It constructs a multimodal perception and intelligent collaborative mechanism for preventing misoperation in high-safety-level operation scenarios, capable of integrating multi-dimensional information such as image recognition, spatial perception, biometric authentication, and permission modeling to dynamically construct on-site operation state vectors and achieve high-precision consistency matching with task-defined states. Furthermore, after identifying abnormal operation behavior, it should possess a configurable hierarchical blocking mechanism, remote intervention capabilities, and full-process audit modeling capabilities, enabling real-time identification, rapid blocking, safe recovery, and risk traceability management of misoperations, providing solid technical support for a new intelligent power operation and maintenance system.

[0005] On the one hand, a smart control method for preventing misoperation of power equipment based on multimodal data is provided, including:

[0006] Scan the QR code that is pre-posted on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest.

[0007] Multimodal data of the power equipment to be maintained is collected and stored in the blockchain; the collected multimodal data is preprocessed to obtain the current state vector of the equipment identity, the current state vector of the spatial location, and the current state vector of the permission digest.

[0008] Based on the target state vector and current state vector of device identity, spatial location, and permission summary, calculate the consistency score for each of the three attributes; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than a set threshold. If it is, it indicates that the multimodal data acquisition has failed, and the multimodal data is re-acquired; otherwise, proceed to the next step.

[0009] A fusion score is obtained based on the verification scores of the three attributes. If the fusion score is less than a set threshold, the power equipment is in an abnormal state, and the abnormal state blocking mechanism and self-locking control mechanism are activated. If the fusion score is greater than the set threshold, the power equipment is activated with a remote intervention mechanism and a dynamic unlocking mechanism.

[0010] On the other hand, an intelligent control system for preventing misoperation of power equipment based on multimodal data is provided, including:

[0011] The decryption module is configured to: scan the QR code pre-attached on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest.

[0012] The multimodal data acquisition module is configured to: acquire multimodal data of the power equipment to be maintained, store the acquired multimodal data in the blockchain; preprocess the acquired multimodal data to obtain the current state vector of the equipment identity, the current state vector of the spatial location, and the current state vector of the permission digest;

[0013] The consistency scoring module is configured to: calculate the consistency score of each of the three attributes based on the target state vector and the current state vector of the device identity, spatial location, and permission summary; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than a set threshold. If it is, it indicates that the multimodal data acquisition has failed and the multimodal data is re-acquired; if not, proceed to the verification scoring module.

[0014] The verification and scoring module is configured to: obtain a fusion score based on the verification scores of the three attributes; determine whether the fusion score is less than a set threshold; if so, the power equipment is in an abnormal state, and activate the abnormal state blocking mechanism and self-locking control mechanism for the power equipment; when the fusion score is greater than the set threshold, activate the remote intervention mechanism and dynamic unlocking mechanism for the power equipment.

[0015] The above technical solution has the following advantages or beneficial effects:

[0016] First, this invention addresses the core problems of traditional QR code recognition technology in complex on-site environments, such as static and simple structure, high false recognition rate, and lack of contextual verification, by designing a multimodal consistency verification mechanism. This mechanism is based on the "state vector modeling" method, constructing multi-layered semantic abstractions from three dimensions: device identity, spatial location, and personnel permissions. Combined with a context-aware strategy, it achieves collaborative matching and deviation tolerance determination between real-time data acquisition status and task target status. A scoring function is introduced to support data heterogeneity and operational fluctuations in complex scenarios, effectively solving the problems of weak adaptability to sensor errors and behavioral uncertainties in traditional comparison methods, and significantly improving the anti-interference capability of false recognition and the strength of operational legitimacy verification in on-site operations.

[0017] Secondly, this invention proposes a consistency-scoring-driven anomaly blocking and terminal self-locking control mechanism, constructing a chain-like security control path of "scoring-judgment-response" from score calculation to physical locking. The system dynamically triggers a three-level blocking process according to the scoring threshold, including task flow freezing, terminal interaction locking, and multimodal risk warnings. All operations are completed in milliseconds, ensuring that the system responds immediately upon detecting a security anomaly. An innovative joint identification strategy based on contact area and trigger duration is introduced, significantly reducing the risk of accidental touches and misjudgments. Simultaneously, multi-dimensional feedback methods such as voice and images enhance the operator's risk perception, effectively achieving real-time closed-loop blocking of misoperations under "unmanned intervention conditions," exhibiting extremely high robustness and responsiveness.

[0018] Third, this invention integrates a distributed security mechanism and proposes an integrated process model for remote intervention and dynamic unlocking, solving the bottleneck problems of complex manual verification and delayed response in traditional systems after terminal locking. In the self-locked state, a task context is automatically constructed and uploaded to the backend through a secure tunnel. The backend verification platform completes data reconstruction and permission verification in a trusted environment and generates a structured remote unlocking command based on the task type. This process integrates core technologies such as multi-factor authentication, structured command signing, hardware security module (HSM) key protection, and blockchain notarization, ensuring the legitimacy, non-repudiation, and full traceability of remote operations, significantly improving the efficiency of critical operation recovery and management compliance.

[0019] Fourth, this invention constructs a closed-loop control architecture for preventing misoperations, encompassing data perception, semantic judgment, proactive blocking, and remote handling. It ensures the verifiability and data integrity of the entire operation process through a trusted log system and on-chain auditing mechanisms. By integrating six technical pathways—multimodal sensing, semantic models, consistency scoring, hardware interruption, remote unlocking, and blockchain evidence storage—it forms a highly robust, confident, and dynamically controllable misoperation prevention technology system. It is applicable to critical infrastructure scenarios with extremely high requirements for security, operational accuracy, and closed-loop response, such as power maintenance, industrial control, rail transit scheduling, and aviation ground support, possessing broad engineering application potential and widespread value. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a flowchart of the method in Example 1. Detailed Implementation

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] Example 1

[0024] This embodiment provides an intelligent management and control method for preventing misoperation of power equipment based on multimodal data;

[0025] like Figure 1 As shown, the intelligent control method for preventing misoperation of power equipment based on multimodal data includes:

[0026] S101: Scan the QR code pre-affixed on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest.

[0027] S102: Collect multimodal data of the power equipment to be maintained, and store the collected multimodal data in the blockchain; preprocess the collected multimodal data to obtain the current state vector of the equipment identity, the current state vector of the spatial location, and the current state vector of the permission digest;

[0028] S103: Calculate the consistency score for each of the three attributes based on the target state vector and current state vector of device identity, spatial location, and permission summary; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than the set threshold. If it is, it indicates that the multimodal data acquisition has failed, and the multimodal data is re-acquired; if not, proceed to the next step.

[0029] S104: Obtain the fusion score based on the verification scores of the three attributes; determine whether the fusion score is less than the set threshold. If it is, the power equipment is in an abnormal state, and activate the abnormal state blocking mechanism and self-locking control mechanism for the power equipment; when the fusion score is greater than the set threshold, activate the remote intervention mechanism and dynamic unlocking mechanism for the power equipment.

[0030] Furthermore, the process of calculating consistency scores for each of the three attributes based on the target state vector and the current state vector of device identity, spatial location, and permission digest specifically includes:

[0031] The consistency score between device identity modalities is calculated based on the target state vector and the current state vector of device identity; the consistency score between spatial location modalities is calculated based on the target state vector and the current state vector of spatial location; and the consistency score between permission digest modalities is calculated based on the target state vector and the current state vector of permission digest.

[0032] Furthermore, the QR code includes: three encrypted key fragments;

[0033] The three encryption key segments include: an encrypted device identifier segment, an encrypted location encoding segment, and an encrypted permission digest segment;

[0034] The device identification segment includes: power equipment ID, current task timestamp, and unique identification code; the encrypted device identification segment is obtained by encrypting the device identification segment using the national cryptographic algorithm SM4;

[0035] The location encoding segment includes: the absolute coordinates of the operator's target work area, the topological location label code, and the preset operator spatial location; the encrypted location encoding segment is obtained by encrypting the location encoding segment using the ECC encryption algorithm;

[0036] The permission digest segment includes: operator permission information, role attributes, and preset operator biometric authentication data; the encrypted permission digest segment is obtained by encrypting the permission digest segment.

[0037] For example, the device identifier segment is encrypted using a dynamic device identifier encryption algorithm based on chaotic mapping, and the encryption formula is:

[0038]

[0039] in, This indicates the result of encryption processing at the device level on the original device identification information. This is a piecewise linear chaotic mapping function, with input values ​​oscillating between 0 and 1; The chaos intensity coefficient (standard value 3.99); For the first generation generated by the chaotic system A random number; Use the SM3 hash function; For task timestamps accurate to milliseconds; These are normalized device space coordinate values; For device ID, Represents element-wise product; This represents the XOR operation.

[0040] The first generated by the chaotic system Each random number is XORed bitwise with the device ID to achieve nonlinear diffusion. Each operation involves a chaos intensity coefficient. and normalized device space coordinates The differences result in a unique ciphertext.

[0041] The encryption formula for the device identifier segment is used to generate a three-dimensional chaotic sequence through the Lorenz system. It is nonlinearly coupled to the physical coordinates of the device. The piecewise linear chaotic mapping function is involved. Number of iterations The risk level is dynamically determined by the task risk level (default value 128 times). The parameters are adjusted in real time by the quantum cryptography module. During encryption, the original device ID is first spatiotemporally bound, and then diffusion and confusion are achieved through bitwise chaotic perturbation operations, ultimately outputting 256-bit ciphertext. This ciphertext has the dynamic characteristic of "one-time pad": under any change in spatiotemporal parameters, the ciphertext similarity will decrease significantly, effectively preventing replay attacks and forgery.

[0042] For example, the location coding segment integrates the geographical information of the device in absolute and relative space, and compresses and binds the spatial location information through hierarchical coding rules. The location coding segment consists of the following two parts: the first part is the absolute position of the device: using GNSS coordinates or BeiDou grid number; the second part is the relative topology number of the device: identifying the relationship between adjacent devices according to the task operation topology.

[0043] When scanning a QR code, the terminal compares the real-time spatial location data with the target location segment encoded in the QR code.

[0044] A spatial similarity determination model based on deep manifold learning is developed, which is error-tolerant to complex spatial structures and outputs spatial location similarity scores. This measure assesses whether the actual operation location is within the spatial range specified in the task ticket. Its value range is [0,1]. The closer the value is to 1, the higher the spatial consistency. Its core formula is:

[0045]

[0046] in, () is implemented based on the ResNet-18 network, mapping the input coordinates to a 128-dimensional embedding space; The actual measurement location of the equipment. This is the reference position of the device, which is a set value. The allowable position error radius; It is a minimal constant ( ) Prevent division by zero; F is the Frobenius norm, used to calculate the global distance between two embedding vectors (i.e., the Euclidean norm of the vector difference).

[0047] The aforementioned method, based on deep manifold learning, maps multi-dimensional spatial data such as device location, region number, and signal features into a high-dimensional semantic embedding space. It then uses geodesic distance within the manifold to determine the similarity between the current operation location and the designated location on the ticket. Compared to traditional geometric distance algorithms, this model can identify spatial topological relationships and semantic proximity between devices, thus accurately determining whether an operation occurs within the designated task area in complex environments, effectively improving the intelligence and robustness of spatial verification.

[0048] The ResNet-18 network, trained on hundreds of thousands of power equipment location samples, supports robust identification of topologically similar locations and has fault tolerance for low-precision measurement signals. The decision model can adjust the error radius based on real-time signal strength, improving the flexibility and accuracy of spatial legitimacy verification.

[0049] For example, the permission summary segment describes the personnel permission summary information associated with the current task, and is constructed using a dynamic hashing and attribute encryption mechanism to ensure that permissions are controllable and cannot be forged.

[0050] The authorization digest segment generates a one-time SHA-3-256 hash digest based on the current task ID, employee ID, and role level, and embeds timeliness control parameters. The backend dynamically generates temporary authorization codes through a role-based access control (RBAC) model to ensure that permissions are strictly bound to the terminal login session and to prevent unauthorized operations across terminals.

[0051] The authorization credential generation adopts a fusion scheme of zero-knowledge proof and attribute-based encryption:

[0052] ;

[0053] in, A character string is assigned to the operator's role and is used to bind the RBAC policy. Use the current UTC time to achieve timeliness control; It is a public generator, used as the basis of the encryption system, representing the system's public parameters; For auxiliary generators (which should satisfy the following conditions): (Irrelevant), serving as a source of randomness or obfuscation, making it impossible for attackers to deduce the original value from the commitment; the fundamental parameter of the bilinear pairing operation; Indicates the first in the access policy The system assigns a unique generator to each rule. Used to construct access policies, keys, or ciphertext. This indicates that there are a total of system-defined access control policies. Strategy rules; Used as the system's public key to verify identity legitimacy; For time-related attribute functions, implement dynamic permission adjustment; bilinear pairing Used for cryptographic relationship verification; It is a non-interactive proof protocol that hides sensitive attribute information. It represents a dynamically generated operator permission digest value based on the current task context. As a combined credential of identity and permissions, it is used for permission verification and on-chain data storage, proving to the verifier that the operator does indeed have the attributes that satisfy a certain access policy and that the generation behavior is within the valid time. This is an attribute parameter binding item, representing the system's encrypted representation of rule j. It transforms the operator's attributes into a verifiable encrypted form, verifying whether the item meets the system-defined access policy (e.g., "role = dispatcher and level ≥ 3"). This represents a public key encrypted value used to bind authorization credentials to the system's root of trust and to check whether the authorization has been validly granted by the system.

[0054] The authorization verification process does not expose any sensitive operator attributes; it only proves that the operator possesses a set of attributes that conform to the policy. The key binding process uses a master key pair generated by the SM2 elliptic curve algorithm, dynamically constructed during task initialization. This ensures a strong binding between authorization credentials and device state and task context, preventing cross-session impersonation and unauthorized operations.

[0055] The aforementioned device identification segment, location code segment, and authorization digest segment are encrypted and embedded into independent logical blocks within the QR code, forming a "three-segment dynamic QR code key fragment structure." During scanning, only in the current task state can the front-end terminal dynamically combine and verify the three segments based on the device and personnel targets defined in the ticket.

[0056] Further, the three encrypted key fragments are decrypted to obtain the target state vector of device identity, the target state vector of spatial location, and the target state vector of permission digest, wherein,

[0057] The target state vector for device identity includes: power device ID, current task timestamp, and unique identifier;

[0058] The target state vector of spatial location includes: the absolute coordinates of the operator's target work area, the topological location label code, and the preset operator spatial location;

[0059] The target state vector of the permission summary includes: operator permission information, role attributes, and preset operator biometric authentication data.

[0060] For example, S102: Collect multimodal data of the power equipment to be maintained, wherein the multimodal data includes: image recognition data, location awareness data, biometric data and time data.

[0061] For example, the multimodal data includes: operator's facial image data, device spatial positioning data, operator's spatial positioning data, and operator's biometric authentication data;

[0062] The multimodal data, along with terminal identifiers and timestamps, is encapsulated into a state dataset and its integrity, authenticity, and traceability are ensured through distributed encryption and chain-based evidence storage.

[0063] Based on the three encryption key fragments extracted from the QR code and combined with the acquired real-time multimodal data, key concatenation and decryption operations are performed.

[0064] The decryption results from the three dimensions are uniformly encapsulated into a state vector of the current operation behavior. The structure includes fields such as device ID, spatial coordinates, orientation angle, personnel ID, and permission tag.

[0065] The PAD terminal-based multimodal data acquisition framework integrates four core modules: image recognition, spatial awareness, biometrics, and time synchronization. This enables unified acquisition, spatiotemporal alignment, and feature fusion of on-site data. It effectively achieves dynamic assembly and trusted verification of QR code key fragments, possessing high-precision, strong synchronization, and multi-dimensional fusion capabilities. This allows for dynamic modeling and semantic matching of operational behaviors across multiple dimensions, including device identity, spatial location, and personnel permissions.

[0066] For example, the data acquisition for multimodal sensing includes:

[0067] (1) Image Recognition: The YOLOv5s model is used for robust real-time detection and accurate localization of QR codes. The overall architecture adopts a two-stage optimization: the first stage uses a lightweight backbone network to complete efficient image feature extraction; the second stage introduces an attention-enhanced decoder to achieve sub-pixel-level boundary recognition. In terms of dealing with complex lighting and environmental interference, an adaptive histogram equalization algorithm is integrated for lighting compensation, and a Wiener filtering strategy based on point spread function estimation is combined to suppress motion blur. This filtering is based on a frequency domain degradation model (signal-to-noise ratio K=0.002) for image restoration to ensure the stability of QR code recognition under extreme operating environments such as strong light, weak light, and shaking.

[0068] The device identifier segment in the QR code is decrypted using the SM4-CBC mode, and a temporary session key is generated by combining it with the PTP timestamp to achieve dynamic binding and forward security. The decryption result is XORed with the session key to output a 256-bit session identifier, which is then encrypted using SHA-3 hashing to form a dynamic identity credential. To prevent man-in-the-middle attacks, the system employs a dual verification mechanism of "hash + signature" and combines it with a trusted execution environment to isolate the decryption process, ensuring the integrity and verifiability of the transmitted identity data.

[0069] (2) Position Awareness: A high-precision position awareness framework with a three-level data fusion mechanism. The original layer uses extended Kalman filter (EKF) technology to achieve tight coupling between Global Navigation Satellite System (GNSS) and Ultra-Wideband (UWB) positioning data. Its core is a state vector containing 15 parameters, which not only covers basic motion states such as position (longitude, latitude, altitude), velocity (three-dimensional components) and attitude (roll angle, pitch angle, yaw angle), but also includes key sensor error compensation terms (such as receiver clock error, drift, etc.). This layer inputs absolute positioning observations such as pseudorange and carrier phase provided by GNSS, along with high-precision relative distance measurements to fixed anchor points provided by UWB, into the iterative update process of EKF. The intermediate layer uses outlier detection based on Mahalanobis distance to remove multipath interference data. If the calculated Mahalanobis distance exceeds a preset statistical threshold, the observation is identified as an anomaly and actively removed by the system to prevent it from contaminating subsequent fusion results. The decision layer outputs the final high-confidence pose estimate using a weighted least squares method (weights proportional to the square of the signal-to-noise ratio). In particular, the mechanism also integrates a dynamic error ellipse model. The model dynamically calculates and updates the distribution of positioning errors on the horizontal plane based on real-time satellite geometric distribution factors and the topology of UWB anchor points. The final output pose information includes precise position, attitude angles, and key confidence scores and error ellipse parameters in a unified coordinate system, directly serving the spatial validity verification of the upper-layer error prevention logic.

[0070] (3) Biometrics: Improving the reliability of liveness detection through multispectral fusion: Near-infrared imaging is used for vein texture analysis, visible light imaging for micro-expression detection, and 3D structured light for surface morphology reconstruction. A feature-level fusion strategy is adopted, and the 256-dimensional feature vectors of the three modalities are concatenated through attention weights and then input into the ResNet-18 classifier. This effectively suppresses common spoofing behaviors such as photo attacks and model attacks, and enhances the recognition accuracy in low-light and partial occlusion scenarios through feature fusion.

[0071] Feature compression and irreversible protection mechanisms are introduced to encrypt biometric information. The original multimodal features undergo nonlinear dimensionality reduction using kernel principal component analysis (KPCA), followed by compression to a 32-dimensional feature space via RBF mapping. This is then normalized using an improved sigmoid function, ultimately outputting a privacy-preserving identity vector. Linear inseparable transformations and normalized compression are implemented within the feature space to ensure that user identities are "verifiable but not reconstructible" during use, achieving "usable but invisible" biometric features. The system is deployed in a distributed computing architecture, with each modality processing node physically isolated, further enhancing overall data security.

[0072] (4) Time synchronization: A hybrid clock synchronization architecture is adopted: the master clock establishes a microsecond-level synchronization network through the PTPv2 protocol, and the slave clock device uses a linear regression algorithm to compensate for crystal oscillator drift. The clock state machine automatically switches the synchronization mode according to the network latency jitter: when the jitter is low, the end-to-end transparent clock mode with hardware timestamps is adopted, and when the jitter is high, it switches to the boundary clock mode.

[0073] When scanning the code, four types of core modal data, namely equipment, location, personnel, and context, are collected and integrated to generate a standardized "on-site operation status vector".

[0074] All permission data is stored in SM4 encryption and supports a dynamic update mechanism based on ABAC policy trees, meeting the needs of fine-grained permission reconstruction in task-driven scenarios.

[0075] To ensure the spatiotemporal consistency of multi-source data, at the hardware level, a dedicated clock chip coordinates the sampling times of each sensor, controlling time deviations to the microsecond level. The software system records the inherent latency characteristics of each sensor and automatically performs time compensation during data processing. Network communication employs a modified NTP protocol, enabling millisecond-level time synchronization between the terminal and the server. A multimodal synchronization consistency scoring function is introduced to quantitatively analyze the synergy between multiple data sources, including image recognition, spatial perception, and biometric authentication.

[0076] For example, S102: storing the collected multimodal data into the blockchain, specifically including:

[0077] CRC-32 checksums are used to ensure data structure integrity, generating a 32-bit checksum for each data segment. Secondly, the national standard SM2 algorithm is used to generate digital signatures, and elliptic curve cryptography is used to verify the authenticity of the data source. The hash values ​​of key data are uploaded to the blockchain in real time for evidence storage, constructing an immutable audit trail chain. The specific implementation includes the following steps: terminal devices calculate SM3 hashes on the collected data; gateway nodes package multiple terminal hash values ​​every 5 minutes to generate a Merkle tree; and a lightweight consensus mechanism is used to write the Merkle root hash into the blockchain.

[0078] Further, S102: Preprocess the collected multimodal data to obtain the current state vector of device identity, the current state vector of spatial location, and the current state vector of permission digest, including:

[0079] The device image and operation time are identified to obtain the current state vector of the device identity;

[0080] The operator's spatial location perception data is identified to obtain the current state vector of the spatial location;

[0081] The biometric data is processed to obtain the target state vector of the authorization summary.

[0082] Further, S103: Based on the target state vector of device identity and the current state vector of device identity, calculate the consistency score between device identity modalities, including:

[0083]

[0084] in, The device identity target state vector is generated from QR code image features or depth features, with a dimension of 128. The current state vector for the device identity, containing the ID hash or encrypted vector representation; The squared Euclidean distance is used to measure the difference between two identity feature vectors. This is the identity consistency sensitivity coefficient, which controls the degree of influence of distance on the score. The larger the value, the more sensitive the score is to differences. It is an exponential function, equivalent to It maps a real number to a positive number and smooths the change around 0, which is used for normalization of probability or rating. This represents the consistency score between device identity modalities.

[0085] The formula is essentially a Gaussian kernel + Sigmoid normalization, which ensures that the score is smooth between 0 and 1. The closer it is to 1, the higher the consistency between image recognition and QR code decryption, and the more reliable the multimodal synchronization.

[0086] Furthermore, S103: Based on the target state vector and the current state vector of the spatial location, calculate the consistency score between the spatial location modes. ,include:

[0087]

[0088] in, The target state vector is the spatial location, including 3D coordinates and attitude angles; This is the current state vector of the spatial position measured by the positioning sensor; This represents the spatial consistency sensitivity coefficient, which controls the speed at which the score responds to the magnitude of the error. The larger the value, the more sensitive the score is to spatial bias; This represents the consistency score between spatial location modes.

[0089] Further, S103: Based on the target state vector and the current state vector of the permission digest, calculate the consistency score between the permission digest modalities. ,include:

[0090]

[0091] in, The current state vector is the permission digest; The target state vector is the permission summary; The function returns the dimension of the vector; The standard total dimension (default 128); This represents the number of missing / outlier values ​​in the vector. For the set integrity sensitivity coefficient, penalize missing or outlier values ​​in the permission feature vector. The larger the missing value, the more significant the negative impact of missing values ​​on the score; This represents the consistency score between permission summary modalities.

[0092] Consistency scoring between permission summary modalities , is used to measure the consistency of attributes between operator permissions, biometric authentication information, and permission data embedded in the QR code. The value range is [0,1], and the closer it is to 1, the higher the data quality.

[0093] Further, S103: Based on the consistency scores among device identity modalities, spatial location modalities, and permission digest modalities, calculate the total consistency score, including:

[0094] Overall consistency score The definition is as follows:

[0095]

[0096] in, It is an exponential expression with the natural constant as the base; The maximum difference in timestamps for all modal data acquisitions is used to measure the degree of time synchronization. These are the consistency scores between device identity modalities, the consistency scores between spatial location modalities, and the consistency scores between permission digest modalities. , , The weighting factors are the three bias terms: device identity, spatial location, and permission summary. This is a weighting factor for the timestamp difference, which can be dynamically adjusted according to the application scenario.

[0097] The consistency score ranges from (0,1). A consistency score closer to 1 indicates better synchronization and higher confidence among the multimodal data. When the value is below the set threshold, it will be determined that there is a potential spatiotemporal deviation in the current state vector construction process, and multimodal data will be re-acquired or auxiliary modal completion will be performed.

[0098] For example, a response control operation is triggered based on the consistency score result. When the consistency score is lower than a preset threshold, an exception response process is executed and structured exception log data is generated.

[0099] The abnormal response process includes: freezing the current task status and terminating subsequent operation commands; disabling terminal input event response and switching the interface to warning status; and using the abnormal type to drive graphical prompts and voice broadcast content.

[0100] The anomaly types include: device mismatch, location out of bounds, and insufficient permissions;

[0101] The generated structured exception log data includes timestamps, exception fields, status vectors, and scoring details.

[0102] For example, the consistency verification adopts a layered semantic matching architecture in its overall design. By dynamically aligning the "task-defined target state vector" with the "real-time on-site acquired state vector," it achieves collaborative determination of three key dimensions: device identity, spatial location, and personnel permissions. The system not only performs surface-level value matching but also introduces multi-layered semantic abstraction and contextual constraint judgment to support error tolerance and fuzzy judgment capabilities in complex on-site environments.

[0103] First, the target state vector of the current task is extracted from the task management subsystem, and its structure includes:

[0104] A set of device identifiers authenticated by digital signatures; a spatial location constraint boundary model (such as geofencing, polygon shapes, yaw angle limits, etc.); an attribute-based access control policy tree (ABAC) covering operational roles, task levels, and dynamic environmental constraints.

[0105] Subsequently, the terminal PAD device triggers a multimodal perception process through a QR code scanning event, collecting data such as images, location, biometrics, and time to construct a current scene state vector. Both data are synchronously transmitted to a Trusted Execution Environment (TEE) via an asynchronous cache channel for verification, and then compared dimension-by-dimensionally by the semantic matching module.

[0106] To achieve a precise and interpretable verification process, the system can also construct an independent scoring function for each dimension, specifically including the following steps:

[0107] (1) First, the device identifier segment in the QR code is decrypted, and the original device ID is parsed out using the SM4-CBC mode. This ID value is pre-screened using a Bloom filter to quickly eliminate device candidates that are not in the whitelist. The filtered device IDs are compared using a constant-time algorithm to prevent time-series attacks; at the same time, a device credibility score based on historical interaction records is introduced to form a two-factor authentication model that combines identity accuracy and historical stability.

[0108] (2) Spatial location dimension integrates GNSS and UWB dual-mode data, uses tight-coupled filtering and Ray-Casting geofencing judgment, and forms a multi-layered location legality verification with yaw angle difference model and dynamic interference radius constraint;

[0109] (3) Combining multi-factor authentication and attribute control strategies to assess personnel legitimacy. After joint authentication using locally collected biometrics and work card information, the ABAC engine is invoked to verify the permission policy. The following multi-dimensional attributes are considered during the policy evaluation process: operation role level, current task risk level, operator's historical violation records, and environmental risk factors. The permission matching degree is quantified through the policy satisfaction rate model to generate a permission confidence score.

[0110] The validation results from each dimension are input into the joint scoring model to calculate the confidence score.

[0111] Further, S104: Calculate the verification score for device identity, the verification score for spatial location, and the verification score for permission digest, wherein,

[0112] Device identity verification score The calculation formula is:

[0113] ;

[0114] in, To match the weighting factor (default value 0.7), it can be adjusted according to the task risk level; The device target ID obtained by decrypting the QR code; The current device ID obtained through real-time data collection; The string matching function is defined as follows:

[0115] ;

[0116] Spatial location verification score The calculation formula is:

[0117] ;

[0118] in, The coordinates of the current location of the barcode scanning device obtained by measurement; The target location coordinates are preset for the task; The position error tolerance radius is generated by the dynamic error ellipse model; The geofencing exponential function includes:

[0119] ;

[0120] in, This indicates the pre-defined geofence area for the task.

[0121] The distance difference between the current location and the target location is quantified using a Gaussian kernel function. The closer the location is to the target location, the higher the score. If the current location is within the preset geofence, it gets 1 point; otherwise, it gets 0 points. The score is the product of two factors to ensure that the location meets both accuracy and range requirements.

[0122] Permission digest verification score The calculation formula is:

[0123] ;

[0124] in, Indicates the total number of permission attributes. Indicates the permission attribute number being compared; Represents the weighting factor for each permission attribute dimension; Indicates the first Operator attribute values ​​obtained through biometric authentication or on-site data collection in each dimension; This refers to the first part of the permission digest obtained by decrypting the QR code. Each corresponding attribute value.

[0125] Furthermore, the aforementioned This is a modal feature matching function used to calculate the degree of matching between the current operator's biometric features and attribute information and the permission summary obtained by decrypting the QR code. When the input is a discrete attribute, exact matching or rule matching is used; when the input is a continuous feature vector, cosine similarity is used for calculation and normalized to the interval of 0 to 1. The closer the return value is to 1, the higher the degree of matching.

[0126] If the input All are discrete attributes, and the discrete attributes match exactly. ;

[0127] If the input Both are discrete attributes, and the discrete attributes do not match. ;

[0128] If the input If all are continuous eigenvectors, then .

[0129] Furthermore, S104: Based on the above three verification scores, the fusion score is obtained, including:

[0130] ;

[0131] in, A verification score indicating the device's identity; Verification score indicating spatial location; The verification score represents the permission digest. The confidence level indicating the device's identity; The confidence level indicating spatial location. The confidence level of the permissions digest;

[0132] A tiered response is triggered after the fusion score is compared with a preset threshold (typically 0.85):

[0133] when A value ≥0.85 indicates that the current operation is valid and can be executed.

[0134] When 0.75≤ If the value is less than 0.85, then the enhanced verification process will be initiated, which refers to secondary biometric identification.

[0135] when If the value is less than 0.75, a hard block will be executed, and a security audit log will be generated.

[0136] The fusion scoring method preserves the sensitivity differences of each dimension when integrating multi-dimensional scores, preventing any single dimension from being too strong or too weak and interfering with the overall judgment. It also possesses differentiability, supporting future optimization of weight allocation through machine learning.

[0137] Furthermore, S104: Activate the abnormal state blocking mechanism and self-locking control mechanism for power equipment, including:

[0138] S104-1: Issue a task interruption command, which suspends all unexecuted operation commands and suspends the current session thread by blocking the command issuance channel in the process management queue; the freeze operation is set to a non-preemptive state by the process priority scheduler to ensure that the interruption is completed within milliseconds and prevents erroneous or delayed commands from entering the downstream execution chain.

[0139] S104-2: Activate the interaction locking mechanism, identify the terminal screen contact area and trigger duration through the input event listener, judge and discard input events below the set threshold, thereby filtering unintended triggered behaviors; load the interaction layer blocking strategy, redirect all GUI controls to a silent state, and retain only the risk confirmation channel and log reporting interface to prevent users from continuing to perform erroneous operations before the risk is eliminated.

[0140] S104-3: Implement hard interruption of instruction execution based on Trusted Execution Environment (TEE). The specific process is as follows: Once the blocking mode is triggered, the Trusted Execution Environment sends a stop command to the driver layer and hardware interface in the execution link. By unloading device control permissions and disabling data interface operations, it cuts off any form of physical or logical output capability. The hard interruption operation is unbypassable and is accompanied by an automatic timed lock-up strategy. It remains locked until a legitimate remote unlock command or physical verification signal is received.

[0141] S104-4: While the blocking process is completed, the current operation context information is summarized to generate a structured risk event log. The log content includes: trigger timestamp, participating device ID, location data, operator identity information, biometric authentication feature summary, consistency score details, inconsistency dimension markers, and blocking trigger factors. The log file is locally encrypted and cached using SM4-CBC mode, then uploaded to the server and written to the blockchain log storage channel to form an immutable audit record.

[0142] Furthermore, when the fusion score exceeds a set threshold, a remote intervention mechanism and a dynamic unlocking mechanism are activated for the power equipment, including:

[0143] (1) The on-site terminal constructs and sends a remote intervention request packet: After the fusion score meets the security conditions, the PAD terminal first extracts the complete operation context data from the local cache; the terminal encapsulates the operation context data according to the hierarchical structure to form a remote intervention request packet, and uses double-layer encryption protection; after encapsulation, the request packet is sent to the remote monitoring platform through the TLS two-way authentication security channel, and waits for confirmation; if no confirmation is received within the set time, the terminal automatically triggers the retransmission process and records the security log to ensure the reliability and traceability of data transmission;

[0144] The operation context data includes: task number, session ID, unique hardware identifier of the terminal, device identity segment and location code segment obtained by decrypting the QR code, real-time spatial location vector (including attitude angle and error ellipse parameters), operator biometric summary and permission verification result, and is supplemented with timestamp and status hash to ensure that the data is tamper-proof.

[0145] The dual-layer encryption protection includes: the outer layer uses SM2 asymmetric key exchange to generate a one-time session key, and the inner layer uses SM4-CBC to encrypt the business data and attach an SM3 integrity check value;

[0146] (2) Remote backend parses request packet and reconstructs operation context: After receiving the encrypted request packet, the backend uses the private key to complete the outer SM2 decryption, obtains the session-level symmetric key, and then decrypts the inner SM4 data and verifies the SM3 hash value to confirm integrity.

[0147] Subsequently, the anomaly analysis engine replays the operation process according to the timestamp order, restores the QR code scanning sequence, spatial trajectory and permission comparison results, and performs an integrated comparison with the target state vector extracted from the task database to generate a complete operation state snapshot.

[0148] The backend not only verifies the validity of spatial location and task tickets, but also combines the permission policy tree to evaluate the matching degree of operator role, task level and biometrics, and marks potential anomalies or differences, providing a reliable contextual basis for subsequent intervention decisions.

[0149] (3) Generate and issue verifiable remote unlocking or intervention commands in the background;

[0150] After the operation status snapshot is confirmed to be legitimate and meets the intervention conditions, the background policy engine generates a structured remote control instruction based on the parsing results. The instruction includes key fields such as the target terminal and device identifier, the allowed unlocking operation range, the control policy number, the effective time window and revocation conditions, the task number and the issuance time.

[0151] After the instruction is generated, the HSM module completes the SM2 digital signature and calculates the SM3 integrity check value, while attaching a one-time authorization credential to bind the instruction to the current task context.

[0152] The backend sends the instruction to the terminal through a secure channel and generates a chain audit log locally, writing the instruction ID, generation time, signature digest and other information into the blockchain evidence storage module to ensure the non-repudiation and traceability of the entire remote intervention process.

[0153] (4) The on-site terminal performs phased dynamic unlocking;

[0154] Upon receiving a remote instruction, the terminal immediately completes digital signature verification, integrity verification, and one-time credential verification. After successful verification, the terminal state is switched from fully self-locked to unlocking preparation state.

[0155] The unlocking process is divided into three stages: communication channel restoration, interactive function restart, and operation interface remapping, which are executed sequentially.

[0156] During the communication channel recovery phase, the terminal rebuilds the data heartbeat and log reporting channels with the backend and monitors communication stability in real time.

[0157] During the interactive function restart phase, input event listening is gradually restored, and non-intent-triggered behaviors are filtered out by touch area and time thresholds.

[0158] During the interface remapping phase, the display changes from a warning state to a restricted operation mode, while simultaneously showing the remaining unlock time, current permission level, and remote monitoring indicator.

[0159] Each stage generates a local audit log with timestamps and status hashes, which are then encrypted with SM4 and asynchronously uploaded to the backend and written to the blockchain for evidence storage.

[0160] If a communication anomaly, inconsistent status, or verification failure occurs at any stage, the terminal immediately reverts to a fully locked state and reports a risk alarm, ensuring that the dynamic unlocking process has redundant revert and security closed-loop characteristics.

[0161] To enhance the auditability and non-repudiation of the unlocking process, a multi-level authentication chain is introduced. High-sensitivity control commands must be bound to attribute-based identity claims and dynamic authentication credentials, and support an atomic chained verification mechanism to achieve consistent binding between unlocking requests and action results. All key authentications and command histories are synchronously generated into operation logs, with embedded integrity verification hash values, and connected to the blockchain trusted evidence storage module to form a traceable evidence chain.

[0162] Example 2

[0163] This embodiment provides an intelligent control system for preventing misoperation of power equipment based on multimodal data;

[0164] A power equipment anti-misoperation intelligent control system based on multimodal data includes:

[0165] The decryption module is configured to: scan the QR code pre-attached on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest.

[0166] The multimodal data acquisition module is configured to: acquire multimodal data of the power equipment to be maintained, store the acquired multimodal data in the blockchain; preprocess the acquired multimodal data to obtain the current state vector of the equipment identity, the current state vector of the spatial location, and the current state vector of the permission digest;

[0167] The consistency scoring module is configured to: calculate the consistency score of each of the three attributes based on the target state vector and the current state vector of the device identity, spatial location, and permission summary; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than a set threshold. If it is, it indicates that the multimodal data acquisition has failed and the multimodal data is re-acquired; if not, proceed to the verification scoring module.

[0168] The verification and scoring module is configured to: obtain a fusion score based on the verification scores of the three attributes; determine whether the fusion score is less than a set threshold; if so, the power equipment is in an abnormal state, and activate the abnormal state blocking mechanism and self-locking control mechanism for the power equipment; when the fusion score is greater than the set threshold, activate the remote intervention mechanism and dynamic unlocking mechanism for the power equipment.

[0169] This embodiment is applicable to high-risk operation scenarios such as power distribution rooms and substations. It has key functions such as operator identity recognition, spatial pose verification, intelligent permission matching, abnormal blocking response, remote collaborative unlocking, and full-process audit tracking. It constructs a fully closed-loop intelligent anti-misoperation control framework with the QR code key combination mechanism as the core.

[0170] The overall system architecture includes the following main functional components:

[0171] Multimodal data acquisition terminal (PAD): Serving as the entry point for operation execution, it integrates an image acquisition module, a UWB positioning module, a GNSS module, a biometric module, and a high-precision clock unit. The image module uses the YOLO model to achieve QR code recognition and device visual positioning. The positioning module achieves centimeter-level pose perception through tightly coupled filtering. The biometric module supports multimodal fusion liveness recognition using near-infrared, 3D structured light, and other technologies. The time module is used for microsecond-level data synchronization to ensure accurate alignment of information across different modalities.

[0172] The three-segment dynamic QR code key fragment encoding module: The system generates a dynamic QR code with a three-segment key fragment structure for each target device, encoding the device identity segment, location constraint segment, and permission digest segment respectively. Each segment is generated based on chaotic encryption, spatial feature embedding, and zero-knowledge authorization, and is encrypted and encapsulated using national cryptographic algorithms. The QR code can only be legally combined within a specific time, space, and task permission context to form a unique and one-time operation authorization credential.

[0173] Multimodal Consistency Verification Module: After scanning the code at the terminal, the system constructs an on-site operation state vector and performs multi-dimensional semantic alignment with the task target state vector in a trusted execution environment. By introducing a device identity matching algorithm, a location legality model, and a permission policy evaluation engine, the system verifies and scores the three dimensions respectively, and generates a final consistency score through a weighted geometric mean function, which serves as the quantitative basis for behavior legality.

[0174] Abnormal State Interception and Terminal Self-Locking Module: When the consistency score falls below a threshold, the system triggers an interception mechanism, freezing the task flow, locking user input, switching the warning interface, and suppressing accidental touch signals through contact area recognition and time filtering mechanisms. Graphical and voice prompts are generated based on the anomaly type to clarify risk attribution and improve operator on-site response efficiency. The interception process generates structured log records, encapsulating information such as operation time, anomaly dimension, and score details.

[0175] Remote intervention and dynamic unlocking module: The system supports remote review and restoration of blocking actions in the background. The PAD terminal transmits encrypted abnormal status packets via a secure channel, containing fields such as task information, status differences, location trajectory, and permission context. The background event parsing engine reconstructs the operation process and generates unlocking commands by combining visual reconstruction and a policy engine. The commands must undergo secondary authentication by the operator (biometric + IC card) and be signed by the hardware security module. After receiving the legitimate command, the terminal restores operating permissions in stages.

[0176] The full lifecycle log and blockchain evidence storage module generates standardized log entries at each key step (scanning and identification, status verification, blocking response, remote unlocking, etc.), calculates digital fingerprints using SM3, and synchronizes them to a relational database and a full-text search engine via dual channels. The system aggregates log summaries based on Merkle trees and writes the hash root to the Hyperledger Fabric consortium blockchain using a lightweight consensus protocol, achieving tamper-proof evidence storage at the log set level and supporting subsequent auditing and compliance verification.

[0177] Through the collaborative operation of the above components, this embodiment realizes a full-link intelligent management and control system for preventing misoperation, with dynamic QR code splicing as the core, multimodal perception as the support, consistency scoring as the basis, anomaly blocking as the response, remote control as the remedy, and trusted logs as the guarantee. It has significant advantages in ensuring operational safety, system reliability, and closed-loop operation, and is applicable to multiple critical infrastructure safety control scenarios such as power, rail transit, and industrial automation.

[0178] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A smart control method for preventing misoperation of power equipment based on multimodal data, characterized in that, include: Scan the QR code that is pre-posted on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest. Multimodal data of the power equipment to be maintained is collected and stored in the blockchain. The collected multimodal data is preprocessed to obtain the current state vector of device identity, the current state vector of spatial location, and the current state vector of permission digest; Based on the target state vector and current state vector of device identity, spatial location, and permission summary, calculate the consistency score for each of the three attributes; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than a set threshold. If it is, it indicates that the multimodal data acquisition has failed, and the multimodal data is re-acquired; otherwise, proceed to the next step. Based on the target state vector and the current state vector of the device identity, a consistency score between device identity modalities is calculated, including: Among them, F image The device identity target state vector is generated from QR code image features or depth features, with a dimension of 128; F qr The current state vector for the device identity, containing the ID hash or encrypted vector representation; η is the squared Euclidean distance, used to measure the difference between two identity feature vectors; η is the identity consistency sensitivity coefficient, controlling the influence of distance on the score; the larger the value, the more sensitive the score is to the difference; exp(x) is an exponential function, equivalent to e^(-∞, 1 / 2)^2. x This maps a real number to a positive number, while smoothing the change around 0, and is used for probability or rating normalization. (D) identity This indicates a consistency score between device identity modalities; Based on the target state vector and the current state vector of the spatial location, the consistency score D between the spatial location modes is calculated. spatial ,include: Among them, P img P is the target state vector in spatial location, including 3D coordinates and attitude angles; sensor D represents the current state vector of the spatial location measured by the positioning sensor; μ represents the spatial consistency sensitivity coefficient, which controls the response speed of the score to the magnitude of the error. The larger μ is, the more sensitive the score is to spatial deviation; spatial Indicates the consistency score between spatial location modes; Based on the target state vector and the current state vector of the permission digest, the consistency score D among the permission digest modalities is calculated. feature ,include: Among them, V current V is the current state vector of the permission summary; target The target state vector for the permission summary; the dim() function returns the vector dimension; N dim ρ is the standard total dimension; NaNCount(V) is the number of missing or outliers in the vector; ρ is the set integrity sensitivity coefficient, which penalizes missing or outliers in the permission feature vector. The larger ρ is, the more significant the negative impact of missing values ​​on the score; D feature This indicates the consistency score between permission summary modalities; A fusion score is obtained based on the verification scores of the three attributes. It is then determined whether the fusion score is less than a set threshold. If it is, the power equipment is in an abnormal state, and the abnormal state blocking mechanism and self-locking control mechanism are activated for the power equipment. If the fusion score is greater than the set threshold, the remote intervention mechanism and dynamic unlocking mechanism are activated for the power equipment. The system calculates the verification scores for device identity, spatial location, and permission digest. Device identity verification score S dev The calculation formula is: S dev =δ×Match str (ID target ,ID current ); Where δ is the matching weight factor; ID target The device target ID obtained by decrypting the QR code; ID current The current device ID obtained through real-time data collection; Match str (x, y) is a string matching function, defined as: Spatial location verification score S pos The calculation formula is: Among them, P current P represents the measured coordinates of the current location of the barcode scanning device. target The target position coordinates are preset for the task; σ is the position error tolerance radius, generated by the dynamic error ellipse model; I GeoFence () represents the geofencing exponential function, which includes: Where V represents the pre-defined geofence area for the task; Permission digest verification score S perm The calculation formula is: Where K represents the total number of permission attributes, and k represents the permission attribute number being compared; ω k Represents the weighting factor for each permission attribute dimension; This represents the operator attribute value obtained through biometric authentication or on-site collection in the k-th dimension; This represents the k-th corresponding attribute value in the permission digest obtained by decrypting the QR code. The Match(x,y) function is a modal feature matching function used to calculate the degree of matching between the current operator's biometric features and attribute information and the permission summary obtained by decrypting the QR code. When the input is a discrete attribute, exact matching or rule matching is used. When the input is a continuous feature vector, cosine similarity is used for calculation and normalized to the range of 0 to 1. The closer the return value is to 1, the higher the degree of matching. If the input x and y are both discrete attributes, and the discrete attributes match perfectly, then Match(x,y) = 1; If the input x and y are both discrete attributes, and the discrete attributes do not match, then Match(x,y) = 0; If the input x and y are both continuous feature vectors, then 2. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 1, characterized in that, The QR code includes: three encrypted key fragments; The three encryption key segments include: an encrypted device identifier segment, an encrypted location encoding segment, and an encrypted permission digest segment; The device identification segment includes: power equipment ID, current task timestamp, and unique identification code; the encrypted device identification segment is obtained by encrypting the device identification segment using the national cryptographic algorithm SM4; The location encoding segment includes: the absolute coordinates of the operator's target work area, the topological location label code, and the preset operator spatial location; the encrypted location encoding segment is obtained by encrypting the location encoding segment using the ECC encryption algorithm; The permission digest segment includes: operator permission information, role attributes, and preset operator biometric authentication data; the encrypted permission digest segment is obtained by encrypting the permission digest segment.

3. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 2, characterized in that, The device identifier segment is encrypted using a dynamic device identifier encryption algorithm based on chaotic mapping. The encryption formula is as follows: Among them, E dev This represents the result of encryption processing of the original device identification information at the device level; S is a piecewise linear chaotic mapping function, with input values ​​oscillating between 0 and 1; γ is the chaos intensity coefficient; x i Let H be the i-th random number generated by the chaotic system; H is the SM3 hash function; T task The task timestamp is accurate to milliseconds; Coord is the normalized device space coordinate value; DeviceID is the device ID, and ⊙ represents element-wise multiplication. This represents the XOR operation.

4. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 1, characterized in that, Based on the individual verification scores of the three attributes, a fusion score is obtained, including: Among them, S dev The verification score indicating device identity; S pos The verification score representing the spatial location; S perm The verification score represents the permission digest, where α represents the confidence level of device identity; β represents the confidence level of spatial location; γ represents the confidence level of the permission digest; and the fusion score S... score A graded response is triggered after comparison with a preset threshold: when S score If the value is ≥0.85, the current operation is valid and allowed to execute; if 0.75≤S score If S < 0.85, then the enhanced verification process is initiated, where the enhanced verification process refers to secondary biometric identification; when S score If the value is less than 0.75, a hard block will be executed, and a security audit log will be generated.

5. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 1, characterized in that, The abnormal state blocking mechanism and self-locking control mechanism for power equipment include: Issue a task interruption command to suspend all unexecuted operation commands and suspend the current session thread by blocking the command issuance channel in the process management queue; the freeze operation is set to a non-preemptive state by the process priority scheduler to ensure that the interruption is completed within milliseconds and prevent erroneous or delayed commands from entering the downstream execution chain. An interaction locking mechanism is activated, which uses an input event listener to identify the terminal screen contact area and trigger duration. Input events below a set threshold are judged and discarded to filter out unintended triggering behaviors. An interaction layer blocking strategy is loaded to redirect all GUI controls to a silent state, retaining only the risk confirmation channel and log reporting interface to prevent users from continuing to perform erroneous operations before the risk is eliminated. The hard interruption of instruction execution is achieved by relying on the trusted execution environment. The specific process is as follows: once the blocking mode is triggered, the trusted execution environment sends a stop instruction to the driver layer and hardware interface in the execution link. By unloading the device control authority and disabling the data interface operation, any form of physical or logical output capability is cut off. The hard interruption operation is unbypassable and is accompanied by an automatic timed lock-up strategy. The locked state is maintained until a legitimate remote unlock instruction or physical verification signal is received. While the blocking process is completed, the current operation context information is summarized to generate a structured risk event log. The log content includes: trigger timestamp, participating device ID, location data, operator identity information, biometric authentication feature summary, consistency score details, inconsistency dimension markers and blocking trigger factors. The log file is locally encrypted and cached using SM4-CBC mode, then uploaded to the server and written to the blockchain log storage channel to form an immutable audit record.

6. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 1, characterized in that, When the fusion score exceeds a set threshold, a remote intervention mechanism and a dynamic unlocking mechanism are activated for the power equipment, including: (1) The on-site terminal constructs and sends a remote intervention request packet; (2) The remote backend parses the request packet and reconstructs the operation context; (3) Generate and issue verifiable remote unlocking or intervention commands in the background; (4) The on-site terminal performs phased dynamic unlocking.

7. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 6, characterized in that, The field terminal constructs and sends a remote intervention request packet, including: After the fusion score meets the security conditions, the PAD terminal first extracts the complete operation context data from the local cache. The terminal encapsulates the operation context data according to a hierarchical structure to form a remote intervention request packet, and uses double-layer encryption protection. After encapsulation, the request packet is sent to the remote monitoring platform through a TLS two-way authentication secure channel, while waiting for confirmation. If no confirmation is received within the set time, the terminal automatically triggers a retransmission process and records a security log to ensure the reliability and traceability of data transmission.

8. The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in claim 6, characterized in that, The remote backend parses the request packet and reconstructs the operation context. include: After receiving the encrypted request packet, the backend uses the private key to decrypt the outer SM2 layer, obtains the session-level symmetric key, and then decrypts the inner SM4 data and verifies the SM3 hash value to confirm its integrity. Subsequently, the anomaly analysis engine replays the operation process according to the timestamp order, restores the QR code scanning sequence, spatial trajectory and permission comparison results, and performs an integrated comparison with the target state vector extracted from the task database to generate a complete operation state snapshot. The backend not only verifies the validity of spatial location and task tickets, but also assesses the matching degree of operator role, task level and biometrics in combination with permission policy tree, and marks potential abnormal or discrepancy dimensions to provide contextual basis for subsequent intervention decisions.

9. A power equipment anti-misoperation intelligent control system based on multimodal data, characterized in that: The intelligent control method for preventing misoperation of power equipment based on multimodal data as described in any one of claims 1-8 includes: The decryption module is configured to: scan the QR code pre-attached on the power equipment to be maintained, extract three encrypted key fragments from the QR code, decrypt the three encrypted key fragments, and obtain the target state vector of the equipment identity, the target state vector of the spatial location, and the target state vector of the permission digest. The multimodal data acquisition module is configured to: acquire multimodal data of the power equipment to be maintained, store the acquired multimodal data in the blockchain; preprocess the acquired multimodal data to obtain the current state vector of the equipment identity, the current state vector of the spatial location, and the current state vector of the permission digest; The consistency scoring module is configured to: calculate the consistency score of each of the three attributes based on the target state vector and the current state vector of the device identity, spatial location, and permission summary; calculate the total consistency score based on the consistency scores of the three attributes; determine whether the total consistency score is less than a set threshold, and if so, it indicates that the multimodal data acquisition has failed and the multimodal data is re-acquired. If not, proceed to the verification and scoring module; The verification and scoring module is configured to: obtain a fusion score based on the verification scores of the three attributes; determine whether the fusion score is less than a set threshold; if so, the power equipment is in an abnormal state, and activate the abnormal state blocking mechanism and self-locking control mechanism for the power equipment; when the fusion score is greater than the set threshold, activate the remote intervention mechanism and dynamic unlocking mechanism for the power equipment.

Images