A service-aware multi-modal fusion bank intelligent cabinet access control system and method

Abstract

The application discloses a service-aware multi-modal fusion bank intelligent cabinet access control system and method, and belongs to the technical field of bank physical security protection and intelligent access control. The cloud key management subsystem is used for composite verification of the work order state, geographical position and time window, and a dynamic key is generated. The near field communication module in the local execution subsystem receives the dynamic key, and the millimeter wave radar module collects the number of people in front of the door at the same time. The master control module verifies the validity of the key and when there are multiple people, only the authorized operator initiating the request is subjected to live detection. If the verification is passed, the door is opened, the facial images of all the entering personnel are captured, a log is generated for recording only and no comparison is made, and if the verification is not passed, the door is locked and an alarm is given. The event hash and the previous block hash are linked to generate a new block and are synchronized to the cloud through the block chain storage subsystem. The application solves the problems of disconnection between the existing access control permission and the service, lack of environmental awareness and untrustworthy audit, and realizes the collaborative control of dynamic authorization and trusted storage.

Description

Technical Field

[0001] This invention relates to the field of physical security protection and intelligent access control technology for banks, and in particular to a business-aware multimodal fusion intelligent bank cabinet access control system and method. Background Technology

[0002] Bank branch computer rooms and network cabinets house core transaction data and critical network equipment; physical security is the first line of defense for bank network security. Currently, bank cabinet access control generally uses mechanical locks, static RFID cards, or single biometric identification solutions. These technologies have revealed the following systemic problems during long-term operation: First, permissions are static and disconnected from business operations. Mechanical keys and RFID cards have permissions that are valid indefinitely once issued, lacking a binding relationship with maintenance work orders. Permission revocation relies on manual operation, resulting in significant delays. Issues such as the inability to promptly revoke permissions for departing employees, permissions remaining valid after work orders are completed, and the need to replace the entire lock set when keys are lost—these shortcomings leave permissions in a state of perpetual uncontrollability. While existing technologies offer dynamic authorization methods based on ITSM systems, both are geared towards data access control within the IT system. Their dynamic authorization models are completely detached from physical access control hardware, failing to address and be unable to extend to physical security elements such as door frame deformation detection, personnel sensing, and anti-tailgating measures.

[0003] Second, the authentication process is isolated and lacks environmental awareness. The existing authentication can only answer "who is requesting to open the door", but cannot detect whether there are risk scenarios such as tailgating or gathering in front of the door. It also cannot dynamically adjust the authentication strategy according to environmental conditions such as lighting and occlusion, resulting in a significant decrease in recognition accuracy in low light or when wearing masks, and lack of effective defense against tailgating attacks.

[0004] Third, the system suffers from passive defense and weak auditing. Identity verification is only performed at the authentication point, lacking the ability to detect and proactively warn against physical attacks such as lock picking and door frame deformation. Operation logs are stored in a local, insecure area, allowing for arbitrary addition, deletion, and modification; they lack cryptographically anchored anti-tampering mechanisms and cannot provide legally valid and credible evidence.

[0005] In summary, existing solutions have significant shortcomings in four dimensions: access control, environmental awareness, proactive defense, and trusted auditing. There is an urgent need for an intelligent access control solution that can dynamically bind permissions to business functions, possess environmental awareness capabilities, support proactive early warning, and build a trusted auditing chain. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems in the related art.

[0007] To address this, this invention proposes a business-aware multimodal fusion intelligent bank cabinet access control system. Through deep integration of a cloud-based key management subsystem and an ITSM system, dynamic keys are generated based on work order status. A parallel sensing mechanism combining millimeter-wave radar and key verification in the local execution subsystem, combined with asymmetric anti-tailgating rules of "responsible person locking + object following record," achieves differentiated anti-tailgating control. A hash chain-based immutable auditing system is constructed through a blockchain evidence storage subsystem. This invention upgrades cabinet access control from a static, passive, and isolated approach to a dynamic, proactive, and collaborative security paradigm, solving the problems of existing technologies such as disconnect between permissions and business operations, lack of context awareness, passive defense, and unreliable auditing.

[0008] Another objective of this invention is to propose a business-aware multimodal fusion access control method for bank smart cabinets.

[0009] To achieve the above objectives, the present invention proposes a business-aware multimodal fusion intelligent bank cabinet access control system, comprising: The cloud-based key management subsystem is used to receive door opening requests from mobile terminals, perform composite verification of work order status, geographical location, and time window, generate a dynamic key after successful verification, and return it to the mobile terminal. The local execution subsystem includes a main control module and a near-field communication module, a millimeter-wave radar module, and an image acquisition module, which are respectively connected to the main control module; The near-field communication module is used to receive the dynamic key forwarded by the mobile terminal and input it into the main control module; The millimeter-wave radar module is used to respond to the activation command sent in parallel by the main control module when it receives the dynamic key, collect data on the number of people in front of the gate, and feed it back to the main control module. The main control module is used to decrypt and verify the dynamic key, and simultaneously receive the number of people. When the key is valid and it is a single person, an opening signal is generated. When the key is valid and it is multiple people, the image acquisition module is triggered to acquire the image in front of the door, and only the authorized operator who initiated the opening request is subjected to liveness detection. If the detection is successful, an opening signal is generated and the image acquisition module is controlled to capture facial images of all entering personnel and generate a log that is only recorded and not compared. If the detection fails, a locking signal and alarm data are generated. The blockchain evidence storage subsystem, connected to the main control module, is used to respond to event evidence storage instructions, obtain the current event data and the hash value of the previous block, calculate the current event hash and link it with the previous hash value to generate a new block, write it to the local node and synchronize it to the cloud node.

[0010] An embodiment of the present invention provides a business-aware multimodal fusion smart bank cabinet access control system that may also have the following additional technical features: In one embodiment of the present invention, when the cloud key management subsystem performs the composite verification, it specifically performs the following sub-steps: obtaining the work order identifier carried in the door opening request, querying the corresponding work order status data from the ITSM system using the work order identifier as an index, as the first input data; determining the validity of the first input data: when the work order status data indicates "approved" and not executed, generating a work order verification passed status as the first intermediate status data; otherwise, generating a rejection instruction and terminating the process; in response to the work order verification passed status, sending a status update instruction to the ITSM system, atomically switching the work order status data from "approved" to "executed", generating status lock confirmation data as the second intermediate status data, the status lock confirmation data being used to prevent the same work order from being reused.

[0011] In one embodiment of the present invention, when the cloud key management subsystem generates the dynamic key, it specifically performs the following sub-steps: obtaining the verified work order identifier, the current system timestamp, and a 16-byte random number output by the random number generation module as plaintext data to be encrypted; concatenating the plaintext data according to a preset format to form a target plaintext string; calling the national cryptographic SM4 symmetric encryption algorithm and using the pre-stored master key to perform encryption operations on the target plaintext string to generate a ciphertext data packet as the dynamic key; the effective time window of the dynamic key is 3 seconds, and the current system timestamp is rounded to the granularity of the effective time window to ensure that the dynamic keys generated within the same time window are consistent.

[0012] In one embodiment of the present invention, when the millimeter-wave radar module collects the number of people in front of the door, it specifically performs the following sub-steps: In response to the activation command sent by the main control module, the millimeter-wave radar module transmits millimeter-wave signals to the area in front of the door and receives the target reflected echo signals as raw sensing input data; performs intermediate frequency signal decoding on the raw sensing input data, extracts the distance features and velocity features of each target point, and generates a point cloud dataset; performs clustering filtering on the point cloud dataset, counts the number of valid targets, generates the number of people in the area in front of the door and the dynamic feature data of the people, and feeds it back to the main control module.

[0013] In one embodiment of the present invention, when the main control module performs the liveness detection on the authorized operator, it specifically performs the following sub-steps: receiving multimodal image data of the area in front of the door acquired by the image acquisition module as input data, the multimodal image data including visible light image and infrared depth image; extracting the face region image of the authorized operator from the visible light image to generate first detection data; sequentially executing action instruction verification on the first detection data to determine whether the authorized operator has completed the specified action according to the randomly generated instruction, and generating a first liveness confidence score; simultaneously, performing three-dimensional facial structure reconstruction on the infrared depth image to generate facial depth feature data, and comparing it with a pre-stored real face depth template to generate a second liveness confidence score; weighted fusion of the first liveness confidence score and the second liveness confidence score to generate a comprehensive liveness detection score as detection result data.

[0014] In one embodiment of the present invention, when the main control module performs the liveness detection on the authorized operator, it further performs the following sub-steps: acquiring the current ambient light intensity data as a first environmental parameter, and acquiring the facial occlusion ratio data of the authorized operator in the visible light image as a second environmental parameter; when the first environmental parameter is lower than a preset light threshold, reducing the judgment benchmark threshold of the comprehensive liveness detection score, and activating infrared auxiliary illumination; when the second environmental parameter indicates that there is facial occlusion, switching to the periorbital local area recognition mode, adjusting the judgment benchmark threshold of liveness detection to the preset occlusion mode threshold; and outputting the final liveness determination result after dynamic adjustment of the environmental parameters.

[0015] In one embodiment of the present invention, when the main control module generates the log that only records without comparison, it specifically performs the following sub-steps: In response to the liveness detection pass result, the main control module generates a continuous capture command and outputs it to the image acquisition module; In response to the continuous capture command, the image acquisition module performs serialized capture of the facial images of all personnel entering the door area to generate original facial image sequence data; The main control module obtains the original facial image sequence data and obtains the work order identifier and operator identifier corresponding to the current door opening event, associates and packages the above data, and generates the log data of the visitor who only records without comparison; The log data of the visitor who only records does not trigger any identity comparison operation and is only stored locally and in the cloud as traceability evidence.

[0016] In one embodiment of the present invention, when the blockchain evidence storage subsystem writes the new block to the local node and synchronizes it to the cloud node, it specifically performs the following sub-steps: It obtains current event data, current trusted timestamp, and the hash value of the previous block as input data. The current event data includes an event type identifier and a sensor status snapshot at the time the event occurred. It calls the SHA256 hash algorithm to perform hash digest calculation on the concatenated string of the previous block hash value, the current event data, and the current trusted timestamp, generating a current event hash value. It links the current event hash value with the previous block hash value to generate target new block data containing a chain structure, and writes it to the storage area of ​​the local edge blockchain node. The local edge blockchain node packages multiple new block data into Merkle tree nodes in chronological order and uploads them in batches to the cloud consortium blockchain node through a preset data synchronization channel, completing cross-node redundant storage.

[0017] In one embodiment of the present invention, the local execution subsystem further includes a door magnetic state sensor and a door frame deformation sensor connected to the main control module; the door magnetic state sensor is used to collect real-time data on the opening and closing status of the cabinet door and the locking status of the electromagnetic lock, and feeds it back to the main control module as first state input data; the door frame deformation sensor is used to collect real-time data on the deformation displacement of the cabinet door frame, and feeds it back to the main control module as second state input data; the main control module is further used to: when the first state input data indicates that the electromagnetic lock is abnormally opened in an unauthorized state, or when the second state input data exceeds a preset deformation threshold, determine it as a physical attack event and generate emergency alarm data; in response to the generation of the emergency alarm data, the main control module executes the following sub-steps in parallel: triggering the image acquisition module to record on-site video data, driving the audible and visual alarm actuator to issue an audible and visual alarm, and uploading the emergency alarm data and the on-site video data to the cloud platform.

[0018] To achieve the above objectives, another aspect of the present invention proposes a business-aware multimodal fusion access control method for bank smart cabinets, comprising: The system receives a door opening request from a mobile terminal, performs a composite verification of the work order status, geographical location, and time window, and generates a dynamic key and returns it to the mobile terminal after the verification is successful. The system receives the dynamic key forwarded by the mobile terminal for verification, and simultaneously senses the number of people in front of the door using millimeter-wave radar. The door lock is opened when the key verification is valid and it is a single person; when the key verification is valid and it is multiple people, only the operator who initiated the request is subject to liveness detection. If the liveness detection is successful, the door is opened and all people entering are captured and recorded without comparison. If the liveness detection is unsuccessful, the door is locked and an alarm is triggered. In response to door opening, door locking, or alarm events, calculate the current event hash and link it with the hash value of the previous block to generate a new block, write it to the local blockchain, and synchronize it to the cloud.

[0019] This invention discloses a business-aware multimodal fusion smart bank cabinet access control system and method. Through real-time linkage between a cloud-based key management subsystem and an ITSM system, it achieves precise binding of dynamic keys and work order lifecycles, effectively solving the problem of static permissions. Using a parallel perception mechanism of millimeter-wave radar and key verification, it acquires the environmental context of the number of people at the door while authenticating. Through an asymmetric anti-tailgating rule of "responsible person locking + object following record," it verifies only the responsible person and records followers in multi-person scenarios, balancing security and operational efficiency. Through a lightweight blockchain evidence storage architecture, it constructs an immutable audit system that meets financial-grade regulatory requirements, significantly improving the intelligence level and security protection capabilities of the bank cabinet access control system.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a system architecture diagram of a business-aware multimodal fusion smart bank cabinet access control system according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the process steps of a business-aware multimodal fusion smart bank cabinet access control method according to an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] The following description, with reference to the accompanying drawings, describes a business-aware multimodal fusion smart bank cabinet access control system and method according to an embodiment of the present invention.

[0025] Example 1 like Figure 1 As shown, a business-aware multimodal fusion smart bank cabinet access control system of the present invention includes: The cloud-based key management subsystem is used to receive door opening requests from mobile terminals, perform composite verification of work order status, geographical location, and time window, and generate a dynamic key after successful verification and return it to the mobile terminal.

[0026] It should be further explained that the cloud-based key management subsystem is deployed in the bank's intranet cloud server cluster and interacts with the bank's ITSM system through a standard RESTful API interface. This subsystem consists of two parts: a key management server and a blockchain evidence storage service node. The key management server is configured to: receive door opening requests from the mobile maintenance app, with the request data packet containing at least the work order number, operator ID, mobile terminal GPS coordinates, and request timestamp; query the ITSM system for the current status of the work order using the work order number as an index; simultaneously verify whether the mobile terminal's GPS coordinates fall within the geofence of the target cabinet; and verify whether the current request time falls within the valid execution time window specified in the work order. After all three verifications pass, the key management server atomically updates the work order status in the ITSM system from "approved" to "in execution"—this status switching operation is executed in the ITSM system using a database transaction method, ensuring the atomicity of the status change and preventing the same work order from being used multiple times to initiate door opening requests.

[0027] Further explanation is needed regarding the dynamic key generation logic: The key management server obtains the work order number, the current system timestamp, and a 16-byte random number output by the cryptographically secure random number generator, concatenating them into a plaintext string in the format "work order number||timestamp||random number"; it then calls the national standard SM4 symmetric encryption algorithm, using the system's preset master key to encrypt the plaintext string, generating a ciphertext data packet as the dynamic key. Since the timestamp is rounded to the nearest 3 seconds, the key generated for the same work order remains consistent within the same 3-second time window; however, the key automatically changes after crossing time windows, and due to the introduction of the 16-byte random number, even two requests within the same time window will generate different keys, exhibiting unpredictability. The key management server encapsulates the dynamic key into a JSON data packet and returns it to the mobile maintenance app via an HTTPS encrypted channel.

[0028] The local execution subsystem includes a main control module and a near-field communication module, a millimeter-wave radar module, and an image acquisition module, which are respectively connected to the main control module.

[0029] It should be further noted that the local execution subsystem is installed on the rack door and consists of a core main control board and peripheral modules connected to it via ribbon cables and a serial bus. The hardware selection and functions of each module are as follows: (1) Main control module: It adopts an industrial-grade ARM Cortex-A55 processor with a main frequency of not less than 1.8GHz, equipped with 2GB DDR4 RAM and 8GB eMMC local storage, and pre-installed with an embedded Linux real-time operating system. As the decision-making center of the local execution subsystem, the main control module is responsible for running the authentication logic, fusing and processing sensor data, controlling the door lock actuator, and communicating securely with the cloud.

[0030] (2) Near Field Communication Module: A 13.56MHz NFC card reader module conforming to ISO / IEC 14443 standard is adopted, supporting Type A / Type B card types and NFC point-to-point communication mode. It is connected to the main control module through a serial peripheral interface. This module is used to read dynamic key data packets transmitted by mobile terminals through the NFC function at close range.

[0031] (3) Millimeter-wave radar module: It adopts a 60GHz band millimeter-wave radar sensor, supports FMCW frequency modulated continuous wave modulation, has a configurable detection range of 0.58 meters, and an angular resolution of not less than 15 degrees. It connects to the main control module via a UART serial port. This module is installed above the rack door, and its detection area covers 120mm in front of the rack door. The fan-shaped area can output information on the number, distance, speed, and angle of targets within the detection area in real time.

[0032] (4) Image Acquisition Module: A binocular camera module is used, including a 4-megapixel visible light RGB camera and an infrared 3D structured light depth camera, which is connected to the main control module through the MIPI-CSI interface. This module is used to acquire multimodal image data of the area in front of the door and supports face detection, liveness detection and video recording functions.

[0033] (5) Actuator module: including electromagnetic lock drive circuit and audible and visual alarm. The electromagnetic lock adopts a 280kg-class magnetic lock. The drive circuit receives the door opening / locking signal from the main control module through the GPIO pin to control the opening and closing of the door lock. The audible and visual alarm consists of a high-brightness LED indicator and a buzzer. It receives the alarm trigger signal from the main control module through the GPIO pin.

[0034] The near-field communication module is used to receive the dynamic key forwarded by the mobile terminal and input it into the main control module.

[0035] It should be further explained that after the mobile maintenance app obtains the dynamic key, it prompts maintenance personnel to bring the phone's NFC antenna area close to the NFC card reader area of ​​the cabinet. Upon detecting the phone's NFC radio frequency field, the NFC reader establishes an ISO-DEP data exchange channel, receives the encrypted data packet of the dynamic key sent by the phone in NFC data exchange format, and forwards this data packet to the main control module via the SPI bus. The extremely short NFC communication distance effectively prevents remote eavesdropping and man-in-the-middle attacks.

[0036] The millimeter-wave radar module is used to respond to the activation command sent in parallel by the main control module when it receives the dynamic key, collect data on the number of people in front of the gate, and feed it back to the main control module.

[0037] It should be further explained that this "parallel execution" mechanism is one of the key collaborative features of this invention. When the main control module begins receiving the dynamic key data packet from the NFC reader via the SPI bus, it sends an activation command to the millimeter-wave radar module via the UART serial port within the same clock cycle. Upon receiving the activation command, the millimeter-wave radar module immediately activates its multi-antenna array, transmitting a 60GHz frequency-modulated continuous wave signal towards the area in front of the door and receiving the target's reflected echo. The radar's back-end processing chip performs intermediate frequency mixing, FFT range dimension processing, MTI moving target detection, and DBSCAN clustering filtering on the echo signal, ultimately outputting the number of people in the area in front of the door and the distance, speed, and angle information of each target. The entire detection and data processing process takes approximately 5080ms, which is roughly matched with the key decryption and verification process, ensuring that the two parallel tasks are completed within approximately the same time window. This allows for the synchronous input of the "key verification result" and the "environmental perception result" to the decision engine—rather than traditional serial processing.

[0038] The main control module is used to decrypt and verify the dynamic key, and simultaneously receive the number of people. When the key is valid and it is a single person, an opening signal is generated. When the key is valid and it is multiple people, the image acquisition module is triggered to acquire images in front of the door, and only the authorized operator who initiated the opening request is subjected to liveness detection. If the detection is successful, an opening signal is generated and the image acquisition module is controlled to capture facial images of all entering personnel and generate a log that is only recorded and not compared. If the detection fails, a locking signal and alarm data are generated.

[0039] It should be further explained that the decision-making logic of the main control module is as follows: Dynamic key decryption verification: The main control module calls the same SM4-CBC decryption algorithm as the cloud-based module, using the master key pre-stored in the main control module's secure storage area to decrypt the ciphertext read by NFC, obtaining the plaintext (work order number, timestamp, random number). The verification logic is as follows: the decrypted timestamp is synchronized with the current system time (via NTP, with high precision). Check if the absolute value of the difference (100ms) is less than 3 seconds - if yes, the key is valid, otherwise the key times out; at the same time, check if the work order number is in the local valid whitelist (this whitelist is periodically synchronized from the cloud to the local area).

[0040] Decision branch: After the main control module obtains the key verification result and the millimeter-wave radar number of people determination result (single / multiple people), it executes the following branch: Branch 1 (Single Person + Key Valid): Directly generates a door lock opening drive signal (high-level pulse, lasting 3 seconds), which drives the electromagnetic lock to be de-energized and unlocked, allowing the door to be opened.

[0041] Branch Two (Multiple Users + Valid Key): The main control module does not directly open the door, but instead triggers an anti-tailgating verification process—specifically, it sends a capture command to the image acquisition module via the I2C bus to obtain the visible light image and infrared depth image of the area in front of the door at the current moment as input data. The image acquisition module sends the captured binocular images back to the main control module. The main control module loads the locally pre-stored face recognition model and liveness detection model, detects and extracts the face region of the authorized operator who initiated the door opening request from the visible light image, and performs multimodal liveness detection only on that authorized operator.

[0042] The multimodal liveness detection is performed as follows: the main control module sequentially executes action command verification, infrared 3D structured light verification, and texture analysis on the authorized operator's face region. The three scores are then weighted and fused (with weights of 0.3, 0.4, and 0.3 respectively) to generate a comprehensive liveness detection score.

[0043] If the overall score A dynamic threshold is used to determine the outcome (base threshold 80 points, dynamically adjusted based on environmental conditions). If liveness detection passes, the main control module generates a door lock unlocking drive signal and simultaneously sends a "continuous capture" command to the image acquisition module. The image acquisition module then sequentially captures facial images of all personnel entering the door area at a frequency of 3 frames per second for 10 seconds, generating a raw facial image sequence. The main control module associates and packages the image sequence with the current work order number, operator ID, and timestamp to generate a "Guest Visit Log" (this log only stores the raw images and does not perform any identity verification). The overall score is then considered. If the dynamic threshold fails the liveness detection, the main control module generates an electromagnetic lock holding signal (to maintain the power-off lock state) and an alarm data packet (including the reason for failure and an image screenshot), and reports it to the cloud.

[0044] Branch 3 (Invalid Key): Regardless of the radar sensing results, the main control module refuses to open the door and generates an authentication failure log.

[0045] The blockchain evidence storage subsystem is connected to the main control module and is used to respond to event evidence storage instructions, obtain the current event data and the hash value of the previous block, calculate the current event hash and link it with the previous hash value to generate a new block, write it to the local node and synchronize it to the cloud node.

[0046] It should be further explained that the blockchain evidence storage subsystem runs on the local lightweight blockchain nodes of the main control module and the cloud consortium blockchain nodes. The local nodes store the complete blockchain ledger (including block headers and hash chains), while the cloud nodes are a consortium blockchain composed of nodes from various branches and the head office, using the Byzantine Fault Tolerance (PBFT) consensus mechanism.

[0047] The data storage process is as follows: When the main control module generates any critical event (successful door opening, failed door opening, anti-tailgating verification passed / failed, emergency alarm, abnormal door status), the storage command is automatically triggered. The main control module obtains the current event data (event type code, event description, operator ID, work order number, number of personnel, event timestamp, and sensor status snapshot data—such as door magnetic status, radar personnel count, electromagnetic lock status, etc.), and reads the hash value of the latest block (i.e., the previous block) in the local blockchain node. The hash value of the previous block, the current event data, and the current trusted timestamp are concatenated into a string, and the SHA256 hash algorithm is used to calculate the hash value of the current event. The hash value of the current event and the hash value of the previous block are written together into the block header of the new block, and the new block body stores the original text of the current event data. After the new block is verified by the local node, it is appended to the local ledger.

[0048] Local nodes cache new block data. When the cached amount reaches a preset batch packaging threshold (e.g., 10 events or 5 minutes), multiple new blocks are constructed into a Merkle Tree. Only the root hash of the Merkle Tree is uploaded to the cloud consortium blockchain node—effectively reducing on-chain storage. After the cloud node completes consensus, it writes the root hash to the consortium blockchain and returns the write result to the local node for confirmation. This edge-cloud dual-write mechanism ensures that even if the network is interrupted, the local node continues to cache new blocks, and batch synchronization occurs after the network is restored, ensuring no data loss.

[0049] It is worth mentioning that the complete workflow of this system is as follows: Step 1: The maintenance personnel initiate an opening request through the mobile maintenance APP, submit fingerprint or facial biometrics to complete identity authentication, and then select the corresponding ITSM work order; Step 2: The APP sends the work order number, operator ID, and GPS coordinates to the cloud key management subsystem; Step 3: The cloud key management subsystem performs a composite condition verification (work order status verification – whether it is “approved” and not executed; time window verification – whether the current time is within the time window specified in the work order; location verification – whether the GPS coordinates are within the geofence of the target rack; risk control strategy verification – in conjunction with the risk control system, detect whether the same operator has initiated requests to racks in different locations within a short period of time to prevent “logic jump” attacks). After all verifications are passed, the work order status is atomically updated to “execution”, and a dynamic key is generated and returned to the APP. Step 4: The APP transmits the dynamic key to the NFC reader of the local execution subsystem via NFC. The main control module receives and decrypts the key, while the millimeter-wave radar module acquires the number of people in front of the door in parallel. Step 5: Single person + valid key - open the door directly; Multiple people + valid key - trigger asymmetric anti-tailgating verification, and only perform multimodal liveness detection on the main operator. Step 6: All key events, such as door opening events, verification results, and snapshot logs, are automatically written into the blockchain evidence storage subsystem, forming an immutable and trustworthy audit record.

[0050] Example 2 like Figure 2 As shown, the business-aware multimodal fusion smart bank cabinet access control method of the present invention includes the following steps: Step S1: Receive the door opening request from the mobile terminal, perform a composite verification of the work order status, geographical location, and time window, generate a dynamic key after the verification is successful, and return it to the mobile terminal.

[0051] In one embodiment of the present invention, the cloud key management subsystem receives an HTTPS access request sent by a mobile maintenance APP, parses the request body to obtain the work order number, operator ID, and mobile terminal GPS coordinates. Using the work order number as an index, it calls the ITSM system open API to query the current status, planned execution start time, planned execution end time, and associated cabinet number of the work order. The queried work order status data is then evaluated: if the status is "Approved" and the work order has not yet been marked as "In Progress" or "Completed," the work order status verification passes; otherwise, a rejection response is returned and the process is terminated. Subsequently, the current system time is obtained through the NTP service, and compared with the planned execution time window (start time) of the work order. The time window is checked against the end time. If the current time falls within the time window, the time window verification passes. Finally, the distance between the mobile terminal's GPS coordinates and the target cabinet's latitude and longitude coordinates stored in the database is calculated. If the distance... Setting the preset geofence radius (default 50 meters) ensures successful location verification. After all three verifications pass, the key management server sends a status update request (PATCH method) to the ITSM system, updating the work order status from "Approved" to "Executing." This update operation is committed in the ITSM database as a transaction, ensuring the atomicity of the status change. Once the update is successful, the same work order number will not pass verification again, effectively preventing work order replay attacks. Subsequently, the key management server generates a dynamic key, encrypts it, and returns it to the mobile terminal.

[0052] Step S2: Receive the dynamic key forwarded by the mobile terminal for verification, and simultaneously sense the number of people in front of the door using millimeter-wave radar.

[0053] In one embodiment of the present invention, maintenance personnel bring a mobile terminal (phone) with a dynamically acquired key close to the NFC card reader area on the cabinet door. After detecting the phone's NFC radio frequency field, the NFC reader establishes a data exchange channel, receives the encrypted data packet of the dynamic key sent by the phone, and forwards it to the main control module via the SPI bus. Upon receiving the data packet, the main control module calls the SM4-CBC decryption algorithm, decrypts the encrypted text using the master key preset in its local secure storage area to obtain the plaintext (work order number + timestamp + random number), calculates the difference between the decrypted timestamp and the main control module's current system time (synchronized via NTP), and if the absolute value... The key verification is valid after 3 seconds; otherwise, the key expires and becomes invalid. In parallel with NFC data reception and decryption verification, the main control module sends an activation command to the millimeter-wave radar module via UART as soon as the NFC reader begins receiving data. The millimeter-wave radar module activates its multi-antenna system to transmit a 60GHz FMCW signal to the area in front of the door and receives the echo. After intermediate frequency sampling, 2D-FFT, CFAR target detection, and DBSCAN clustering, it outputs the number of valid targets—that is, the number of people in front of the door (single / multiple people)—as well as the distance, speed, and angle information for each target. The radar sensing process and the key verification process are executed in parallel, with a total time of approximately 80-100ms.

[0054] Step S3: If the key verification is valid and it is a single person, the door lock will be opened; if the key verification is valid and it is multiple people, only the operator who initiated the request will be subject to liveness detection. If the liveness detection is successful, the door will be opened and all people entering will be captured and recorded without comparison. If the liveness detection is unsuccessful, the door will be locked and an alarm will be triggered.

[0055] In one embodiment of the present invention, after the main control module acquires the key verification status (valid / invalid) and the number of personnel (single / multiple) of the millimeter-wave radar in parallel, it inputs both into the decision engine and outputs control commands according to a preset strategy: Decision Result 1 (Key Valid + Single Person): The main control module directly outputs a high-level door-opening pulse (3-second pulse width) to the electromagnetic lock drive circuit via GPIO, de-energizing the electromagnetic lock and unlocking the door. Simultaneously, the blockchain evidence storage subsystem is triggered to record the door-opening event.

[0056] Decision Result Two (Valid Key + Multiple Users): The main control module does not directly open the door, but instead sends a data acquisition command to the image acquisition module via the I2C bus. The image acquisition module activates the visible light camera and the infrared depth camera to simultaneously capture multimodal image data of the area in front of the door. The main control module loads a pre-trained face detection model (MTCNN) to detect all faces appearing in the visible light image, retrieves the locally registered face database based on the operator ID, and locates the face region of the authorized operator who initiated the door opening request (through facial feature vector comparison, similarity). 0.85 is considered a target face. The main control module only performs triple liveness detection on the authorized operator, sequentially performing action command verification, infrared 3D structured light verification, and texture analysis. Simultaneously, the main control module obtains the current light intensity (lux value) through the built-in ambient light sensor and estimates the facial occlusion ratio by comparing the area of ​​the face detection box with the reference area of ​​the complete face—if the light intensity is low... 100 lux, lower the liveness detection threshold from 80 to 75 and activate infrared auxiliary illumination; if the occlusion ratio If 30% (e.g., wearing a mask) is detected, the threshold will be lowered to 70 points and the system will switch to the periorbital area recognition mode. After dynamic threshold adjustment, the overall liveness detection score will be considered. If the threshold is reached and the liveness detection is successful, the main control module generates a door lock unlocking pulse and simultaneously sends a "continuous capture" command (3 frames / second) to the image acquisition module. (10 seconds) The image acquisition module captures a sequence of facial images of all personnel entering the premises (including the person in charge and accompanying personnel); the main control module associates the captured image sequence with the current work order number, operator ID, timestamp, and the number of people detected by the radar, packages it, generates a JSON-formatted "Guest Visit Log," and stores it locally—this log contains all the original images but does not perform any identity verification. If the overall liveness detection score is... If the threshold for determining liveness detection is reached and the detection fails, the main control module will not generate an opening signal but will instead generate an alarm data packet (including the reason for the verification failure and a multi-modal image screenshot of the moment of failure). This packet is uploaded in real time to the cloud-based operations and maintenance platform via the 4G / 5G communication module, notifying security personnel for review. Simultaneously, the main control module writes this failure event to the blockchain evidence storage subsystem.

[0057] Decision Result 3 (Invalid Key): The main control module refuses to open the door, generates an authentication failure log and triggers evidence storage, but does not trigger an alarm.

[0058] Step S4: In response to door opening, door locking, or alarm events, calculate the current event hash and link it with the hash value of the previous block to generate a new block, write it to the local blockchain, and synchronize it to the cloud.

[0059] In one embodiment of the present invention, any key event generated in step S3 above (successful door opening, successful anti-tailgating verification, failed liveness detection, or failed authentication) triggers the blockchain evidence storage process. The main control module packages the event type (encoded as an event type identifier, such as 0x01 for successful door opening, 0x02 for successful anti-tailgating verification, and 0x03 for failed liveness detection), event description, operator ID, work order number, number of personnel on the millimeter-wave radar, door sensor status, and event timestamp (synchronized via NTP) into raw event data in JSON format. The main control module reads the hash value of the latest block (previous block) of the local edge blockchain node, concatenates the hash value of the previous block, the raw event data (converted to a string), and the current trusted timestamp into a string in the format "prev_hash||event_data||timestamp", and calls the SHA256 hash algorithm to calculate the digest of this string to obtain the current event hash value. Constructing a new block—the block header contains the current event hash, the previous block hash, a timestamp, and a version number; the block body contains the original event data. After local verification, the new block is appended to the ledger file of the local edge blockchain node. The local node caches new blocks using a Merkle tree: every 10 new blocks accumulated or a 5-minute time window is reached, the hash values ​​of the 10 blocks are used as leaf nodes to construct a Merkle tree, and the root hash is calculated. The root hash is uploaded to the cloud consortium blockchain node (composed of the head office and branch nodes) via the MQTT protocol (encrypted with AES-128). After PBFT consensus, the cloud node writes the root hash to the consortium blockchain's public ledger and returns on-chain confirmation to the local node. This entire process realizes the complete lifecycle of a block from local generation to cloud confirmation, ensuring the immutability of any operational event and cross-node verifiability, meeting financial-grade regulatory audit requirements.

[0060] In summary, the business-aware multimodal fusion bank smart cabinet access control system and method of this invention, through deep integration of the cloud key management subsystem and the ITSM system, achieves dynamic key generation and composite condition verification driven by "work order status," strictly binding the permission lifecycle to the work order lifecycle, effectively solving the problems of static permissions and delayed eviction in traditional solutions. Through the parallel perception mechanism of millimeter-wave radar and key verification in the local execution subsystem, and the asymmetric anti-tailgating rule of "responsible person locking + object following record," the access control system possesses environmental context perception capabilities and differentiated anti-tailgating control capabilities, balancing security and operational efficiency in multi-person collaborative operation scenarios. Through multimodal liveness detection and dynamic threshold adjustment strategies, the system ensures authentication accuracy under complex lighting and occlusion environments. Through a lightweight blockchain evidence storage architecture, an immutable and trustworthy audit system meeting financial-grade regulatory requirements is constructed, providing a complete and legally valid chain of evidence for security incident investigations. This invention achieves a leapfrog upgrade of cabinet access control from static, passive, and isolated to dynamic, proactive, and collaborative, significantly improving the intelligent level of bank physical security protection.

[0061] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0062] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A business-aware multimodal fusion intelligent bank cabinet access control system, characterized in that, include: The cloud-based key management subsystem is used to receive door opening requests from mobile terminals, perform composite verification of work order status, geographical location, and time window, and generate a dynamic key after successful verification and return it to the mobile terminal. The local execution subsystem includes a main control module and a near-field communication module, a millimeter-wave radar module, and an image acquisition module, which are respectively connected to the main control module; The near-field communication module is used to receive the dynamic key forwarded by the mobile terminal and input it into the main control module; The millimeter-wave radar module is used to respond to the activation command sent in parallel by the main control module when it receives the dynamic key, collect data on the number of people in front of the gate, and feed it back to the main control module. The main control module is used to decrypt and verify the dynamic key, and simultaneously receive personnel count data; when the key is valid and it is a single person, it generates an opening signal; when the key is valid and it is multiple people, it triggers the image acquisition module to acquire images in front of the door, and only performs liveness detection on the authorized operator who initiated the opening request; if the detection is successful, it generates an opening signal and controls the image acquisition module to capture facial images of all entering personnel and generate a log that is only recorded and not compared; if the detection fails, it generates a locking signal and alarm data. The blockchain evidence storage subsystem, connected to the main control module, is used to respond to event evidence storage instructions, obtain the current event data and the hash value of the previous block, calculate the current event hash and link it with the previous hash value to generate a new block, write it to the local node and synchronize it to the cloud node.

2. The system according to claim 1, characterized in that, When the cloud-based key management subsystem performs composite verification, it specifically executes the following sub-steps: Obtain the work order identifier carried in the door opening request, and use the work order identifier as an index to query the corresponding work order status data from the ITSM system, which is used as the first input data; The first input data is validated: when the work order status data indicates that it has been approved but not executed, a work order verification pass status is generated as the first intermediate status data; otherwise, a rejection instruction is generated and the process is terminated. In response to the work order verification pass status, a status update instruction is sent to the ITSM system to atomically switch the work order status data from approved to executing, and generate status lock confirmation data as the second intermediate status data. The status lock confirmation data is used to prevent the same work order from being reused.

3. The system according to claim 1, characterized in that, When the cloud-based key management subsystem generates a dynamic key, it specifically performs the following sub-steps: Obtain the verified work order identifier, the current system timestamp, and the 16-byte random number output by the random number generation module, and use them as the plaintext data to be encrypted; The plaintext data is concatenated according to a preset format to form the target plaintext string; The national standard SM4 symmetric encryption algorithm is invoked to perform encryption operations on the target plaintext string using the pre-stored master key, generating a ciphertext data packet as a dynamic key. The effective time window for dynamic keys is 3 seconds. The current system timestamp is rounded to the granularity of the effective time window to ensure that dynamic keys generated within the same time window remain consistent.

4. The system according to claim 1, characterized in that, When the millimeter-wave radar module collects data on the number of people in front of the gate, it specifically performs the following sub-steps: In response to the activation command sent by the main control module, the millimeter-wave radar module transmits millimeter-wave signals to the area in front of the door and receives the target reflected echo signal as the raw sensing input data; The original sensing input data is processed by intermediate frequency signal decoding to extract the distance and velocity features of each target point and generate a point cloud dataset. Clustering and filtering are performed on the point cloud dataset to count the number of valid targets, generate data on the number of people in the area in front of the gate and their dynamic characteristics, and then feed this data back to the main control module.

5. The system according to claim 1, characterized in that, When the main control module performs a liveness detection on an authorized operator, it specifically executes the following sub-steps: The system receives multimodal image data of the area in front of the door acquired by the image acquisition module as input data. The multimodal image data includes visible light images and infrared depth images. Extract the face region image of the authorized operator from the visible light image to generate the first detection data; The action instructions are executed sequentially on the first data to be detected to verify whether the authorized operator has completed the specified action according to the randomly generated instructions, and to generate the first liveness confidence score. Simultaneously, three-dimensional facial structure reconstruction is performed on the infrared depth image to generate facial depth feature data, which is then compared with a pre-stored real face depth template to generate a second liveness confidence score. The first and second liveness confidence scores are weighted and fused to generate a comprehensive liveness detection score as the detection result data.

6. The system according to claim 1, characterized in that, When the main control module performs a liveness detection on an authorized operator, it also performs the following sub-steps: The current ambient light intensity data is obtained as the first environmental parameter, and the facial occlusion ratio data of the authorized operator in the visible light image is obtained as the second environmental parameter. When the first environmental parameter is lower than the preset illumination threshold, the threshold for determining the comprehensive liveness detection score is lowered, and infrared auxiliary illumination is activated. When the second environmental parameter indicates that the face is occluded, switch to the local area recognition mode around the eyes and adjust the judgment threshold of liveness detection to the preset occlusion mode threshold. Output the final liveness determination result after dynamic adjustment of environmental parameters.

7. The system according to claim 1, characterized in that, When the main control module generates a log that is only recorded and not compared, it specifically executes the following sub-steps: In response to the successful liveness detection, the main control module generates a continuous capture command and outputs it to the image acquisition module. The image acquisition module responds to the continuous capture command and performs serialized capture of the facial images of all people entering the area in front of the door, generating original facial image sequence data. The main control module acquires the original facial image sequence data, and obtains the work order identifier and operator identifier corresponding to the current door opening event. It then associates and packages the above data to generate a guest visit log data that is only recorded and not compared. The visitor log data does not trigger any identity verification operation and is stored locally and in the cloud only as evidence for tracing.

8. The system according to claim 1, characterized in that, When the blockchain evidence storage subsystem writes a new block to the local node and synchronizes it to the cloud node, it specifically performs the following sub-steps: The current event data, the current trusted timestamp, and the hash value of the previous block are obtained as input data. The current event data includes the event type identifier and a snapshot of the sensor status at the time the event occurred. The SHA256 hash algorithm is invoked to calculate the hash digest of the concatenated string of the previous block hash value, the current event data, and the current trusted timestamp, generating the current event hash value. Link the current event hash value with the previous block hash value to generate a new target block data containing a chain structure, and write it to the storage area of ​​the local edge blockchain node; The local edge blockchain node packages multiple new block data into Merkle tree nodes in chronological order, and uploads them in batches to the cloud consortium blockchain node through a preset data synchronization channel to complete cross-node redundant storage.

9. The system according to claim 1, characterized in that, The local execution subsystem also includes a door magnetic status sensor and a door frame deformation sensor connected to the main control module; The door magnetic status sensor is used to collect the opening and closing status data of the cabinet door and the locking status data of the electromagnetic lock in real time, and feed them back to the main control module as the first status input data. The door frame deformation sensor is used to collect deformation displacement data of the cabinet door frame in real time and feed it back to the main control module as second state input data; The main control module is also used to: determine a physical attack event and generate emergency alarm data when the first state input data indicates that the electromagnetic lock is abnormally opened in an unauthorized state, or when the second state input data exceeds a preset deformation threshold. In response to the generation of emergency alarm data, the main control module executes the following sub-steps in parallel: triggering the image acquisition module to record on-site video data, driving the audible and visual alarm actuator to issue an audible and visual alarm, and uploading the emergency alarm data and on-site video data to the cloud platform.

10. A business-aware multimodal fusion method for access control of smart bank teller machines, characterized in that, include: Receive the door opening request from the mobile terminal, perform a composite verification of the work order status, geographical location and time window, and generate a dynamic key after the verification is successful and return it to the mobile terminal; It receives and verifies the dynamic key forwarded by the mobile terminal, and simultaneously senses the number of people in front of the door through millimeter-wave radar; The door lock is opened when the key verification is valid and it is a single person; when the key verification is valid and it is multiple people, only the operator who initiated the request is subject to liveness detection. If the liveness detection is successful, the door is opened and all people entering are captured and recorded without comparison. If the liveness detection is unsuccessful, the door is locked and an alarm is triggered. In response to door opening, door locking, or alarm events, calculate the current event hash and link it with the hash value of the previous block to generate a new block, write it to the local blockchain, and synchronize it to the cloud.

Images