Multi-modal based non-intrusive triggering of nuclear island four-way access control

Abstract

The application discloses a multi-modal based non-sensing trigger nuclear island four-way access control, and relates to the technical field of nuclear island security and intelligent access control. The method comprises the following steps: perceiving the crowd density through a top wide-angle camera, combining two side auxiliary cameras to perform preliminary analysis on the hand-held object features, generating a panoramic cooperative wake-up instruction to activate a four-way camera array; after scheduling the array to collect multi-view images and depth data, performing accurate object state recognition and generating an adaptive recognition strategy; finally, fusing multi-modal data to perform three-dimensional living body detection and face feature comparison, and outputting an access control signal according to the identity verification result. Through the above method, non-contact and high-precision personnel access control is realized, the system power consumption is reduced, and the safety and intelligent level of the nuclear island access control are effectively improved.

Description

Technical Field

[0001] This application relates to the field of nuclear island security and intelligent access control technology, and in particular to nuclear island four-way access control based on multimodal contactless triggering. Background Technology

[0002] Traditional nuclear island access control systems mostly rely on single-modal identity verification methods, such as card swiping or fingerprint recognition. These methods pose security risks such as proxy swiping and fingerprint duplication. Furthermore, the recognition efficiency and pass rate decrease significantly when people are carrying items with both hands or when there is a large flow of people. It is difficult to achieve an efficient and seamless passage experience while ensuring strict security control. Summary of the Invention

[0003] The main purpose of this application is to provide a nuclear island four-way access control system based on multimodal contactless triggering, which can solve the technical problems of low recognition accuracy and insufficient security of nuclear island access control in the prior art.

[0004] To achieve the above objectives, this application provides a multimodal, contactless triggering method for four-way access control in a nuclear island, the method comprising: The system uses the wide-angle image captured by the top camera to perceive the density of people and obtain the status information of people in the area. It also uses the image data captured by the auxiliary cameras on both sides to analyze the hand area features and obtain preliminary information on the holding status. Based on the regional personnel status information and the preliminary object holding status information, a trigger condition is determined to obtain a panoramic collaborative wake-up command; According to the panoramic collaborative wake-up command, the four-way camera array is coordinated and parameters are scheduled to collect multi-view image data and depth data. Accurately identify the holding features of the person's hand in the multi-view image data to obtain the accurate holding state, and generate an adaptive recognition strategy based on the accurate holding state; Based on the adaptive recognition strategy, three-dimensional liveness detection and authorization feature comparison are performed on the multi-view images and face data to obtain the identity verification result, and an access control signal is output based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0005] In one embodiment, the step of performing hand region feature analysis on the image data collected by the auxiliary cameras on both sides to obtain preliminary information on the holding state includes: The auxiliary cameras on both sides are controlled to intermittently acquire images according to a preset sleep and wake-up cycle to obtain a sequence of images of a local area; Feature extraction of the hand contour and associated object contours is performed on the local region image sequence to obtain hand region contour data; Extract morphological and contour features from the hand region contour data, and generate preliminary holding status information based on the morphological and contour features.

[0006] In one embodiment, the step of extracting features of the hand contour and associated object contours from the local region image sequence to obtain hand region contour data includes: The local region image sequence is subjected to grayscale conversion and noise suppression processing to obtain preprocessed image data; Edge gradient feature calculation is performed on the preprocessed image data to obtain image edge intensity and direction information; Based on the edge strength and direction information, contour continuity analysis is performed to separate the hand contour and the object contour connected to the hand. The hand contour and the object contour are combined to obtain the hand region contour data.

[0007] In one embodiment, the step of determining the trigger condition based on the area personnel status information and the preliminary object holding status information to obtain the panoramic collaborative wake-up command includes: The population status information of the area is analyzed to obtain the population density of the area; The initial holding state information is analyzed to obtain the holding characteristics; When the pedestrian density in the area is greater than the preset pedestrian density or the object holding feature is a valid object holding feature, the panoramic collaborative wake-up command is generated to start the four-way camera array to operate at full power.

[0008] In one embodiment, the step of parsing the population status information of the area to obtain the population density of the area includes: Cluster analysis is performed on the personnel location coordinates contained in the personnel status information of the area to obtain several independent personnel target points; The number of personnel target points within the preset identification area is counted, and the number of personnel per unit area in real time is calculated by combining the area parameter of the area. The real-time number of people per unit area is smoothed and filtered with historical data within a preset time threshold to obtain the regional population density.

[0009] In one embodiment, the step of performing three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data based on the adaptive recognition strategy to obtain the identity verification result includes: Based on the adaptive recognition strategy, the multi-view image data and depth data are fused to construct a three-dimensional face model data; Liveness feature analysis is performed on the aforementioned 3D face model data to obtain liveness detection results; When the liveness detection result is a preset detection result, two-dimensional facial features are extracted from the multi-view image data and the matching degree is calculated with the locally pre-stored authorized personnel feature library to obtain the matching confidence. An authentication result is generated based on the matching confidence level.

[0010] In one embodiment, the step of performing liveness feature analysis on the 3D face model data to obtain liveness detection results includes: Surface curvature change analysis was performed on the three-dimensional facial model data to obtain the three-dimensional deformation features caused by micro-expression movements; Temporal analysis was performed on the multi-view images to extract iris micro-motion features of the iris texture; The three-dimensional deformation features and the iris micro-motion features are combined and compared with a preset live biometric model to obtain the comparison results. The liveness detection result is generated based on the comparison results.

[0011] In one embodiment, the step of performing collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command to acquire multi-view image data and depth data includes: According to the panoramic collaborative wake-up command, the top wide-angle camera, the two auxiliary cameras on both sides and the front depth camera are activated simultaneously. Based on the current location information of the person, the focal length and shooting angle parameters of each camera in the four-way camera array are dynamically adjusted to form a three-dimensional acquisition field of view covering the person's face. The controlled and adjusted four-way camera array synchronously acquires visible light image data and depth information data containing facial information of people, thus obtaining multi-view image data and depth data.

[0012] In one embodiment, the step of accurately identifying the holding features of a person's hand image data in the multi-view image data to obtain an accurate holding state, and generating an adaptive recognition strategy based on the accurate holding state, includes: The image data collected from the auxiliary cameras on both sides is extracted from the multi-view image data to accurately calibrate the hand feature points and the bounding box of the handheld object, so as to obtain the accurate holding state. When the precise holding state is holding the object with both hands, the preset standard face recognition time window is extended to a first predetermined duration, and the exposure compensation parameters are updated simultaneously. An adaptive recognition strategy is generated based on the first predetermined duration and the exposure compensation parameters.

[0013] Furthermore, to achieve the above objectives, this application also proposes a multimodal, contactless triggering four-way access control device for a nuclear island, comprising: The information perception module is used to perceive the density of people in the full-area image captured by the top wide-angle camera to obtain the status information of people in the area, and to perform hand area feature analysis on the image data captured by the auxiliary cameras on both sides to obtain preliminary information on the holding status. The trigger determination module is used to determine the trigger conditions based on the personnel status information and the preliminary object holding status information in the area, and to obtain the panoramic collaborative wake-up command. The collaborative scheduling module is used to perform collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command, and to collect multi-view image data and depth data. The object holding recognition module is used to accurately identify the object holding features of the person's hand image data in the multi-view image data, obtain the accurate object holding state, and generate an adaptive recognition strategy based on the accurate object holding state. The identity verification module is used to perform three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data based on the adaptive recognition strategy, obtain the identity verification result, and output the access control signal based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0014] Furthermore, to achieve the above objectives, this application also proposes a nuclear island four-way access control device based on multimodal contactless triggering. The nuclear island four-way access control device based on multimodal contactless triggering includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the nuclear island four-way access control method based on multimodal contactless triggering described above.

[0015] In addition, to achieve the above objectives, the present invention also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described multimodal sensorless triggering four-way access control method for the nuclear island.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multimodal-based sensorless triggering four-way access control method for the nuclear island described above.

[0017] This application provides a multimodal, contactless triggering method for four-way access control in a nuclear island. It uses a top wide-angle camera to sense pedestrian density, combined with auxiliary cameras on both sides for preliminary analysis of hand-held object features, generating a panoramic collaborative wake-up command to activate the four-way camera array. After the array is scheduled to collect multi-view images and depth data, precise object-held status recognition is performed, and an adaptive recognition strategy is generated. Finally, multimodal data is fused for 3D liveness detection and facial feature comparison, outputting access control signals based on the identity verification results. This method achieves contactless, high-precision personnel access control, effectively improving the security and intelligence level of the nuclear island access control system while reducing system power consumption. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the nuclear island four-way access control method based on multimodal sensorless triggering in this application; Figure 2 This is a schematic diagram of a four-way access control system according to an embodiment of the non-sensory triggering nuclear island four-way access control method based on multimodal operation in this application. Figure 3 This is a schematic diagram of the module structure of the nuclear island four-way access control device based on multimodal contactless triggering, according to an embodiment of this application. Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the multimodal, contactless triggering four-way access control method for the nuclear island in this application embodiment.

[0021] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is: to perceive the crowd density of the full-area image captured by the top wide-angle camera to obtain the status information of people in the area, and to analyze the hand area features of the image data captured by the auxiliary cameras on both sides to obtain preliminary information on the holding status. Based on the regional personnel status information and the preliminary object holding status information, a trigger condition is determined to obtain a panoramic collaborative wake-up command; According to the panoramic collaborative wake-up command, the four-way camera array is coordinated and parameters are scheduled to collect multi-view image data and depth data. Accurately identify the holding features of the person's hand in the multi-view image data to obtain the accurate holding state, and generate an adaptive recognition strategy based on the accurate holding state; Based on the adaptive recognition strategy, three-dimensional liveness detection and authorization feature comparison are performed on the multi-view images and face data to obtain the identity verification result, and an access control signal is output based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0025] Traditional nuclear island access control systems mostly rely on single-modal identity verification methods, such as card swiping or fingerprint recognition. These methods pose security risks such as proxy swiping and fingerprint duplication. Furthermore, the recognition efficiency and pass rate decrease significantly when people are carrying items with both hands or when there is a large flow of people. It is difficult to achieve an efficient and seamless passage experience while ensuring strict security control.

[0026] This application provides a solution that uses a top wide-angle camera to sense pedestrian density, combines this with auxiliary cameras on both sides to perform preliminary analysis of hand-held object features, and generates a panoramic collaborative wake-up command to activate the four-way camera array. After the array is scheduled to collect multi-view images and depth data, precise object-held status recognition is performed, and an adaptive recognition strategy is generated. Finally, multimodal data is fused for 3D liveness detection and facial feature comparison, and access control signals are output based on the identity verification results. Through this method, contactless, high-precision personnel access control is achieved, effectively improving the security and intelligence level of the nuclear island access control system while reducing system power consumption.

[0027] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a nuclear island four-way access control device based on multimodal contactless triggering, etc. This embodiment does not specifically limit it. The following uses a nuclear island four-way access control device based on multimodal contactless triggering as an example to describe this embodiment and the following embodiments.

[0028] This application provides a multimodal, contactless triggering method for four-way access control in a nuclear island, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the nuclear island four-way access control method based on multimodal sensorless triggering in this application.

[0029] In this embodiment, the nuclear island four-way access control method based on multimodal sensorless triggering includes steps S10~S50: Step S10: Perform crowd density perception on the full-area image captured by the top wide-angle camera to obtain the status information of people in the area. Perform hand area feature analysis on the image data captured by the auxiliary cameras on both sides to obtain preliminary information on the holding status.

[0030] It should be noted that the overall image refers to the scene captured by a wide-angle camera deployed at the top of the access control area, covering the entire access control passage and the nearby waiting area. The personnel status information is structured data obtained after analyzing this image, mainly including the number of people in the area, their real-time location coordinates, and movement trends. The auxiliary cameras on both sides refer to cameras symmetrically installed on the sides of the access control passage, focusing on capturing local features of the upper body and hands of personnel. The preliminary object-holding status information is a preliminary, imprecise judgment made after quickly analyzing the images from the side cameras regarding whether a person is holding an object.

[0031] Understandably, referring to Figure 2 The top wide-angle camera 1 operates continuously at a low frame rate. First, it scales and corrects distortion in the RGB images it captures. Then, it uses a lightweight object detection network (such as the YOLO series) or background subtraction algorithm to quickly locate all people in the image. It then analyzes the center points of the bounding boxes of the detected people using a clustering algorithm (such as DBSCAN) to distinguish individual people and count the number of people in real time. Combined with the preset area, it calculates the dynamic crowd density. At the same time, the auxiliary cameras 3 on both sides are set to wake up intermittently at a specific period. When a person is detected entering their field of view, a brief image acquisition is triggered. Subsequently, the system applies preprocessing based on grayscale and Gaussian filtering to the acquired local images. Then, it uses an efficient edge detection operator (such as Canny) or a trained hand keypoint detection model to extract the approximate outline of the hand and the edge features of the objects connected to it. By analyzing the geometric shape of the outline (such as whether there are regular rectangular box features suggesting the holding of a toolbox, or whether a non-skin-colored continuous outline extending from the arm is detected), it quickly generates preliminary status labels such as "empty hands", "possibly holding an object", or "holding an object with both hands".

[0032] In one feasible implementation, the step of performing hand region feature analysis on the image data collected by the auxiliary cameras on both sides to obtain preliminary information on the holding state includes: The auxiliary cameras on both sides are controlled to intermittently acquire images according to a preset sleep and wake-up cycle to obtain a sequence of images of a local area; Feature extraction of the hand contour and associated object contours is performed on the local region image sequence to obtain hand region contour data; Extract morphological and contour features from the hand region contour data, and generate preliminary holding status information based on the morphological and contour features.

[0033] It should be noted that the preset sleep and wake-up cycle refers to a fixed time interval pre-set by the system to control the camera to alternate between a low-power sleep state and an active working state in order to balance power consumption and real-time performance. The local area image sequence consists of multiple frames of images continuously acquired by the two auxiliary cameras during the wake-up period, forming an observation segment in the time dimension. The hand region contour data is an ordered set of pixel-level boundary information describing the hand and objects in direct contact or very close proximity to it, extracted from the image by an algorithm. Morphological features refer to the geometric properties of the region enclosed by the contour, such as area, perimeter, and minimum bounding rectangle. Contour features refer to the shape characteristics of the contour itself, such as edge curvature, convex hull defects, and Hu moments.

[0034] In its implementation, the system first controls the auxiliary cameras on both sides to switch between sleep and wake-up states according to a predefined period T (in seconds). Wake-up window duration. Inside, the camera operates at a frame rate Images are acquired (frames per second) to form a dataset containing... Local region image sequence of frame image This intermittent acquisition strategy can significantly reduce the power consumption of the device during long-term operation.

[0035] For extracting the contour data of the hand region, a deep learning-based instance segmentation model (such as Mask R-CNN) can be used. For each frame of the image sequence... The model output includes a binary mask of the hand region. Contour extraction algorithms (such as the Suzuki85 algorithm) are then applied to... The pixel-level contour point set of the hand region is obtained. , where each point This represents the coordinates of a single contour pixel. If multiple hand regions are detected, multiple contour point sets are generated.

[0036] Morphological feature extraction first involves calculating the contour. The area enclosed and perimeter .area The perimeter can be calculated using the number of pixels in the contour point set or Green's formula. This is the sum of the Euclidean distances between adjacent points in the contour point set. Contour compactness is a commonly used morphological feature, and its calculation formula is:

[0037] in, It is a dimensionless compactness value. When holding a regular object (such as a toolbox), The value is usually smaller than that of an empty hand, reflecting a more regular shape.

[0038] Contour feature extraction includes calculating the Hu moment-invariant features of the contour. First, the contour is calculated... of Order geometric moments :

[0039] in, These are the coordinates of the contour points. and It is a non-negative integer. Then calculate the central moments. To eliminate the effects of translation, and further calculate the normalized central moments. To eliminate the effects of scale, seven Hu moment eigenvectors, invariant to translation, rotation, and scale changes, were finally calculated based on the second and third normalized central moments. These features can effectively describe the global shape of the contour.

[0040] The initial information about the held object's state is generated based on the aforementioned features. The system pre-trains a lightweight classifier (such as a Support Vector Machine, SVM) using samples. For each contour data point, the corresponding feature vector (such as one containing...) is... and The classifier takes a subset of the input as input. The classifier outputs a category label, corresponding to the initial holding state such as "empty hands", "holding a small item", or "holding a large / regular item".

[0041] In one feasible implementation, the step of extracting features of the hand contour and the contour of associated objects from the local region image sequence to obtain hand region contour data includes: The local region image sequence is subjected to grayscale conversion and noise suppression processing to obtain preprocessed image data; Edge gradient feature calculation is performed on the preprocessed image data to obtain image edge intensity and direction information; Based on the edge strength and direction information, contour continuity analysis is performed to separate the hand contour and the object contour connected to the hand. The hand contour and the object contour are combined to obtain the hand region contour data.

[0042] It should be noted that grayscale conversion and noise suppression processing involves converting a color image into a single-channel grayscale image containing only brightness information, and then using filtering algorithms to reduce the interference of randomly fluctuating pixels in the image. Edge gradient features describe the degree and direction of brightness value changes at each pixel in the image. Edge strength and direction information are numerical representations of the above gradient features; strength represents the salience of the edge, and direction is perpendicular to the edge's orientation. Contour continuity analysis refers to the process of connecting edge pixels into a continuous and complete boundary curve based on the coherence of their position and direction.

[0043] In the specific implementation, firstly, each frame of the color image in the acquired local area image sequence is preprocessed. Grayscale conversion typically uses a weighted average method, combining the values ​​of the three RGB channels according to human eye sensitivity. The calculation formula is as follows:

[0044] in These represent the pixel values ​​for the red, green, and blue channels, respectively. This is the obtained grayscale value. Subsequently, Gaussian filtering is used for noise suppression, and the convolution kernel function is:

[0045] in, These are relative coordinates within the kernel. It represents the standard deviation, which controls the smoothness. This process effectively reduces false edges in subsequent edge detection.

[0046] Next, edge gradients are calculated on the preprocessed grayscale image. The most commonly used method is the Sobel operator, which uses two 3x3 convolution kernels ( For use in the horizontal direction, Convolving the image (for the vertical direction) with the image yields the horizontal gradient components. and vertical gradient components Edge intensity of each pixel and direction Calculated using the following formula:

[0047] in, It reflects the magnitude of the brightness change. It indicates the direction perpendicular to the edge line.

[0048] Then, contour continuity analysis is performed based on edge strength and direction information. Typically, non-maximum suppression is performed first, retaining only pixels with the highest local intensity along the gradient direction to refine the edges. Subsequently, a dual-threshold algorithm is used for hysteresis thresholding: a high threshold is set... and low threshold Strength greater than Pixels below are judged as strong edges. Strong edges are suppressed, while weak edges lie between the two. Finally, using an edge tracking algorithm, strong edge pixels are used as starting points, and connected weak edge pixels are found in their neighborhoods to form a complete and continuous contour line. This process can separate and extract the hand contour from the object contour that is continuous in position and direction due to being held.

[0049] Finally, the pixel coordinates of the hand contour line obtained from the above analysis, as well as the object contour lines extending from and connected to the hand contour line, are combined to form the hand region contour data.

[0050] Step S20: Based on the regional personnel status information and the preliminary object holding status information, a trigger condition judgment is made to obtain a panoramic collaborative wake-up command.

[0051] It should be noted that the panoramic collaborative wake-up command is a control signal generated by the system when specific triggering conditions are met. This command is used to simultaneously activate or wake up multiple high-resolution cameras or other depth sensing sensors deployed in the access control area, so that they can switch from a low-power standby state to a collaborative working mode, in order to collect and analyze target personnel from multiple angles and with greater precision.

[0052] Understandably, the system first fuses and spatiotemporally aligns the information on the status of people in the area sensed by the top camera with the preliminary information on the objects held by the auxiliary cameras on both sides. The trigger judgment logic is usually based on preset thresholds and rules. For example, the system determines that the trigger condition is met only when a single pedestrian is detected in the passage and that pedestrian enters a predefined trigger area, or when the auxiliary cameras on both sides report an object-holding status that is not "empty-handed" in their latest wake-up cycle. Subsequently, the central processing unit generates and broadcasts a panoramic collaborative wake-up command. This command contains the location coordinates or tracking ID of the target person. The command is sent to the controllers of all high-resolution cameras via an internal communication bus (such as CAN or Ethernet), causing them to immediately exit the sleep state, adjust the pan-tilt angle and aim at the target, and activate high-frame-rate recording or image capture functions to obtain high-quality, multi-view image data.

[0053] In one feasible implementation, the step of determining the trigger condition based on the area personnel status information and the preliminary object holding status information to obtain the panoramic collaborative wake-up command includes: The population status information of the area is analyzed to obtain the population density of the area; The initial holding state information is analyzed to obtain the holding characteristics; When the pedestrian density in the area is greater than the preset pedestrian density or the object holding feature is a valid object holding feature, the panoramic collaborative wake-up command is generated to start the four-way camera array to operate at full power.

[0054] It should be noted that regional pedestrian density refers to the number of people present simultaneously within a unit of monitored area, or the ratio of people to the area occupied by people, used to quantify the level of activity in that area. Object holding characteristics are a set of data extracted from preliminary object holding status information that characterizes the presence or absence of a handheld object and its key attributes. A four-way camera array refers to a camera array consisting of a top wide-angle camera 1, two auxiliary cameras on either side 3, and a front depth-sensing camera 2.

[0055] It should be understood that the four-way camera array includes one top wide-angle low-power camera, two auxiliary cameras on the left and right sides, and one front depth-sensing camera. All four cameras are integrated into a protective housing and support motorized adjustment of focus and shooting angle. Top wide-angle camera: Low-power CMOS module, constant low frame rate continuous acquisition, responsible for macroscopic perception of crowd density in the entire area, the only camera that works continuously under normal conditions; Side auxiliary cameras: lightweight image acquisition module, intermittent start and stop, only running a lightweight neural network for hand area feature detection, without performing full-screen face inference; Front-facing depth camera: Equipped with a depth ranging module, it only activates when triggered, outputting 2D facial images + 3D depth data for liveness detection.

[0056] In the specific implementation, firstly, the system analyzes the status information of people in the area to calculate the area's pedestrian density. From the personnel location data acquired by the top wide-angle camera, the system counts the number N of people located within the core monitoring area A (e.g., a specific rectangular area in front of the turnstile). This determines the area's pedestrian density. The calculation formula is:

[0057] in, It is the calculated crowd density (unit: people / square meter). It is a count of people in the area. This refers to the area of ​​the core monitoring zone (unit: square meters). Preset pedestrian density threshold. The settings are based on the actual traffic capacity and safety standards of the scenario.

[0058] Secondly, the preliminary holding state information is analyzed to extract holding features. This feature can be a multi-dimensional vector F, containing key indicators calculated from the contour data, such as the ratio of the contour area to the estimated hand area. The tightness of the contour (C), and the contour Similarity between the rectangle and the pre-stored item outline template Its expression can be summarized as:

[0059] in, Reflects the size of the object, Reflecting the regularity of shape, Used to match the shape of a specific item. The system predefines a set of valid item features. When the extracted features When the matching degree with the set exceeds the threshold, it is determined to be a valid holding feature.

[0060] The final trigger determination is a logical "OR" relationship. The system continuously compares real-time pedestrian density. With preset threshold At the same time, determine whether the holding feature F is a valid holding feature. As long as the conditions are met. (High density may increase security risks) or (If a suspicious object is detected), the logic decision unit will immediately generate a panoramic collaborative wake-up command. This command is sent to the controller of the four-way camera array, which controls it to switch from sleep mode to full-power operation mode, and initiates a high-resolution, multi-angle collaborative capture process.

[0061] In one feasible implementation, the step of parsing the population status information of the area to obtain the population density of the area includes: Cluster analysis is performed on the personnel location coordinates contained in the personnel status information of the area to obtain several independent personnel target points; The number of personnel target points within the preset identification area is counted, and the number of personnel per unit area in real time is calculated by combining the area parameter of the area. The real-time number of people per unit area is smoothed and filtered with historical data within a preset time threshold to obtain the regional population density.

[0062] It should be noted that the personnel target point refers to the center coordinate point corresponding to each individual person, which is determined after processing the original location coordinates through a clustering algorithm.

[0063] In the specific implementation, firstly, cluster analysis is performed on the personnel location coordinates to distinguish individual locations. Since the original location information may contain multiple detection points from the same person or noise, a density-based clustering algorithm (such as DBSCAN) is used for processing. This algorithm defines two core parameters: neighborhood radius. and minimum points For each data point, the algorithm will calculate its... The number of points in the neighborhood, if the number of points is not less than If a point is identified as a core point, it is then labeled as such, and the core point and its neighboring points are grouped into the same cluster using density-connected relationships. Ultimately, each identified cluster represents an independent human target, and its center position can be represented by the mean coordinates of all points within the cluster.

[0064] Secondly, the number of personnel target points within the preset identification area is counted, and the real-time number of personnel per unit area is calculated. Let the counted number of independent personnel target points be... The area of ​​the preset recognition region (such as a rectangular region) is Real-time number of people per unit area The calculation formula is:

[0065] in, This is an unsmoothed estimate of instantaneous pedestrian density (unit: people / square meter), where N is the statistically obtained number of people. It is the area of ​​the preset region (unit: square meters).

[0066] Finally, a smoothing filter is applied to the real-time number of people per unit area to obtain a more stable and trend-reflecting regional population density. An exponentially weighted moving average (EWMA) filter is used, and its calculation formula is as follows:

[0067] in, It is the smoothed output value of the regional population density at the current time t. It is the current moment. The calculated number of people per unit area in real time. It was the previous moment The smooth output value. It is a smoothing factor ( The smaller the value, the greater the dependence on historical data and the stronger the smoothing effect, but the slower the response to the latest changes. This processing can effectively reduce density value jumps caused by short-term personnel stays or instantaneous fluctuations in detection.

[0068] Step S30: According to the panoramic collaborative wake-up command, the four-way camera array is coordinated and parameters are scheduled to collect multi-view image data and depth data.

[0069] It should be noted that multi-view image data refers to a set of two-dimensional images containing the same target, simultaneously acquired from the perspectives of cameras located at different spatial positions. Depth data refers to a set of data obtained through a frontal depth camera, used to describe the distance information between each pixel in the scene and the camera.

[0070] Understandably, the instructions typically include the target's position or tracking ID in the global coordinate system. The scheduling algorithm uses this information, combined with the preset intrinsic and extrinsic parameters of each camera (such as focal length and orientation angle), to calculate in real-time the gimbal turning angle and zoom parameters that center the target in each camera's view or in the optimal observation area. These parameters are then sent to each camera controller in the array via a unified control bus, driving them to quickly align with the target. Secondly, to ensure strict temporal synchronization of multi-view images, the system uses hardware trigger signals or high-precision network time protocols to simultaneously send image acquisition commands to all cameras, controlling the exposure time deviation of each camera to within milliseconds. For depth data acquisition, if the array includes binocular cameras or depth cameras, the system synchronously starts their depth calculation mode and uniformly sets the parameters required for depth calculation (such as parallax range and filtering threshold). Finally, the acquired multiple sets of synchronized image data and corresponding depth maps or point cloud data are encapsulated into data packets with a unified timestamp and data source identifier, which constitute the multi-view image data and depth data.

[0071] In one feasible implementation, the step of performing collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command to acquire multi-view image data and depth data includes: According to the panoramic collaborative wake-up command, the top wide-angle camera, the two auxiliary cameras on both sides and the front depth camera are activated simultaneously. Based on the current location information of the person, the focal length and shooting angle parameters of each camera in the four-way camera array are dynamically adjusted to form a three-dimensional acquisition field of view covering the person's face. The controlled and adjusted four-way camera array synchronously acquires visible light image data and depth information data containing facial information of people, thus obtaining multi-view image data and depth data.

[0072] It should be noted that a top-mounted wide-angle camera refers to a camera installed at the top of the monitored area with a wide horizontal field of view, used to provide a global perspective and initial personnel location. Side auxiliary cameras refer to cameras symmetrically deployed on the left and right sides of the monitored area, used to capture targets from the side and compensate for blind spots in the frontal view. A frontal depth camera refers to a special camera deployed facing the main direction of movement of people, capable of simultaneously acquiring color images and depth information. Focal length parameters refer to the optical or digital zoom settings that control the camera lens group to change the range of image sharpness. Shooting angle parameters refer to the angle values ​​that control the tilt and horizontal rotation of the pan / tilt unit or the camera itself, used to change the shooting direction.

[0073] In the specific implementation, the first step is to perform hardware synchronization activation and initial parameter settings. After parsing the panoramic collaborative wake-up command, the system's main controller simultaneously sends activation commands to the top wide-angle camera, the left and right auxiliary cameras, and the front depth camera via precise hardware synchronization signals (such as the PTP protocol or trigger pulses). During initialization, each camera loads its default parameter set to ensure the system quickly enters a standby state. The top wide-angle camera maintains its maximum wide-angle setting to continuously provide global personnel position tracking. The front depth camera is ready to simultaneously output RGB color images and depth maps.

[0074] Secondly, based on the real-time acquired location information of the personnel, the focal length and angle of each camera are dynamically adjusted. Let the coordinates of the target person in the world coordinate system obtained from the top wide-angle camera be... For each camera in the array, the tilt angle that its gimbal needs to be adjusted. and horizontal yaw angle It can be calculated using the following spatial geometric relationships:

[0075]

[0076] in, These are the camera's own spatial coordinates. It is to set the target coordinates Transform to the coordinates in this camera's coordinate system. The focal length parameter (or digital zoom Z) is then determined based on the target distance. Make adjustments and follow The relationship between the facial features and the preset facial imaging size requirements are used for fine-tuning to ensure that the facial area occupies enough pixels in the image.

[0077] Finally, all cameras are controlled to acquire data synchronously. After all cameras have completed parameter adjustments and provided a ready signal, the main controller issues a global acquisition trigger signal. The frontal depth camera captures high-resolution frontal visible light images and precise depth information data; the side auxiliary cameras capture the contour texture of the face from approximately a 45-degree side angle; the top wide-angle camera may provide top-view reference. All data frames carry a unified timestamp, collectively forming multi-view image data and depth data.

[0078] Step S40: Accurately identify the holding features of the person's hand image data in the multi-view image data to obtain the accurate holding state, and generate an adaptive recognition strategy based on the accurate holding state.

[0079] It should be noted that personnel hand image data refers to image blocks or pixel sets containing the personnel hand region, specifically segmented or extracted from multi-view image data acquired by a four-way camera array using object detection technology. Precise object holding status is a structured result obtained after accurate identification of object holding features, typically including quantitative information such as the determination of whether an object is being held, the category label of the held object, and the object's posture relative to the hand.

[0080] Understandably, the system first utilizes a pre-trained multi-view object detection network (such as an improved model based on Faster R-CNN) to detect and associate hand regions in images from all viewpoints, ensuring that even with occlusion, complete hand bounding boxes can be obtained through view complementarity. Next, the cropped hand image patches from each viewpoint are input into a fine-grained classification network. This network typically employs a two-branch structure: one branch focuses on a binary classification task of identifying "holding an object / not holding an object," while the other branch performs fine-grained object classification on images already identified as holding an object. The network output includes the holding confidence score, object category probability distribution, and keypoint coordinates. Based on this accurate holding status, the system dynamically generates an adaptive recognition strategy: if holding an object is detected, the tracking priority of the person in the overall monitoring is immediately increased, and a dedicated recognition pipeline for that object category is activated, while computational resources are preferentially allocated to subsequent frame processing from relevant cameras. If not holding an object, the system may reduce the processing frame rate for that target to save computational power. This entire process ensures efficient allocation of recognition resources and focused monitoring of high-risk targets. Based on the object being held, a corresponding adaptive recognition strategy can be generated. When an object is being held, the access control recognition time can be appropriately extended to generate the corresponding adaptive recognition strategy.

[0081] In one feasible implementation, the step of accurately identifying the holding features of a person's hand image data in the multi-view image data to obtain an accurate holding state, and generating an adaptive recognition strategy based on the accurate holding state, includes: The image data collected from the auxiliary cameras on both sides is extracted from the multi-view image data to accurately calibrate the hand feature points and the bounding box of the handheld object, so as to obtain the accurate holding state. When the precise holding state is holding the object with both hands, the preset standard face recognition time window is extended to a first predetermined duration, and the exposure compensation parameters are updated simultaneously. An adaptive recognition strategy is generated based on the first predetermined duration and the exposure compensation parameters.

[0082] It should be noted that hand feature points refer to the two-dimensional or three-dimensional coordinate positions of human hand joints (such as finger joints and wrist points) located by a keypoint detection model. The bounding box of a handheld object refers to the smallest rectangular area in the image that can completely enclose an object that is being grasped or held by a person. Holding with both hands is a precise holding state, specifically referring to the situation where a person simultaneously uses both hands to grasp or support the same or different objects. The first predetermined duration refers to a new time window value, dynamically set by the system and longer than the standard time window, when a two-handed holding state is determined. Exposure compensation parameters refer to the adjustment values ​​made to the exposure of the camera sensor to improve image quality, usually expressed in EV values.

[0083] In the specific implementation, the first step is to accurately calibrate the hand feature points and the bounding box of the held object. The system prioritizes image data from the auxiliary cameras on both sides, as the side view angle usually shows the spatial relationship between the hand and the object more clearly. A pre-trained keypoint detection network (such as an improved version of HRNet or MediaPipe Hands) is used to detect 21 keypoints for each hand in the image, outputting a set of pixel coordinates for the keypoints. Meanwhile, an attention-based object detection network was used to generate candidate bounding boxes that might contain handheld objects.

[0084] in The coordinates of the center point of the bounding box. Width and height are respectively. This is the confidence score.

[0085] Precise determination of the object-holding state is achieved by analyzing the spatial relationship between feature points and the bounding box. A handheld state determination function is defined. If a bounding box It contains a sufficient number (e.g., more than 5) of key points from the same hand. If the location and distribution of these key points match the grip posture, then the bounding box is confirmed as the bounding box of the held object, and the precise holding state is updated. If confirmed bounding boxes of the held object are detected in both the left and right hand regions, the state is marked as "held by both hands".

[0086] Secondly, an adaptive recognition strategy is generated. When the precise object holding state is "held with both hands," the strategy generation module performs two main adjustments. First, the standard face recognition time window is adjusted. Extended to the first scheduled duration This adjustment is based on an empirical model, which assumes that holding an object with both hands may imply more complex scenarios or require a longer, stable observation period to ensure accurate recognition. The adjustment formula can be expressed as:

[0087] in, It is a standard recognition time window. It is the basic time constant. It is a weighted coefficient dynamically calculated based on scene complexity (such as object size and occlusion level). ).

[0088] Second, update the exposure compensation parameters simultaneously. Holding objects in both hands can alter a person's natural posture, causing changes in facial lighting conditions (e.g., obstruction or reflection by the held object). The system uses recently acquired average brightness data for the facial area. Compared to ideal brightness The difference is used to calculate the new exposure compensation value:

[0089] Should The values ​​are sent to the relevant cameras to optimize exposure settings for subsequent image acquisition, ensuring facial image quality. Ultimately, this is achieved through an extended time window. and the adjusted exposure compensation parameters Together, they constitute the adaptive recognition strategy for the current scenario.

[0090] Step S50: Based on the adaptive recognition strategy, perform three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data to obtain the identity verification result, and output the access control signal based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0091] It should be noted that 3D liveness detection refers to a technique that uses 3D spatial information obtained from multi-view images and depth data to analyze facial contours, micro-expressions, or specific physiological activities (such as blinking) to distinguish real live faces from forgeries such as photos, screen captures, or mockups. Authorization feature comparison refers to the process of calculating the similarity between the biometric features (such as 3D facial feature vectors) extracted after liveness detection confirms a real face and the features of legitimate users pre-stored in an authorization database. The authentication result is a binary judgment result generated after authorization feature comparison, namely "pass" or "fail," indicating whether the current person has been confirmed as an authorized user. Access control signals refer to the electronic commands issued by the system to physical access control mechanisms (such as electric locks or turnstiles) after the authentication result is generated, used to open or deny access.

[0092] Understandably, the system first integrates depth data from a frontal depth camera and texture images from multi-view cameras based on an adaptive recognition strategy. The 3D liveness detection module utilizes multi-view geometric consistency checks and depth map analysis: it calculates the 3D coordinates of key facial points (such as the corners of the eyes and the tip of the nose) from different viewpoints and verifies whether they conform to the non-planar rigid motion laws of a real face. Simultaneously, it analyzes the continuous curvature of areas such as the bridge of the nose and eye sockets in the depth map to avoid planar attacks. After successful liveness detection, highly robust feature descriptors are extracted from the reconstructed 3D face model and compared with corresponding feature templates in the authorized database. Cosine similarity or Euclidean distance is used as a metric; if the similarity exceeds a preset threshold, authentication is considered successful. Finally, the system generates corresponding access control signals based on the verification results and sends them to the access controller via I / O interface or network protocol to complete access authorization or denial.

[0093] In one feasible implementation, the step of performing three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data based on the adaptive recognition strategy to obtain the identity verification result includes: Based on the adaptive recognition strategy, the multi-view image data and depth data are fused to construct a three-dimensional face model data; Liveness feature analysis is performed on the aforementioned 3D face model data to obtain liveness detection results; When the liveness detection result is a preset detection result, two-dimensional facial features are extracted from the multi-view image data and the matching degree is calculated with the locally pre-stored authorized personnel feature library to obtain the matching confidence. An authentication result is generated based on the matching confidence level.

[0094] It should be noted that the 3D facial model data is a 3D point cloud or mesh model reconstructed by fusing multi-view images and depth data, capable of representing the geometric shape of the facial surface. The 2D facial features are feature vectors extracted from a grayscale or color facial image at an optimal viewpoint or fused viewpoint among multi-view images, used for identity comparison. The locally pre-stored authorized personnel feature library is a database stored on the access control system's local server or within the security chip, containing standard facial feature templates for authorized personnel.

[0095] In its implementation, the system first constructs a 3D model. Based on an adaptive recognition strategy (such as a specified time window and exposure parameters), the system simultaneously acquires multi-view RGB images. Depth map from depth camera The 3D reconstruction process employs a method based on the fusion of stereo vision and depth maps. For each viewpoint... Its camera parameters (intrinsic parameter matrix) and extrinsic parameter matrix (The following is a separate, unrelated sentence:) Known. By using the depth map... Back-projecting onto 3D space and performing texture mapping with the RGB image yields a preliminary 3D point cloud. Simultaneously, feature point matching between multi-view images is used to generate supplementary 3D points through triangulation. Finally, iterative nearest neighbor (ICP) or voxel fusion algorithms are used to optimize and fuse these two point clouds, outputting dense and textured 3D face model data. Its core formula is to obtain the point cloud through backprojection of the depth map:

[0096] in, These are the pixel coordinates of the depth map. It is the intrinsic parameter matrix of the depth camera. These are the corresponding three-dimensional point coordinates.

[0097] Secondly, liveness feature analysis is performed on the constructed 3D model. Calculate the curvature characteristics of its surface, especially the average curvature of areas such as the tip of the nose and eye sockets. and Gaussian curvature Live human faces exhibit specific curvature distribution patterns. Simultaneously, the model is analyzed for minute deformations over short time series (such as subtle periodic skin movements caused by heart rate). Liveness detection function. It can be represented as:

[0098] in, It is a scoring function based on curvature characteristics. It is a scoring function based on micro-motion features. and These are weighting coefficients. If... Greater than the threshold If the result is positive, the result of the liveness detection will be the preset result (i.e., liveness).

[0099] Finally, feature comparison and result generation are performed. Once the subject is confirmed to be alive, the image is generated from the highest quality viewing angle (usually a frontal or near-frontal view). In this process, a pre-trained face recognition network (such as ArcFace) is used to extract two-dimensional feature vectors of the face. Compare it with each template feature vector in the authorized personnel feature library. Cosine similarity is typically used to calculate the matching degree.

[0100] in, Is with the first Similarity score of each template. Match confidence. Take all The maximum value in. Authentication results are based on... Generation: If If the result is "passed", the result is "failed"; otherwise, the result is "failed".

[0101] In one feasible implementation, the step of performing liveness feature analysis on the three-dimensional face model data to obtain liveness detection results includes: Surface curvature change analysis was performed on the three-dimensional facial model data to obtain the three-dimensional deformation features caused by micro-expression movements; Temporal analysis was performed on the multi-view images to extract iris micro-motion features of the iris texture; The three-dimensional deformation features and the iris micro-motion features are combined and compared with a preset live biometric model to obtain the comparison results. The liveness detection result is generated based on the comparison results.

[0102] It should be noted that the three-dimensional deformation features caused by micro-expression movements refer to the minute displacement patterns of the skin surface in three-dimensional space caused by slight, involuntary contractions of facial muscles. The iris micro-motion features of the iris texture refer to the periodic or non-periodic displacement patterns of the fine texture of the iris region in high-resolution images over time; these micro-motions are a physiological characteristic of the living eye.

[0103] In the specific implementation, firstly, surface curvature change analysis is performed to extract three-dimensional deformation features. This is based on the reconstructed 3D face mesh model. Calculate the average curvature of each vertex. and Gaussian curvature Micro-expressions cause instantaneous changes in local curvature. By comparing consecutive time points... and The curvature field can be used to quantify deformation. A three-dimensional deformation eigenvector is defined. Statistics on curvature changes in key areas (such as the corners of the eyes and mouth):

[0104] in, Indicates a specific facial area Inner mean curvature The average change It is the standard deviation of curvature change within the region, used to describe the severity and extent of deformation.

[0105] Secondly, temporal analysis was performed to extract iris micro-motion features. Frames that clearly captured the eyeball were selected from the multi-view image sequence, aligned, and then the iris region was extracted. The displacement field of the iris texture is calculated using optical flow or feature point tracking. Iris micromotion characteristics It can be extracted from the frequency domain characteristics of the displacement field:

[0106] in, By analyzing displacement components or The dominant frequency obtained after performing a Fourier transform reflects the periodicity of the micro-motion; and They are and The standard deviation of a component over a time series characterizes the magnitude of the fluctuation.

[0107] Finally, feature synthesis and liveness detection are performed. The extracted features... Input a pre-defined live biometric model (e.g., a support vector machine (SVM) or a neural network classifier). The model has learned the decision boundaries of liveness features during the training phase. Consistency comparison score. Output from the model:

[0108] in, These are the model's weight parameters. Liveness detection results are based on scores. With decision threshold Comparison generation: If If the result matches, it is determined to be a living organism; otherwise, it is determined to be a non-living organism.

[0109] This embodiment provides a multimodal, contactless triggering method for four-way access control in a nuclear island. A top wide-angle camera senses pedestrian density, and auxiliary cameras on both sides perform preliminary analysis of hand-held object features to generate a panoramic collaborative wake-up command to activate the four-way camera array. The array is then scheduled to collect multi-view images and depth data for precise object-held state recognition and to generate an adaptive recognition strategy. Finally, multimodal data is fused for 3D liveness detection and facial feature comparison, and access control signals are output based on the identity verification results. This method achieves contactless, high-precision personnel access control, effectively improving the security and intelligence level of the nuclear island access control system while reducing system power consumption.

[0110] In one feasible embodiment, the dual scanning cameras can be replaced with near-infrared binocular cameras, which improves the accuracy of object recognition in low-light conditions at night, with a slight increase in hardware costs, and the algorithm network is adapted to infrared grayscale images simultaneously.

[0111] It should be noted that, to improve recognition accuracy in cloudy, dark, or other low-light environments, the dual scanning cameras can be replaced with near-infrared binocular cameras to enhance object recognition accuracy in low-light conditions at night. The corresponding image recognition algorithm will be adaptively updated to accommodate infrared grayscale images.

[0112] In one feasible embodiment, the edge unit can adopt a multi-module distributed edge node, with each of the four cameras equipped with a micro computing module. After local preprocessing, the data is aggregated to the central control unit, which is suitable for ultra-long channel multi-access control serial scenarios.

[0113] It should be noted that, in order to improve processing efficiency, the four cameras can be equipped with a micro computing module. After local preprocessing, the preprocessed data is aggregated to the central control unit, which is suitable for scenarios with multiple access control systems connected in ultra-long channels.

[0114] In one feasible embodiment, a millimeter-wave radar can be added as an auxiliary macroscopic sensing mode to form a dual-redundant crowd detection with the top wide-angle camera, thereby improving sensing stability in high-dust and strong-steam nuclear island environments.

[0115] It should be noted that, in order to improve the accuracy of crowd detection, millimeter-wave radar can be added as an auxiliary macroscopic sensing mode for high dust and strong steam nuclear island environments, so as to achieve redundant detection of crowd density.

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the nuclear island four-way access control method based on multimodal sensorless triggering. Any simple modifications based on this technical concept are within the protection scope of this application.

[0117] This application also provides a multimodal, contactless triggering four-way access control device for the nuclear island, please refer to... Figure 3 The nuclear island four-way access control device based on multimodal sensorless triggering includes: Information perception module 10 is used to perceive the density of people in the full-area image captured by the top wide-angle camera, obtain the status information of people in the area, and perform hand area feature analysis on the image data captured by the auxiliary cameras on both sides to obtain preliminary information on the holding status. Trigger determination module 20 is used to determine trigger conditions based on the regional personnel status information and the preliminary object holding status information, and obtain a panoramic collaborative wake-up command; The collaborative scheduling module 30 is used to perform collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command, and to collect multi-view image data and depth data. The object holding recognition module 40 is used to accurately recognize the object holding features of the person's hand image data in the multi-view image data, obtain the accurate object holding state, and generate an adaptive recognition strategy based on the accurate object holding state. The identity verification module 50 is used to perform three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data based on the adaptive recognition strategy, obtain the identity verification result, and output the access control signal based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0118] In one feasible implementation, the information sensing module 10 is further used to control the auxiliary cameras on both sides to perform intermittent image acquisition according to a preset sleep and wake-up cycle, so as to obtain a local area image sequence. Feature extraction of the hand contour and associated object contours is performed on the local region image sequence to obtain hand region contour data; Extract morphological and contour features from the hand region contour data, and generate preliminary holding status information based on the morphological and contour features.

[0119] In one feasible implementation, the information sensing module 10 is further used to perform grayscale conversion and noise suppression processing on the local region image sequence to obtain preprocessed image data; Edge gradient feature calculation is performed on the preprocessed image data to obtain image edge intensity and direction information; Based on the edge strength and direction information, contour continuity analysis is performed to separate the hand contour and the object contour connected to the hand. The hand contour and the object contour are combined to obtain the hand region contour data.

[0120] In one feasible implementation, the trigger determination module 20 is further used to parse the personnel status information in the area to obtain the population density in the area; The initial holding state information is analyzed to obtain the holding characteristics; When the pedestrian density in the area is greater than the preset pedestrian density or the object holding feature is a valid object holding feature, the panoramic collaborative wake-up command is generated to start the four-way camera array to operate at full power.

[0121] In one feasible implementation, the trigger determination module 20 is further used to perform cluster analysis on the personnel location coordinates contained in the personnel status information of the area to obtain several independent personnel target points. The number of personnel target points within the preset identification area is counted, and the number of personnel per unit area in real time is calculated by combining the area parameter of the area. The real-time number of people per unit area is smoothed and filtered with historical data within a preset time threshold to obtain the regional population density.

[0122] In one feasible implementation, the identity verification module 50 is further configured to fuse the multi-view image data and depth data based on the adaptive recognition strategy to construct a three-dimensional face model data. Liveness feature analysis is performed on the aforementioned 3D face model data to obtain liveness detection results; When the liveness detection result is a preset detection result, two-dimensional facial features are extracted from the multi-view image data and the matching degree is calculated with the locally pre-stored authorized personnel feature library to obtain the matching confidence. An authentication result is generated based on the matching confidence level.

[0123] In one feasible implementation, the identity verification module 50 is further used to perform surface curvature change analysis on the three-dimensional face model data to obtain three-dimensional deformation features caused by micro-expression movements. Temporal analysis was performed on the multi-view images to extract iris micro-motion features of the iris texture; The three-dimensional deformation features and the iris micro-motion features are combined and compared with a preset live biometric model to obtain the comparison results. The liveness detection result is generated based on the comparison results.

[0124] In one feasible implementation, the collaborative scheduling module 30 is further configured to simultaneously activate the top wide-angle camera, the side auxiliary cameras, and the front depth camera according to the panoramic collaborative wake-up command. Based on the current location information of the person, the focal length and shooting angle parameters of each camera in the four-way camera array are dynamically adjusted to form a three-dimensional acquisition field of view covering the person's face. The controlled and adjusted four-way camera array synchronously acquires visible light image data and depth information data containing facial information of people, thus obtaining multi-view image data and depth data.

[0125] In one feasible implementation, the object holding recognition module 40 is further used to extract image data from the auxiliary cameras on both sides from the multi-view image data to accurately calibrate the hand feature points and the bounding box of the handheld object, so as to obtain the accurate object holding state. When the precise holding state is holding the object with both hands, the preset standard face recognition time window is extended to a first predetermined duration, and the exposure compensation parameters are updated simultaneously. An adaptive recognition strategy is generated based on the first predetermined duration and the exposure compensation parameters.

[0126] The multimodal, contactless triggering-based four-way access control device for the nuclear island provided in this application employs the multimodal, contactless triggering-based four-way access control method for the nuclear island described in the above embodiments, which can solve the technical problems of low access control recognition accuracy and insufficient security in the nuclear island. Compared with the prior art, the beneficial effects of the multimodal, contactless triggering-based four-way access control device for the nuclear island provided in this application are the same as those of the multimodal, contactless triggering-based four-way access control method for the nuclear island provided in the above embodiments, and other technical features in the multimodal, contactless triggering-based four-way access control device for the nuclear island are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0127] This application provides a multimodal, contactless triggering four-way access control device for a nuclear island. The multimodal, contactless triggering four-way access control device for a nuclear island includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multimodal, contactless triggering four-way access control method for a nuclear island described in Embodiment 1 above.

[0128] The following is for reference. Figure 4 This document illustrates a structural schematic diagram of a core island four-way access control device suitable for implementing the embodiments of this application based on multimodal contactless triggering. The core island four-way access control device based on multimodal contactless triggering in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, tablets, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The nuclear island four-way access control device based on multimodal sensorless triggering shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0129] like Figure 4As shown, the multimodal, contactless triggering-based four-way access control device for the nuclear island may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the multimodal, contactless triggering-based four-way access control device for the nuclear island. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the multimodal, contactless triggering-based nuclear island four-way access control device to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a multimodal, contactless triggering-based nuclear island four-way access control device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented or possessed alternatively.

[0130] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0131] The multimodal, contactless triggering-based four-way access control device for the nuclear island provided in this application employs the multimodal, contactless triggering-based four-way access control method for the nuclear island described in the above embodiments, and can solve the technical problem of multimodal, contactless triggering-based four-way access control for the nuclear island. Compared with the prior art, the beneficial effects of the multimodal, contactless triggering-based four-way access control device for the nuclear island provided in this application are the same as the beneficial effects of the multimodal, contactless triggering-based four-way access control method for the nuclear island provided in the above embodiments, and other technical features in this multimodal, contactless triggering-based four-way access control device for the nuclear island are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0132] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0133] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0134] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the multimodal-based sensorless triggering four-way access control method for the nuclear island described in the above embodiments.

[0135] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0136] The aforementioned computer-readable storage medium may be included in a multimodal, contactless triggering nuclear island four-way access control device; or it may exist independently and not be assembled into a multimodal, contactless triggering nuclear island four-way access control device.

[0137] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the multimodal contactless triggering core island four-way access control device, the multimodal contactless triggering core island four-way access control device: performs crowd density perception on the full-area image captured by the top wide-angle camera to obtain regional personnel status information, and performs hand area feature analysis on the image data captured by the auxiliary cameras on both sides to obtain preliminary holding status information. Based on the regional personnel status information and the preliminary object holding status information, a trigger condition is determined to obtain a panoramic collaborative wake-up command; According to the panoramic collaborative wake-up command, the four-way camera array is coordinated and parameters are scheduled to collect multi-view image data and depth data. Accurately identify the holding features of the person's hand in the multi-view image data to obtain the accurate holding state, and generate an adaptive recognition strategy based on the accurate holding state; Based on the adaptive recognition strategy, three-dimensional liveness detection and authorization feature comparison are performed on the multi-view images and face data to obtain the identity verification result, and an access control signal is output based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

[0138] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0140] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0141] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described multimodal, contactless triggering-based four-way access control method for the nuclear island. This solves the technical problem of multimodal, contactless triggering-based four-way access control for the nuclear island. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multimodal, contactless triggering-based four-way access control method for the nuclear island provided in the above embodiments, and will not be repeated here.

[0142] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multimodal-based sensorless triggering four-way access control method for the core island as described above.

[0143] The computer program product provided in this application can solve the technical problems of low accuracy and insufficient security of nuclear island access control. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the nuclear island four-way access control method based on multimodal contactless triggering provided in the above embodiments, and will not be repeated here.

[0144] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for non-sensory triggering of a nuclear island four-way access control based on multimodal operation, characterized in that, The nuclear island four-way access control method based on multimodal sensorless triggering includes: The system uses the wide-angle image captured by the top camera to perceive the density of people and obtain the status information of people in the area. It also uses the image data captured by the auxiliary cameras on both sides to analyze the hand area features and obtain preliminary information on the holding status. Based on the regional personnel status information and the preliminary object holding status information, a trigger condition is determined to obtain a panoramic collaborative wake-up command; According to the panoramic collaborative wake-up command, the four-way camera array is coordinated and parameters are scheduled to collect multi-view image data and depth data. Accurately identify the holding features of the person's hand in the multi-view image data to obtain the accurate holding state, and generate an adaptive recognition strategy based on the accurate holding state; Based on the adaptive recognition strategy, three-dimensional liveness detection and authorization feature comparison are performed on the multi-view images and face data to obtain the identity verification result, and an access control signal is output based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

2. The method as described in claim 1, characterized in that, The steps for performing hand region feature analysis on image data collected by the auxiliary cameras on both sides to obtain preliminary information on the holding status include: The auxiliary cameras on both sides are controlled to intermittently acquire images according to a preset sleep and wake-up cycle to obtain a sequence of images of a local area; Feature extraction of the hand contour and associated object contours is performed on the local region image sequence to obtain hand region contour data; Extract morphological and contour features from the hand region contour data, and generate preliminary holding status information based on the morphological and contour features.

3. The method as described in claim 2, characterized in that, The step of extracting features of the hand contour and the contour of associated objects from the local region image sequence to obtain hand region contour data includes: The local region image sequence is subjected to grayscale conversion and noise suppression processing to obtain preprocessed image data; Edge gradient feature calculation is performed on the preprocessed image data to obtain image edge intensity and direction information; Based on the edge strength and direction information, contour continuity analysis is performed to separate the hand contour and the object contour connected to the hand. The hand contour and the object contour are combined to obtain the hand region contour data.

4. The method as described in claim 1, characterized in that, The step of determining the trigger condition based on the regional personnel status information and the preliminary object holding status information to obtain the panoramic collaborative wake-up command includes: The population status information of the area is analyzed to obtain the population density of the area; The initial holding state information is analyzed to obtain the holding characteristics; When the pedestrian density in the area is greater than the preset pedestrian density or the object holding feature is a valid object holding feature, the panoramic collaborative wake-up command is generated to start the four-way camera array to operate at full power.

5. The method as described in claim 4, characterized in that, The step of parsing the population status information of the area to obtain the population density of the area includes: Cluster analysis is performed on the personnel location coordinates contained in the personnel status information of the area to obtain several independent personnel target points; The number of personnel target points within the preset identification area is counted, and the number of personnel per unit area in real time is calculated by combining the area parameter of the area. The real-time number of people per unit area is smoothed and filtered with historical data within a preset time threshold to obtain the regional population density.

6. The method as described in claim 1, characterized in that, Based on the adaptive recognition strategy, the steps of performing three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data to obtain the identity verification result include: Based on the adaptive recognition strategy, the multi-view image data and depth data are fused to construct a three-dimensional face model data; Liveness feature analysis is performed on the aforementioned 3D face model data to obtain liveness detection results; When the liveness detection result is a preset detection result, two-dimensional facial features are extracted from the multi-view image data and the matching degree is calculated with the locally pre-stored authorized personnel feature library to obtain the matching confidence. An authentication result is generated based on the matching confidence level.

7. The method as described in claim 6, characterized in that, The step of performing liveness feature analysis on the 3D face model data to obtain liveness detection results includes: Surface curvature change analysis was performed on the three-dimensional facial model data to obtain the three-dimensional deformation features caused by micro-expression movements; Temporal analysis was performed on the multi-view images to extract iris micro-motion features of the iris texture; The three-dimensional deformation features and the iris micro-motion features are combined and compared with a preset live biometric model to obtain the comparison results. The liveness detection result is generated based on the comparison results.

8. The method as described in claim 1, characterized in that, The step of performing collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command to acquire multi-view image data and depth data includes: According to the panoramic collaborative wake-up command, the top wide-angle camera, the two auxiliary cameras on both sides and the front depth camera are activated simultaneously. Based on the current location information of the person, the focal length and shooting angle parameters of each camera in the four-way camera array are dynamically adjusted to form a three-dimensional acquisition field of view covering the person's face. The controlled and adjusted four-way camera array synchronously acquires visible light image data and depth information data containing facial information of people, thus obtaining multi-view image data and depth data.

9. The method as described in claim 1, characterized in that, The steps of accurately identifying the holding features of the person's hand image data in the multi-view image data to obtain the accurate holding state, and generating an adaptive recognition strategy based on the accurate holding state, include: The image data collected from the auxiliary cameras on both sides is extracted from the multi-view image data to accurately calibrate the hand feature points and the bounding box of the handheld object, so as to obtain the accurate holding state. When the precise holding state is holding the object with both hands, the preset standard face recognition time window is extended to a first predetermined duration, and the exposure compensation parameters are updated simultaneously. An adaptive recognition strategy is generated based on the first predetermined duration and the exposure compensation parameters.

10. A nuclear island four-way access control device based on multimodal, non-sensory triggering, characterized in that, The multimodal, contactless triggering four-way access control device for the nuclear island includes: The information perception module is used to perceive the density of people in the full-area image captured by the top wide-angle camera to obtain the status information of people in the area, and to perform hand area feature analysis on the image data captured by the auxiliary cameras on both sides to obtain preliminary information on the holding status. The trigger determination module is used to determine the trigger conditions based on the personnel status information and the preliminary object holding status information in the area, and to obtain the panoramic collaborative wake-up command. The collaborative scheduling module is used to perform collaborative parameter scheduling on the four-way camera array according to the panoramic collaborative wake-up command, and to collect multi-view image data and depth data. The object holding recognition module is used to accurately identify the object holding features of the person's hand image data in the multi-view image data, obtain the accurate object holding state, and generate an adaptive recognition strategy based on the accurate object holding state. The identity verification module is used to perform three-dimensional liveness detection and authorization feature comparison on the multi-view images and face data based on the adaptive recognition strategy, obtain the identity verification result, and output the access control signal based on the identity verification result. The face data is determined from the depth data collected by the frontal depth camera.

Images