Adaptive control method and system for access control

Abstract

The application relates to the technical field of image recognition, in particular to a self-adaptive control method and system for access control. The method comprises the following steps: acquiring an infrared thermal image of an access control area, determining a candidate extreme point pair from the infrared thermal image; determining whether a thermal separation boundary point exists in the infrared thermal image based on the candidate extreme point pair; acquiring a visible light image of the access control area, and determining whether a contour interruption feature exists in the visible light image by using the thermal separation boundary point; taking the thermal separation boundary point and the contour interruption feature as the reference, determining a first priority feature cluster and a second priority feature cluster; and outputting an anti-following control instruction for the access control device in the case that illegal accompanying entities exist in the first priority feature cluster and the second priority feature cluster. Through the above technical scheme, self-adaptive control of the access control can be realized.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to an adaptive control method and system for access control. Background Technology

[0002] Access control and attendance management systems are core components of modern building security and personnel scheduling. They are used in various corporate parks and commercial office buildings for authentication, access control, and recording of attendance times. Currently, facial recognition can be performed using visible light cameras. After obtaining the facial features of the person passing through, the system confirms the person's identity by comparing them with a local database, controls the gate to open and release the person, and simultaneously records the current time as attendance data.

[0003] For example, Chinese patent application CN112836634A discloses a method for preventing tailgating at turnstiles by using multi-sensor information fusion. This method involves using at least two sensors to acquire images at the same time, correcting and aligning the acquired images, normalizing the processed images, and inputting the normalized images into a pre-trained neural network model to obtain the position coordinates of the heads in the image. The method then determines whether tailgating exists based on the number of heads.

[0004] However, when a tailgating entity gets close to a legitimate entity, the recognition system may have blind spots. In visible light images, due to similar clothing colors or changes in lighting conditions, the visual features of two people who are close together can easily become blurred, making it difficult to distinguish their edge contours. This can cause the access control and attendance system to mistakenly identify the authorized person and the tailgating person as a single legitimate entity, thus granting access to the tailgating person as well, which poses a security risk to the controlled area. Therefore, in order to ensure the security of the controlled area, it is necessary to implement adaptive control of the access control system. Summary of the Invention

[0005] To achieve adaptive control of access control, this application provides an adaptive control method and system for access control.

[0006] According to a first aspect of the embodiments of this application, an adaptive control method for access control is provided, comprising: acquiring an infrared thermal image of an access control area; locating at least two local thermal extrema points in the infrared thermal image with a spatial distance within a preset interval as candidate extrema point pairs; based on the candidate extrema point pairs, determining whether there are thermal separation boundary points in the infrared thermal image by analyzing the temperature concavity features on the path connecting the two candidate extrema points; the thermal separation boundary points are used to characterize the physical isolation between two independent heat sources; acquiring a visible light image of the access control area; when it is determined that there are thermal separation boundary points, mapping the coordinates of the thermal separation boundary points to the visible light image, and determining whether there are contour interruption features in the neighborhood of the mapped coordinates that characterize the obstruction of a front entity by a rear entity; when it is determined that there are contour interruption features, using the thermal separation boundary points and contour interruption features as a reference, segmenting feature data that are spatially adjacent and stuck together in the visible light image and the infrared thermal image into a first priority feature cluster and a second priority feature cluster; and when it is determined that there are illegal accompanying entities in the first priority feature cluster and the second priority feature cluster, outputting an anti-tailgating control command for the access control device.

[0007] This enables adaptive control of access control, effectively preventing tailgating behavior that may occur within the access control area and ensuring the security of the access control area.

[0008] Optionally, local thermal extrema in the infrared thermal image are determined in the following way: for any candidate pixel grid in all pixel grids in the infrared thermal image, if the temperature information of the candidate pixel grid is greater than the temperature information of other pixel grids in the neighborhood, the center point of the candidate pixel grid is determined as the local thermal extrema.

[0009] Optionally, at least two local thermal extrema points in the infrared thermal image are located within a preset interval as candidate extrema point pairs, including: determining the straight-line pixel distance between any two local thermal extrema points in the two-dimensional plane coordinate system of the infrared thermal image, and combining the extrema points whose straight-line pixel distance is between a preset lower limit and a preset upper limit to determine the candidate extrema point pairs.

[0010] Optionally, by analyzing the temperature concavity features along the path connecting two candidate extreme points, it is determined whether there are thermal separation boundary points in the infrared thermal image. This includes: constructing a virtual straight line segment connecting the centers of the two extreme points in the candidate extreme point pair in a two-dimensional temperature matrix, and determining the target pixel grid that the virtual straight line segment passes through in the two-dimensional temperature matrix; extracting absolute temperature values ​​one by one along the path of the target pixel grid, and assembling the extracted absolute temperature values ​​into a temperature data sequence according to the spatial arrangement order of the target pixel grid; determining the minimum point based on the gradient rate of change in the temperature data sequence, and determining the thermal separation boundary point based on the spatial coordinates of the minimum point.

[0011] In this way, by constructing a temperature data sequence along the path connecting extreme points and finding the minimum point, it is possible to accurately capture the slight temperature attenuation zone formed between two close heat sources due to clothing or air gaps. Even if two entities are closely attached, the boundary coordinates that characterize physical isolation can be accurately located, thus improving the precision of entity separation.

[0012] Optionally, the minimum point in the temperature data sequence based on the gradient rate of change is determined, and the thermal separation boundary point is determined according to the spatial coordinates of the minimum point. This includes: obtaining the temperature gradient descent slope between adjacent pixel grids in the temperature data sequence, extracting the minimum point with the lowest temperature value based on the temperature gradient descent slope; obtaining the temperature concavity difference between the absolute temperature value of two candidate extreme points and the absolute temperature value of the minimum point, and in response to determining that all temperature concavity differences are greater than a preset lower limit, using the spatial coordinates of the minimum point as the thermal separation boundary point.

[0013] In this way, by introducing a two-way comparison between the temperature concavity difference and the lower limit of the difference, false temperature fluctuations caused by clothing wrinkles or uneven local heat dissipation on a single heat source surface are effectively filtered out, ensuring that the extracted thermal separation boundary points truly correspond to the physical gap between two independent entities, thus reducing the false alarm rate caused by environmental interference.

[0014] Optionally, the coordinates of the thermal separation boundary points are mapped to the visible light image to determine whether there are contour interruption features caused by the occlusion of the front entity by the rear entity in the neighborhood of the mapped coordinates. This includes: mapping the coordinates of the thermal separation boundary points in the infrared thermal image to the pixel grid of the visible light image to obtain the corresponding mapped coordinates; extending the edge analysis window downward with the mapped coordinates as the origin to define the edge analysis window; extracting continuous horizontal edge lines within the edge analysis window and tracing them to both horizontal sides; and determining the contour interruption features by the absolute value change of the vertical gradient on the tracing path.

[0015] Optionally, a continuous horizontal edge line is extracted within the edge analysis window and tracked horizontally to both sides. The contour interruption feature is determined by the change in the absolute value of the vertical gradient along the tracked path. This includes: performing edge detection on the pixel grayscale matrix of the visible light image to obtain a continuous horizontal edge line representing the features of the human shoulder; advancing pixel by pixel along the continuous horizontal edge line to the left and right sides to extract the absolute value of the vertical gradient along the tracked path; and determining that the horizontal gradient decreases during continuous tracking and the absolute value of the vertical gradient increases within a preset number of adjacent pixels and exceeds a preset abrupt gradient threshold, thus determining that the lateral connectivity of the continuous horizontal edge line is blocked, and using the abscissa of the location where the lateral connectivity is blocked as the abscissa of the contour interruption feature.

[0016] In this way, by monitoring the dramatic changes in the vertical gradient along the horizontal edge of the visible light image, it is possible to accurately identify the visual obstruction caused by the body of a following entity occluding the shoulder line of a legitimate entity in front, thus guiding infrared positioning to visual detail analysis and further confirming the spatial occlusion relationship of the following entity.

[0017] Optionally, based on thermal separation boundary points and contour interruption features, feature data that are spatially adjacent and adhered in visible light images and infrared thermal images are segmented into a first priority feature cluster and a second priority feature cluster. This includes: obtaining the absolute value of the coordinate difference between the abscissa of the contour interruption feature and the abscissa of the thermal separation boundary point; if the absolute value of the coordinate difference is less than a preset tolerance error parameter, confirming that the entity corresponding to the second priority feature cluster has truncated the contour of the entity corresponding to the first priority feature cluster; and inserting a data segmentation surface into the feature data based on the line connecting the thermal separation boundary point and the location where the contour interruption occurs, thereby dividing the feature data into a first priority feature cluster and a second priority feature cluster.

[0018] In this way, after verifying the consistency between the infrared mapping coordinates and the horizontal coordinates of the visible light interruption features, the data segmentation surface is inserted. This fully combines the spatial alignment constraints of the two modes, avoids single-mode noise interference, and makes the segmented first priority feature cluster and second priority feature cluster have extremely high spatial independence.

[0019] Optionally, if it is determined that there is an illegal accompanying entity in the first priority feature cluster and the second priority feature cluster, an anti-tailgating control command for the access control device is output, including: extracting the facial geometric information contained in the first priority feature cluster and the second priority feature cluster respectively, and sending the facial geometric information to the permission module to perform the identity verification procedure; in response to the first priority feature cluster meeting the preset access permission and the second priority feature cluster not containing registration information, an anti-tailgating control command is output to block the access control device's release loop.

[0020] Optionally, the method further includes: performing grayscale difference operations on consecutive video sequence frames in the visible light image to determine the moving target that has generated displacement; extracting the geometric contour of the moving target to generate dynamic bounding box coordinates; mapping the dynamic bounding box coordinates to the infrared thermal image to establish a local tracking window that follows the movement of the moving target; and determining the infrared boundary contour of the moving target by obtaining the relative temperature fluctuations inside the local tracking window.

[0021] Optionally, the infrared boundary contour of the moving target is determined by acquiring the relative temperature fluctuations within the local tracking window, including: taking the moving average of the temperature of all pixel grids within the core area of ​​the local tracking window as a reference temperature value, obtaining the relative temperature difference between the background pixel grids at the inner edge of the local tracking window and the reference temperature value; performing line-by-line scanning within the local tracking window to obtain the dynamic gradient slope of the relative temperature difference, and determining the abrupt change position of the dynamic gradient slope as the infrared boundary contour point of the moving target.

[0022] Optionally, the method further includes: analyzing and extracting local pixel trailing regions from a single frame of visible light image; performing two-dimensional directional derivative calculations on the high-frequency spatial components of the local pixel trailing regions to obtain the two-dimensional instantaneous motion vector direction and trailing length value of the moving target; and determining the relative displacement deviation coordinate vector of the moving target in the infrared thermal image based on the time difference constant between the triggering time of the infrared thermal image acquisition and the exposure period of the acquired visible light image, combined with the two-dimensional instantaneous motion vector direction and trailing length value.

[0023] Optionally, the method further includes: determining the inverse vector of the relative displacement deviation coordinate vector as the spatial compensation offset, applying the spatial compensation offset in reverse to the bottom pixel coordinate system of the infrared thermal image, and performing translation reconstruction on the pixel grid of the infrared thermal image through a bilinear interpolation algorithm to obtain a corrected infrared thermal image with spatial alignment.

[0024] According to a second aspect of the present application, an adaptive control system for access control is provided, comprising: a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions, when executed by the processor, implement the steps of the adaptive control method for access control provided in the first aspect of the present application.

[0025] The technical solutions provided by the embodiments of this application may include the following beneficial effects: by extracting the concave features of temperature through infrared thermography and combining them with the contour interruption features in the visible light image, the interference of thermal overlap caused by clothing obstruction or close body contact in a single mode can be avoided, thus achieving accurate feature segmentation of tailgating personnel. When it is determined that there are tailgating personnel, the illegal intrusion behavior of unauthorized tailgating personnel can be effectively blocked, thereby improving the reliability of the access control system for protection.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an adaptive control method for access control according to an exemplary embodiment; Figure 2This is a comparative schematic diagram of the segmentation results of the first priority feature cluster and the second priority feature cluster in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the structure of an adaptive control system for access control according to an exemplary embodiment. Detailed Implementation

[0028] To achieve adaptive control of access control, embodiments of this application provide an adaptive control method and system for access control. Figure 1 This is a flowchart illustrating an adaptive control method for access control according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps.

[0029] In step S101, candidate extreme point pairs are determined from the infrared thermal image.

[0030] Acquire an infrared thermal image of the access control area, and locate at least two local thermal extrema points within a preset spatial distance from the infrared thermal image, and use them as candidate extrema point pairs.

[0031] The adaptive control method for access control provided in this application can be applied to an integrated time and attendance access control system, which is typically deployed in pedestrian passageways with complex physical environments.

[0032] The multispectral sensor module deployed in the access control system can simultaneously acquire multiple modal data of the access control area. The multispectral sensor module can include an infrared thermal imaging sensor and a visible light high-definition sensor, with the infrared thermal imaging sensor and the visible light high-definition sensor in a fixed relative position.

[0033] The adaptive control method for access control can be executed by an edge computing terminal with an integrated microprocessor to achieve the goals of real-time anti-tailgating and accurate attendance statistics. The edge computing terminal is connected to the gate controller of the access control equipment through a data bus to ensure low latency in the transmission of control signals.

[0034] The integrated attendance and access control system can integrate attendance machines into the access control decision-making chain, realizing the synchronization of physical blocking and digital recording. The access control's internal permission module can embed an employee database and clock-in timestamp generation logic. Only after eliminating the suspicion of tailgating from behind and confirming the legitimate identity of the entity in front will the gate motor be driven to perform the door opening operation, and the attendance details of the passerby will be written to the local flash memory within the same clock cycle.

[0035] Access control areas are typically divided into warning zones and verification zones. When an entity enters the warning zone beyond the detection range of the distance sensor, the infrared thermal imaging sensor can trigger a continuous sampling mechanism. The infrared thermal image reflects the spatial distribution of thermal radiation intensity on various surfaces within the access control area and can be converted into a two-dimensional temperature matrix through a front-end analog-to-digital conversion circuit.

[0036] In a two-dimensional temperature matrix with a resolution of, for example, 640 x 512, each pixel represents the temperature information corresponding to a location point in real three-dimensional space after calibration. By traversing the entire two-dimensional temperature matrix and extracting the coordinates of pixels with higher local thermal radiation intensity, local thermal extreme points that can represent the core location of the heat source can be located.

[0037] Local thermal extreme points usually correspond to exposed skin areas such as the face or neck of the human body that are not covered by clothing, reflecting the spatial projection position of the human body as a heat source on the two-dimensional focal plane; by setting a spatial preset interval to filter out heat source combinations with too large or too small spatial spans, only extreme point combinations that may have a close relationship can be retained.

[0038] By using preset intervals for filtering, we avoid misjudging independent pedestrians who are far apart and do not interfere with each other as tailgating personnel, thus reducing the computational burden of subsequent high-complexity spatial analysis. The lower limit of the preset interval is used to filter multiple scattered thermal high points of the same entity, and the upper limit of the preset interval is used to define the distance boundary with tailgating risk.

[0039] In one embodiment, local thermal extrema in an infrared thermal image are determined as follows: for any candidate pixel grid among all pixel grids in the infrared thermal image, if the temperature information of the candidate pixel grid is greater than the temperature information of other pixel grids in the neighborhood, the center point of the candidate pixel grid is determined as the local thermal extrema.

[0040] The corresponding two-dimensional temperature matrix is ​​extracted from the acquired infrared thermal image, and a sliding window convolution traversal operation is performed inside the matrix. For example, a square analysis window with a side length of 5 pixels is set, and the two-dimensional temperature matrix is ​​scanned row by row in the order from left to right and from top to bottom.

[0041] There are multiple pixel grids inside the square analysis window. The grid located at the geometric center is extracted and defined as a candidate pixel grid. The temperature information inside the candidate pixel grid is read. The temperature information of the candidate pixel grid is compared with the temperature information of the surrounding pixel grids in the square analysis window so that the extracted local thermal extreme points can represent the center of the human body heat source and avoid mistaking a large area of ​​warm background wall for a human body.

[0042] If the temperature information of a candidate pixel grid is greater than the temperature information of all surrounding neighboring pixel grids, it can be determined that the candidate pixel grid belongs to a local maximum temperature accumulation area. The two-dimensional row and column coordinates of the candidate pixel grid are extracted, and the coordinate values ​​of the geometric center point of the candidate pixel grid are recorded as the local thermal extreme point.

[0043] In one embodiment, locating at least two local thermal extrema points in an infrared thermal image whose spatial distance is within a preset range as candidate extrema point pairs includes: determining the linear pixel distance between any two local thermal extrema points in the two-dimensional plane coordinate system of the infrared thermal image, and combining extrema points whose linear pixel distance is between a preset lower limit and a preset upper limit to determine candidate extrema point pairs.

[0044] The two-dimensional plane coordinate system of the infrared thermal image takes the upper left corner of the image as the origin, extends horizontally to the right as the horizontal axis, and extends vertically downwards as the vertical axis. By extracting the coordinate values ​​of any two located local thermal extreme points, performing the sum of squares of the coordinate differences and taking the square root, the Euclidean distance between any two points in the Cartesian coordinate system can be calculated, thus obtaining the straight-line pixel distance.

[0045] After obtaining the straight-line pixel distance, the straight-line pixel distance can be compared with the preset lower limit and preset upper limit values ​​respectively. For example, the preset lower limit can be set to 20 pixels. When the calculated distance is less than 20 pixels, it means that the two extreme points may belong to the same human body's brow and nose tip, and do not have the value of tailing judgment.

[0046] For example, a preset upper limit can be set to 150 pixels. When the calculated distance is greater than 150 pixels, it means that there is enough space between the two entities and there is no possibility of tailing them by sticking together.

[0047] In step S102, it is determined whether there are thermal separation boundary points in the infrared thermal image.

[0048] Based on candidate extreme point pairs, the presence of thermal separation boundary points in the infrared thermal image is determined by analyzing the temperature concavity features on the path connecting the two candidate extreme points.

[0049] The thermal separation boundary point is used to characterize the physical isolation between two independent heat sources. When two people move closely together in a channel, their respective heat radiation will be conducted to the surrounding air medium to form a thermal halo effect. There will be clothing fabric or tiny air gaps between the adjacent parts of the two, causing the surface temperature of the adjacent parts to be lower than the extreme temperature of the human body.

[0050] The gaps at the connection points will exhibit a curve shape in the spatial dimension of the infrared thermal image, where the temperature decreases and then increases along a straight path. This reflects the concave feature of the temperature under separation. Path exploration based on candidate extreme point pairs can simplify the global image scanning range and avoid the redundant floating-point computational consumption caused by full-image traversal search.

[0051] By monitoring the temperature decay caused by tiny gaps, it is possible to avoid the influence of dark clothing of the same color under visible light. The location where this concave shape decays most significantly is located at the cross-sectional projection position at the interface between the two body surfaces, thus confirming the existence of a thermal separation boundary point that can separate the entities.

[0052] In one embodiment, determining whether a thermal separation boundary point exists in an infrared thermal image by analyzing the temperature concavity features along the path connecting two candidate extreme points includes: constructing a virtual straight line segment connecting the centers of the two extreme points in a two-dimensional temperature matrix, and determining the target pixel grid that the virtual straight line segment passes through in the two-dimensional temperature matrix; extracting absolute temperature values ​​one by one along the path of the target pixel grid, and assembling the extracted absolute temperature values ​​into a temperature data sequence according to the spatial arrangement order of the target pixel grid; determining the minimum point based on the gradient rate of change in the temperature data sequence, and determining the thermal separation boundary point based on the spatial coordinates of the minimum point.

[0053] Within the two-dimensional temperature matrix, based on the coordinates of the selected candidate extreme points at both ends, a discretized line generation algorithm can be invoked, such as the Bressenham line algorithm, to generate virtual line segments. The Bressenham line algorithm calculates all pixel grid arrays passed through by the virtual line segment using integer addition and subtraction, forming a connected spatial continuous path.

[0054] All grid cells traversed by the virtual straight line segment are marked as target pixel grids. These grid cells represent the shortest heat transfer attenuation path from the previous heat source center to the next heat source center.

[0055] Temperature concave features refer to the curve shape where the temperature changes from high to low and then rises again along the connection path. According to the spatial order of pixels, the extracted absolute temperature values ​​are sequentially assembled into a one-dimensional set of numbers to obtain a temperature data sequence that represents the thermal fluctuation distribution along the connection direction. The first-order differential gradient can be calculated for the extracted temperature data sequence. By monitoring the process of the differential gradient changing from negative to positive, the minimum point reflecting the local trough within the temperature data sequence can be found.

[0056] The minimum point represents a spatial node where the temperature drops to a lower value along the direction of the virtual straight line segment. It reflects the location of the relatively weak isolation zone between the two heat source entities. The row and column numbers corresponding to the minimum point in the original two-dimensional temperature matrix can be extracted, and the positions corresponding to the row and column numbers can be used as the thermal separation boundary points.

[0057] In one embodiment, determining the minimum point in the temperature data sequence based on the gradient rate of change, and determining the thermal separation boundary point based on the spatial coordinates of the minimum point, includes: obtaining the temperature gradient descent slope between adjacent pixel grids in the temperature data sequence, extracting the minimum point with the lowest temperature value based on the temperature gradient descent slope; obtaining the temperature concavity difference between the absolute temperature values ​​of two candidate extreme points and the absolute temperature value of the minimum point, and in response to determining that all temperature concavity differences are greater than a preset lower limit of difference, using the spatial coordinates of the minimum point as the thermal separation boundary point.

[0058] It can iterate through each numerical cell of the temperature data sequence, subtract the absolute temperature value of the previous cell from the absolute temperature value of the next cell, and divide by the cell unit distance to obtain the slope of the temperature gradient descent at the relative position.

[0059] The sign property of the temperature gradient descent slope along the entire path is continuously monitored. When the slope value transitions from the negative range to the positive range, it indicates that a local temperature trough has been reached. If multiple local troughs with sign reversal are detected along the entire straight path, the absolute temperature values ​​carried by these local troughs are further compared to determine the minimum point.

[0060] By using the slope of a continuous gradient descent to determine the location of the local temperature lower, we can avoid misjudging slow temperature measurement noise fluctuations as physical isolation boundaries, thus enhancing the noise resistance of gap location. This ensures that the discovered concave area is actually located between two obvious heat sources, eliminating interference from unilateral temperature slippage caused by the heat dissipation area at the edge of the human body.

[0061] In step S103, it is determined whether there are contour interruption features in the visible light image.

[0062] Acquire a visible light image of the access control area. When a thermal separation boundary point is determined to exist, map the coordinates of the thermal separation boundary point to the visible light image. Determine whether there are contour interruption features in the neighborhood of the mapped coordinates that characterize the obstruction of the front entity by the rear entity.

[0063] Infrared thermal imaging is used to locate suspected isolated hot spots under conditions of no light or visual masking, and secondary morphological verification can be performed using visible light images. Visible light images, such as RGB images, can be captured by color image sensors and contain details of light and shadow transitions, textures, and the outline geometry of the human body's external clothing.

[0064] The physical coordinate systems of visible light camera lenses and infrared camera lenses can achieve a pixel-level three-dimensional spatial mapping relationship. By using the affine transformation matrix parameters, the coordinates of the thermodynamic separation boundary points determined in the temperature matrix can be projected onto the pixel coordinate system of the visible light image to determine the corresponding mapping area on the color visual plane.

[0065] The affine transformation matrix can be obtained by jointly calibrating infrared and visible light cameras using Zhang's calibration method. For details, please refer to the prior art for the calibration techniques. The embodiments of this application will not be repeated here.

[0066] When a follower tries to break through a gate by sticking close to the authorized personnel in front, they often use their own body structure to directly block the back or shoulders of the personnel in front, creating overlapping obstructions from a side view or a high-angle view.

[0067] By mapping infrared isolation coordinates to visible light space for contour breakage verification, the advantages of the two sensors are complemented, reducing the possibility of false alarms caused by clothing patterns when relying on visible light analysis. If an abnormal truncated line shape is found within a limited spatial distance around the mapped coordinates, combined with the isolation indication given by the infrared mode, it can be confirmed that a tailing event targeting the access control and attendance equipment has occurred.

[0068] In one embodiment, mapping the coordinates of thermal separation boundary points to a visible light image and determining whether there are contour interruption features caused by a rear entity occluding a front entity within the neighborhood of the mapped coordinates includes: mapping the coordinates of the thermal separation boundary points in the infrared thermal image to the pixel grid of the visible light image to obtain the corresponding mapped coordinates; extending the edge analysis window downwards from the origin of the mapped coordinates; extracting continuous horizontal edge lines within the edge analysis window and tracing them to both horizontal sides; and determining the contour interruption features by the absolute value change of the vertical gradient along the tracing path.

[0069] It can retrieve the pre-stored multispectral camera intrinsic and extrinsic parameter calibration matrix, substitute the lateral and longitudinal coordinates of the thermal separation boundary points determined in the infrared thermal image into the mapping transformation expression, and determine the pixel position in the visible light image.

[0070] Infrared gaps correspond to the gaps in the jaw or neck of the human body, and usually appear in the area above the two human torsos. By combining prior knowledge of human structure, the search area of ​​visible light features is determined, and the obtained mapping coordinates are used as the geometric origin. A certain rectangular width and height size limit is set in the vertical downward direction, and an edge analysis window can be established to limit the execution range of the edge operator.

[0071] Within the edge analysis window, for example, the edge extraction filter can capture the light and dark interface reflecting the outer contour line segment of the shoulder, extracting a continuous horizontal edge line. The pixel at the center of the line segment is selected as the starting node, and the line is gradually moved to the left and right along the texture trend of the line segment to determine the continuous extension of the horizontal edge line. When moving horizontally point by point, the pixel grayscale change rate between the pixels directly above and below the current analysis position is calculated to determine the absolute value of the vertical gradient, so as to characterize the sharpness of the edge contrast.

[0072] By directionally searching and tracking the horizontal contour gradient changes in the lower region from the determined mapping coordinates, the blind search for contour breakpoints in the complex clothing texture of the whole body is avoided, thus improving the frame rate of edge analysis.

[0073] In a normal and continuous human shoulder edge, the vertical gradient has a relatively stable and high numerical value. If the person in front is obscured by the arm or chest of the person behind, the edge texture will be destroyed, resulting in a sharp drop or abnormal jump in the vertical gradient, showing the specific location where the contour interruption feature exists.

[0074] In one embodiment, extracting continuous horizontal edge lines within an edge analysis window and tracing them horizontally to both sides, and determining contour interruption features by the change in the absolute value of the vertical gradient along the tracing path, includes: performing edge detection on the pixel grayscale matrix of a visible light image to obtain continuous horizontal edge lines characterizing the features of a human shoulder; advancing pixel by pixel along the continuous horizontal edge lines to the left and right sides, and extracting the absolute value of the vertical gradient along the tracing path; if it is determined that the horizontal gradient decreases during continuous tracing, and the absolute value of the vertical gradient increases within a preset number of adjacent pixels and exceeds a preset abrupt gradient threshold, it is determined that the lateral connectivity of the continuous horizontal edge lines is blocked, and the abscissa of the location where the lateral connectivity is blocked is used as the abscissa of the contour interruption feature.

[0075] By performing edge detection on the pixel grayscale matrix of a visible light image, continuous horizontal edge lines representing the features of a human shoulder can be obtained. For example, a color visible light image can be converted into a single-channel pixel grayscale matrix. Using the vertical convolution kernel of the Sobel operator, sliding convolution operations can be performed within the edge analysis window to determine the continuous horizontal edge lines used to represent the features of a human shoulder.

[0076] For the continuous horizontal edge line with the largest cumulative length, a region connectivity growth strategy is executed to advance to each pixel on both sides. At each moving node, a vertical convolution kernel is applied to calculate the luminance difference between the pixels above and below the current node, and the absolute value of the vertical gradient is obtained.

[0077] For example, the preset number of breaks can be set to 3 consecutive pixel widths. If the horizontal gradient of the edge direction shows a gradual decreasing trend, it indicates that the originally strong horizontal extension edge is weakening and is suspected to be interfered with by the structure of other entities.

[0078] If the absolute value of the vertical gradient increases sharply within a preset number of pixels and exceeds the preset abrupt gradient threshold used to distinguish the boundaries of different materials, by monitoring the natural decay of the horizontal gradient and the increase of the vertical gradient, the slow grayscale changes caused by the printed patterns or local shadows on the clothing surface can be eliminated, and the occlusion section between the occluding entity and the occluded entity can be accurately determined.

[0079] For example, the horizontally oriented shoulder contour is cut off by a vertical boundary, indicating that the continuous external boundary of the human body may be interrupted by the rigid or flexible clothing structure of the person following behind, and it can be determined that the lateral connectivity of the continuous horizontal edge line is blocked.

[0080] It can record the column index coordinates of the lateral connectivity blockage location in the two-dimensional pixel matrix system, and use the value of this lateral pixel that causes a spatial step change in pixel grayscale as the x-coordinate of the contour interruption feature reflecting the location of the occlusion, so that it can be called by the subsequent fusion and segmentation module.

[0081] The specific value of the preset mutation gradient threshold can be between 40 and 60. Specifically, it can be determined in advance based on sample images that can distinguish adjacent people who are tailing. For example, sample images of people who are tailing in different situations can be tested in advance, and the gradient values ​​that can effectively realize the contour interruption feature can be statistically analyzed. The preset mutation gradient threshold is determined based on the statistical analysis results. This application embodiment will not elaborate further on this.

[0082] In step S104, the first priority feature cluster and the second priority feature cluster are determined.

[0083] In cases where contour interruption features are identified, based on thermal separation boundary points and contour interruption features, feature data that are spatially adjacent and stuck together in visible light images and infrared thermal images are segmented into first-priority feature clusters and second-priority feature clusters.

[0084] When the presence of contour interruption features is confirmed, the attendance and access control system can determine that there are two independent heat sources in front of the passage, and they are in a front-to-back occlusion relationship. Since the two are extremely close, in the output results of the target detection model of the existing technology, the body contours of these two may be incorrectly enclosed in the same rectangular bounding box, resulting in incorrect identity identification.

[0085] The first priority feature cluster represents the multi-dimensional feature set corresponding to the legitimate authorized entity that is in the foreground of the screen, has a relatively larger exposed area, and occupies a dominant position in passage; the second priority feature cluster represents the multi-dimensional feature set corresponding to the entity that is attached to the background and attempts to follow.

[0086] This avoids the segmentation errors that may occur with single-modal segmentation, and can separate potentially contiguous tile pixels and thermal field data, separating the original fused data into two independent feature clusters, which facilitates subsequent separate review of access identity and attendance permissions.

[0087] In one embodiment, based on thermal separation boundary points and contour interruption features, feature data that are spatially adjacent and adhered in visible light images and infrared thermal images are segmented into a first priority feature cluster and a second priority feature cluster. This includes: obtaining the absolute value of the coordinate difference between the abscissa of the contour interruption feature and the abscissa of the thermal separation boundary point; if the absolute value of the coordinate difference is less than a preset tolerance error parameter, confirming that the entity corresponding to the second priority feature cluster has truncated the contour of the entity corresponding to the first priority feature cluster; and inserting a data segmentation surface into the feature data based on the line connecting the thermal separation boundary point and the location where the contour interruption occurs, thereby dividing the feature data into a first priority feature cluster and a second priority feature cluster.

[0088] Feature data refers to the fused data obtained by matching data corresponding to the same spatial location point in visible light images and infrared thermal images. For example, before segmenting the first priority feature cluster or the second priority feature cluster, data corresponding to the same spatial location point in visible light images and infrared thermal images can be matched to obtain fused data after matching the data of these two modalities. Feature data includes at least the fused data corresponding to the bodies of two people who may be following each other in the complete fused data.

[0089] Two people who may be tailing each other typically have partial overlap in at least one of the two modalities of visible light image or infrared light image, which may cause partial overlap between the first priority feature cluster and the second priority feature cluster. The smaller the distance between two adjacent people who may be tailing each other, the greater the overlap ratio or data volume of the first priority feature cluster and the second priority feature cluster. Conversely, the greater the distance between two adjacent people who may be tailing each other, the smaller the overlap ratio or data volume of the first priority feature cluster and the second priority feature cluster.

[0090] The obtained contour interruption features can be used to extract the abscissa values ​​in the visible light image coordinate system, and the thermal separation boundary points can be extracted in the visible light image coordinate system after affine mapping. The two are directly subtracted and the absolute value of the coordinate difference is calculated. The absolute value of the coordinate difference reflects the spatial dispersion of the isolation boundary point between the visible light mode and the infrared thermal mode in the horizontal dimension. The smaller the absolute value of the coordinate difference, the more consistent the isolation breakpoint positions given by the two modes are.

[0091] For example, a preset tolerance error parameter of 15 pixels can be set in advance to determine whether the absolute value of the coordinate difference is less than the allowable fluctuation range of 15 pixels, thereby effectively avoiding false alarms that may be caused by sensor array assembly and calibration jitter.

[0092] Within the data matrix of the two-dimensional hybrid image, a virtual baseline can be determined that passes through the mapped thermal separation boundary points and the location where the contour is interrupted. A virtual data segmentation surface is formed by projecting along this virtual baseline into the three-dimensional depth direction.

[0093] A mask blocking operation is performed based on the data segmentation surface, so that the connected pixels in the feature space are segmented and assigned to the first priority feature cluster or the second priority feature cluster according to their spatial location.

[0094] Figure 2 This is a comparative schematic diagram of the segmentation results of the first priority feature cluster and the second priority feature cluster in an embodiment of this application, as shown below. Figure 2 As shown, the original data before segmentation is displayed. The legal entities in front and the entities behind are close together due to their physical distance, and are represented in the image as feature data of mutual adhesion due to spatial proximity. Figure 2 The segmentation results were also shown. When contour interruption features and thermal separation boundary points were identified, virtual data segmentation surfaces were inserted into the feature data based on the line connecting the thermal separation boundary points and the location where the contour interruption occurred.

[0095] The originally contiguous feature data was successfully divided into a first priority feature cluster representing the entity in front and a second priority feature cluster representing the trailing person behind, providing an independent data foundation for subsequent identity verification.

[0096] In step S105, it is determined whether to perform anti-tailgating control on the access control device.

[0097] If an illegal accompanying entity is determined to exist in the first priority feature cluster and the second priority feature cluster, an anti-tailgating control command for the access control device is output.

[0098] The biometric information of tailgating individuals is usually not registered in the locally authorized whitelist database, or they are individuals who forcibly enter without independent card-swiping authorization. By sending the isolated data sets into the subsequent feature comparison network for deep feature matching and attendance status verification, it can be determined whether there is any violation.

[0099] By segmenting the system and then conducting independent identity verification and attendance authentication for each segment, it is possible to prevent followers from using the main person's facial features to bypass the system, thus improving the recognition performance of the identity and access control system. If tailing is confirmed, anti-tailgating control commands can be generated to intercept the intruder.

[0100] If both the first priority feature cluster and the second priority feature cluster are determined to be legitimate entities, the anti-tailgating control command for the access control device may not be executed, thus ensuring the passage efficiency of the access control device.

[0101] In one embodiment, if it is determined that there is an illegal accompanying entity in the first priority feature cluster and the second priority feature cluster, an anti-tailgating control command for the access control device is output, including: extracting facial geometric information contained in the first priority feature cluster and the second priority feature cluster respectively, and sending the facial geometric information to the permission module to perform an identity verification procedure; in response to the first priority feature cluster meeting the preset access permission and the second priority feature cluster not containing registration information, an anti-tailgating control command is output to block the access control device's release loop.

[0102] A facial feature point localization model is used to extract the two-dimensional spatial coordinates of key parts such as the corners of the eyes, the tip of the nose, and the corners of the mouth from the pixel regions of the first priority feature cluster, and to construct facial geometric information that reflects the facial structure of the first person in front.

[0103] Referring to the verification steps for the pixel region of the first priority feature cluster segmented to the front, the face extraction and detection steps can be performed on the pixel region of the second priority feature cluster segmented to the rear. Local effective facial geometric information is extracted for subsequent comparison. The two sets of independently extracted facial geometric information are transmitted to the access control module inside the access control through an encrypted onboard data link. The access control module performs information comparison and executes the identity verification procedure.

[0104] If the identity feature vector parsed from the first priority feature cluster matches the employee attendance database in the background, the system will return an "allowed" status, confirming compliance with the preset access permissions and generating an entry attendance record with a timestamp.

[0105] If the identity vector parsed from the second priority feature cluster cannot find a match in the attendance database or the status is marked as a visitor, it can be determined that the second feature cluster does not contain registration information, thus identifying the visitor as a tailgating visitor.

[0106] By using front and rear feature clusters for independent authentication and verification, it can identify unauthorized personnel following authorized employees. The main control processing chip of the attendance machine can send a level transition signal to the motor controller of the gate through the underlying communication bus, and output an anti-tailgating control command to lock the channel motor gear, blocking the access control equipment's release circuit to prevent tailgating behavior.

[0107] In one embodiment, grayscale difference operations can also be performed on continuous video sequence frames in the visible light image to determine the moving target that has generated displacement, extract the geometric contour of the moving target to generate dynamic bounding box coordinates; map the dynamic bounding box coordinates onto the infrared thermal image to establish a local tracking window that follows the movement of the moving target; and determine the infrared boundary contour of the moving target by obtaining the relative temperature fluctuations inside the local tracking window.

[0108] The high-speed front-end interface continuously acquires visible light images and arranges them in chronological order according to the local system clock to form a high-speed continuous video sequence frame. The algorithm extracts the grayscale matrix of the latest Nth frame and subtracts the grayscale matrix of the N-1th frame of the previous cycle to perform grayscale difference operation. In the inter-frame difference operation image, the pixel difference of the static background wall is zero and is canceled out, while the pixel clusters with brightness changes caused by the rapid movement of people are retained. These high-frequency changing pixel clusters represent moving entities and can identify moving targets that have displacement in complex backgrounds.

[0109] By performing mathematical morphological dilation and closing operations on discretely distributed moving pixels, the overall geometric contour of the entity's periphery can be extracted by connecting the tiny gaps within it. By finding the extreme points of this contour in the four directions of up, down, left, and right, the coordinates of the dynamic bounding rectangle that encloses the moving entity can be obtained.

[0110] By projecting the coordinates of the dynamic circumscribed rectangle in reverse space onto the corresponding region in the planar physical coordinate system of the infrared thermal image, a local tracking window that surrounds the target at any time is constructed. By using the method of mapping the circumscribed rectangle of the moving target to the infrared domain in reverse to guide local infrared tracking, interference sources caused by static high-temperature heating equipment around the channel are filtered out, making the infrared analysis process more focused on the human body.

[0111] In one embodiment, determining the infrared boundary contour of a moving target by acquiring the relative temperature fluctuations within a local tracking window includes: using the moving average of the temperature of all pixel grids within the core region of the local tracking window as a reference temperature value to obtain the relative temperature difference between the background pixel grids at the inner edge of the local tracking window and the reference temperature value; performing line-by-line scanning within the local tracking window to obtain the dynamic gradient slope of the relative temperature difference; and determining the abrupt change position of the dynamic gradient slope as the infrared boundary contour point of the moving target.

[0112] The constructed dynamic local tracking window can be shrunk inward toward the geometric center to determine the core area. All infrared temperature measurement data units within the core area are traversed, and the moving average of the temperature data is taken as the dynamic average temperature of the human body surface.

[0113] The surrounding background pixel grid located at the inner edge of the local tracking window but outside the core area is extracted. The initial temperature value of the infrared thermal image is subtracted from the dynamic average temperature of the human body surface to obtain the relative temperature difference reflecting the environmental contrast. Dynamic temperature difference extraction is performed by regional moving average, avoiding the problem that traditional fixed threshold methods are easily affected by changes in ambient temperature.

[0114] Within the tracking window, the relative temperature difference of different grid columns is scanned sequentially from left to right along the horizontal direction. The derivative of the relative temperature difference between two adjacent spatial nodes with respect to their physical location is calculated to obtain the dynamic gradient slope that reflects the drastic temperature fluctuations.

[0115] If the slope of the dynamic gradient at a specific pixel coordinate rises sharply to form a prominent peak, it indicates that the thermal radiation attribute at this location crosses the boundary between the warm skin of the human body and the cold air. The spatial node that produces the dramatic change in the temperature gradient slope can be identified as the infrared boundary contour point that forms the external shape of the target and used as the input feature for subsequent morphological auxiliary positioning.

[0116] Figure 3 This is a schematic diagram illustrating the structure of an adaptive access control system 1000 according to an exemplary embodiment. (Refer to...) Figure 3 The adaptive control system 1000 for access control includes a processor 1100 and a memory 1200. The memory 1200 stores computer program instructions, which, when executed by the processor 1100, implement all or part of the steps of the adaptive control method for access control in this application.

[0117] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

[0118] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. An adaptive control method for access control, characterized in that, include: Acquire an infrared thermal image of the access control area, and locate at least two local thermal extrema points within a preset spatial distance from the infrared thermal image as candidate extrema point pairs; Based on candidate extreme point pairs, the presence of thermal separation boundary points in infrared thermal images is determined by analyzing the temperature concavity features along the path connecting the two candidate extreme points. This includes: constructing a virtual straight line segment connecting the centers of the two extreme points in the candidate extreme point pair within a two-dimensional temperature matrix, and determining the target pixel grid that the virtual straight line segment passes through in the two-dimensional temperature matrix; extracting absolute temperature values ​​one by one along the path of the target pixel grid, and assembling the extracted absolute temperature values ​​into a temperature data sequence according to the spatial arrangement order of the target pixel grid; obtaining the temperature gradient descent slope between adjacent pixel grids in the temperature data sequence, and extracting the minimum point with the lowest temperature value based on the temperature gradient descent slope; obtaining the temperature concavity difference between the absolute temperature values ​​of the two candidate extreme points and the absolute temperature value of the minimum point, and in response to determining that all temperature concavity differences are greater than a preset lower limit, using the spatial coordinates of the minimum point as the thermal separation boundary point; the thermal separation boundary point is used to characterize the physical isolation between two independent heat sources. Acquire a visible light image of the access control area. When it is determined that there is a thermal separation boundary point, map the coordinates of the thermal separation boundary point to the visible light image and determine whether there is a contour interruption feature in the neighborhood of the mapped coordinates to characterize the occlusion of the front entity by the rear entity. In the case where contour interruption features are determined, based on thermal separation boundary points and contour interruption features, feature data that are spatially adjacent and stuck together in visible light images and infrared thermal images are divided into first priority feature clusters and second priority feature clusters. If an illegal accompanying entity is determined to exist in the first priority feature cluster and the second priority feature cluster, an anti-tailgating control command for the access control device is output.

2. The adaptive control method for access control according to claim 1, characterized in that, Local thermal extrema in infrared thermal images are determined in the following way: For any candidate pixel grid in all pixel grids within an infrared thermal image, if the temperature information of the candidate pixel grid is greater than the temperature information of other pixel grids in its neighborhood, the center point of the candidate pixel grid is determined as the local thermal extremum point.

3. The adaptive control method for access control according to claim 1, characterized in that, Locate at least two local thermal extrema points in the infrared thermal image that are spatially separated by a preset interval, as candidate extrema point pairs, including: In the two-dimensional plane coordinate system of infrared thermal image, the straight-line pixel distance between any two local thermal extreme points is determined, and extreme points whose straight-line pixel distance is between a preset lower limit and a preset upper limit are combined to determine candidate extreme point pairs.

4. The adaptive control method for access control according to claim 1, characterized in that, Mapping the coordinates of thermal separation boundary points to a visible light image, determining whether there are contour interruption features in the neighborhood of the mapped coordinates due to rear entities occluding front entities, including: The coordinates of the thermal separation boundary points in the infrared thermal image are mapped to the pixel grid of the visible light image to obtain the corresponding mapped coordinates; The edge analysis window is defined by extending downwards from the origin of the mapped coordinates. Continuous horizontal edge lines are extracted within the edge analysis window and tracked to both sides horizontally. The contour interruption feature is determined by the absolute value change of the vertical gradient along the tracked path.

5. The adaptive control method for access control according to claim 4, characterized in that, Extract continuous horizontal edge lines within the edge analysis window and trace them horizontally to both sides. Determine contour interruption features by analyzing the absolute value change of the vertical gradient along the traced path, including: Edge detection is performed on the pixel grayscale matrix of the visible light image to obtain continuous horizontal edge lines that characterize the features of the human shoulder; the process is then advanced pixel by pixel along the continuous horizontal edge lines to the left and right to extract the absolute value of the vertical gradient along the tracking path. If the horizontal gradient decreases during continuous tracking and the absolute value of the vertical gradient increases within a preset number of adjacent pixels and exceeds a preset abrupt gradient threshold, the lateral connectivity of the continuous horizontal edge line is determined to be blocked, and the x-coordinate of the location where the lateral connectivity is blocked is used as the x-coordinate of the contour interruption feature.

6. The adaptive control method for access control according to claim 1, characterized in that, Based on thermal separation boundary points and contour interruption features, feature data that are spatially adjacent in visible light images and infrared thermal images are segmented into a first-priority feature cluster and a second-priority feature cluster, including: Obtain the absolute value of the coordinate difference between the x-coordinate of the contour interruption feature and the x-coordinate of the thermal separation boundary point. If the absolute value of the coordinate difference is less than the preset tolerance error parameter, confirm that the entity corresponding to the second priority feature cluster has truncated the contour of the entity corresponding to the first priority feature cluster. Using the line connecting the thermal separation boundary point and the location where the contour is interrupted as a reference, a data segmentation surface is inserted into the feature data. The feature data is then divided into a first priority feature cluster and a second priority feature cluster through the data segmentation surface.

7. The adaptive control method for access control according to claim 1, characterized in that, If an illegal accompanying entity is determined to exist in the first priority feature cluster and the second priority feature cluster, an anti-tailgating control command for the access control device is output, including: The facial geometric information contained in the first priority feature cluster and the second priority feature cluster is extracted respectively, and the facial geometric information is sent to the permission module to execute the identity verification procedure. In response to the first priority feature cluster meeting the preset access permissions and the second priority feature cluster not containing registration information, an anti-tailgating control command is output to block the access control device's release loop.

8. An adaptive control system for access control, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement the adaptive control method for access control according to any one of claims 1-7.

Images