Intelligent access control and situation management system based on integrated cognition

Abstract

According to one embodiment of the present invention, a detection unit includes at least one sensor installed at a plurality of detection points along the movement path of an object, and the at least one sensor can acquire detection data regarding an object within a surveillance area. A data receiving unit receives the detection data and can extract at least one piece of information among the object's face information, appearance information, movement path information, and biometric information. A complex perception and situation judgment unit processes the extracted information using a pre-trained deep learning module to generate identification information and movement history information of the object, and can determine whether an abnormal situation exists by comparing it with pre-defined judgment conditions. A control unit can output a voice or visual warning, control an access device, or communicate with a remote control system depending on the judgment result.

Description

Technology Field

[0001] The present invention relates to the field of image recognition and access control technology, and more specifically, to a system capable of detecting objects within a surveillance area, controlling access, and managing abnormal situations in real time by utilizing artificial intelligence-based identification and situation judgment functions. Background Technology

[0002] The scope of application for AI-based video recognition and access control technology is gradually expanding not only to surveillance systems, unmanned stores, and smart homes, but also to multi-use facilities such as accommodations.

[0003] In particular, recent image recognition technology has achieved a level of precision capable of comprehensively analyzing various information such as face, age, appearance, and movement patterns. As a result, its importance is becoming increasingly prominent in environments where the accuracy of identifying entrants and real-time situational judgment are critical.

[0004] The lodging industry is also actively attempting to adopt this technology amidst a trend toward automation, including the minimization of operational personnel, nighttime unmanned operations, and remote monitoring.

[0005] However, existing accommodation access systems rely on methods such as card keys, passwords, and reservation information authentication, which limits the ability to accurately verify the identity of actual users or check entry conditions.

[0006] Particularly in situations where legal regulations for youth protection apply, existing systems alone are insufficient to effectively detect or control abnormal behaviors such as cohabitation or disguised entry by minors. Furthermore, the passive structure requiring administrators to manually identify entrants is vulnerable to various issues, including missed monitoring, personnel fatigue, and difficulties in control due to dispersed entry routes, all of which result in persistent legal risks.

[0007] Against this backdrop, there is a growing need for intelligent access control systems capable of precisely analyzing the identity and movement characteristics of entrants and responding to situations within facilities in real time based on this analysis. The problem to be solved

[0008] The main problem that the present invention aims to solve is to provide an intelligent system capable of reliably controlling and managing access to a facility even in an unmanned environment, by extracting at least one of facial information, appearance information, movement path information, and biometric information based on object detection data collected from multiple detection points, and by comparing and analyzing this with identification information and movement history information to automatically determine abnormal situations.

[0009] Another challenge is to implement an automated security management structure capable of real-time response to entrants without surveillance blind spots by performing various response functions, such as voice guidance, visual warnings, automatic control of access devices, and remote communication, based on the results of such identification and situational assessment. means of solving the problem

[0010] According to one embodiment of the present invention, a detection unit includes at least one sensor installed at a plurality of detection points along the movement path of an object, and the at least one sensor can acquire detection data regarding an object within a surveillance area. A data receiving unit receives the detection data and can extract at least one piece of information among the object's face information, appearance information, movement path information, and biometric information. A complex perception and situation judgment unit processes the extracted information using a pre-trained deep learning module to generate identification information and movement history information of the object, and can determine whether an abnormal situation exists by comparing it with pre-defined judgment conditions. A control unit can output a voice or visual warning, control an access device, or communicate with a remote control system depending on the judgment result.

[0011] According to one embodiment of the present invention, a data receiving unit divides the detection data into frames and processes each divided frame in parallel through a plurality of recognition modules to extract at least one of the object's face information, appearance information, movement path information, and biometric information.

[0012] According to one embodiment of the present invention, the data receiving unit divides the detected data into frames and sequentially processes the divided frames through a single recognition module to integrally extract at least one of the object's face information, appearance information, movement path information, and biometric information.

[0013] According to one embodiment of the present invention, the complex perception and situation determination unit can utilize a deep learning module to determine an abnormal situation when the identification information of an object does not match the identification information registered in advance, or when the movement history information differs from the normal movement history information defined in advance.

[0014] According to one embodiment of the present invention, when an abnormal situation is determined, the control unit may perform the operation of providing a voice or visual warning through an output device placed within a monitoring area, controlling an access device to restrict the entry and exit of an object, and transmitting information regarding the abnormal situation to a remote control system.

[0015] According to one embodiment of the present invention, the method may include the steps of: acquiring detection data of an object within a surveillance area by a detection unit comprising at least one sensor; receiving the detection data by a data receiving unit and extracting at least one of the following: face information, appearance information, movement path information, and biometric information of the object; processing the information by a complex perception and situation judgment unit using a pre-learned deep learning module to generate identification information and movement history information of the object, and determining whether there is an abnormal situation by comparing it with a pre-defined judgment condition; and outputting a voice or visual warning, controlling an access device, or communicating with a remote control system according to the judgment result by a control unit.

[0016] According to one embodiment of the present invention, a computer-readable recording medium may be included on which a program for executing a complex cognitive-based intelligent access control and a method for operating the management system is recorded. Effects of the invention

[0017] According to the present invention, the accuracy of identification of persons entering and exiting a facility and the reliability of abnormal situation detection can be significantly improved through complex recognition and situation judgment utilizing at least one of an object's face information, appearance information, movement path information, and biometric information.

[0018] Furthermore, since the entire process from detection to identification, judgment, and control is automated, real-time response is possible without the intervention of operational personnel, ensuring the stability and efficiency of security management even in unmanned environments.

[0019] Furthermore, by diversifying object recognition methods and systematizing judgment criteria, the possibility of monitoring omissions or judgment errors can be reduced. Additionally, since access can be effectively controlled even in situations where legal regulations, such as age restrictions, apply, it provides tangible benefits in terms of legal compliance and safety management. Brief explanation of the drawing

[0020] FIG. 1 is an overall configuration block diagram of a complex cognitive-based intelligent access control and situation management system according to one embodiment of the present disclosure. FIG. 2 is a diagram illustrating a process of extracting face information, appearance information, movement path information and biometric information by processing object detection data according to one embodiment of the present disclosure. FIG. 3 is a diagram illustrating a complex perception and situation judgment process according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating the flow of a control operation performed by a control unit according to the result of determining an abnormal situation according to one embodiment of the present disclosure. FIG. 5 is a drawing illustrating an example of the configuration of a monitoring area within a facility where a system according to one embodiment of the present disclosure is installed. Specific details for implementing the invention

[0021] Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0022] To clearly explain the present invention, parts unrelated to the explanation have been omitted, and the same reference numerals are assigned to identical or similar components throughout the specification. Accordingly, the reference numerals described above may also be used in other drawings.

[0023] Furthermore, the size and thickness of each component shown in the drawings are depicted arbitrarily for convenience of explanation, and thus the present invention is not necessarily limited to what is illustrated. Thickness may be exaggerated in the drawings to clearly represent various layers and regions.

[0024] Furthermore, the expression "identical" in the explanation may mean "substantially identical." In other words, it may be an identicality to the extent that a person with ordinary knowledge would accept it as identical. Other expressions may also be those in which "substantially" has been omitted.

[0025] Furthermore, when a description states that a part 'includes' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0026] As used in this specification, '~part' refers to a unit that processes at least one function or operation, and may mean, for example, software, FPGA, or hardware components. The function provided by the '~part' may be performed separately by a plurality of components or may be integrated with other additional components.

[0027] The term '~part' in this specification is not necessarily limited to software or hardware, and may be configured to reside in an addressable storage medium or configured to operate one or more processors. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0029] FIG. 1 is an overall configuration block diagram of a complex cognitive-based intelligent access control and situation management system according to one embodiment of the present disclosure.

[0030] Referring to FIG. 1, the complex cognitive-based intelligent access control and situation management system (1) according to the present embodiment may be composed of a detection unit (100), a data receiving unit (200), a complex cognitive and situation judgment unit (300), and a control unit (400).

[0031] Each component (100, 200, 300, 400) can collect detection data of an object at multiple detection points installed along the object's entry and exit path, generate identification information and movement history information based on this, and determine whether there is an abnormal situation and perform a corresponding action.

[0032] The above components (100, 200, 300, 400) may each be formed as independent physical modules, and according to one embodiment, may be implemented as a single device with some functions integrated. In addition, each component (100, 200, 300, 400) may be interconnected via a wired or wireless communication network and may include cloud-based computing resources if necessary.

[0033] Meanwhile, the complex cognitive-based intelligent access control and situation management system (1) according to the present embodiment may omit or add some of the components shown in FIG. 1, which means that various modifications and changes are possible within the scope of not departing from the technical concept of the present invention.

[0034] The detection unit (100) may include a plurality of sensors (e.g., image sensors) installed at a plurality of detection points along the movement path of an object.

[0035] The detection unit (100) can adjust the spacing between image sensors according to the regional distribution of the monitoring area, and multiple sensors can operate independently of each other and provide overlapping data for the same object.

[0036] In addition to the image sensor, the detection unit (100) may additionally include at least one of an acoustic sensor for detecting abnormal sounds such as screams or breaking sounds, a thermal sensor for detecting the body temperature distribution of a subject, an RF (Radio Frequency) signal detector for detecting radio signals of a smart device, and a LiDAR sensor for acquiring three-dimensional spatial information and distance information. Accordingly, the data receiving unit (200) can improve the precision and reliability of situation awareness and judgment by integrating and analyzing acoustic data, thermal distribution data, RF signal data, or point cloud data collected from each of the sensors with image data.

[0037] For clarity of explanation, the configuration and operation of the present invention will be described exemplarily below, focusing on the image sensor.

[0038] Furthermore, while the present invention is described with reference to embodiments applied to accommodation facilities, its scope of application is not limited thereto. It goes without saying that the present invention is generally applicable to all multi-use facilities or restricted areas where security and access control are essential, such as unmanned stores, shared offices, data centers, medical institutions, educational institutions, controlled areas, and control facilities, where access control is required based on reservation information, registered employee information, or visitor information.

[0039] The image sensor included in the detection unit (100) can acquire detection data for objects within the surveillance area. The image sensor of the detection unit (100) may be installed at key locations such as the entrance, lobby, in front of the elevator, corridors on each floor, and guest room entrances of the accommodation facility, but the present invention is not limited thereto.

[0040] Each image sensor may be configured as a fixed or rotating camera, and, if necessary, may additionally include an infrared sensor (IR sensor) capable of capturing images in nighttime environments or a depth camera for recognizing the distance and stereoscopic information of objects.

[0041] The image sensor of the detection unit (100) can transmit detection data to the data receiving unit (200) via a network-based or wired communication module. The detection unit (100) may include a local cache memory and a compression engine to maintain the quality of the detection data and transmit it in real time.

[0042] The detection unit (100) can be classified as a ceiling type, a wall type, or a stand type depending on the installation environment, and can be positioned to minimize the overlap of the field of view between cameras. The detection unit (100) can be linked with a GPS module or an internal coordinate-based location tag system to merge the location information of an object.

[0043] The detection unit (100) may include a time-based control module to automatically adjust the shooting frequency and resolution according to specific time periods or conditions. The detection unit (100) may also detect abnormal movement patterns or long-term stays of a stationary object.

[0044] The data receiving unit (200) receives detection data received from the detection unit (100) and can extract at least one piece of information among the object's face information, appearance information, movement path information, and biometric information.

[0045] The data receiving unit (200) can divide the input detection data into frames and analyze each frame through a recognition module to obtain the information.

[0046] The data receiving unit (200) can process frames in parallel through a plurality of recognition modules or process frames sequentially through a single recognition module.

[0047] The parallel method can rapidly analyze information elements such as face, appearance, and movements through separate paths, while the sequential method can be advantageous for analyzing behavioral patterns over time.

[0048] The data receiving unit (200) may include an image preprocessing module, a noise removal module, a feature extraction module, and a metadata generation module, and the modules may be implemented as FPGA or GPU-based hardware.

[0049] The data receiving unit (200) can assign a reliability score to each information item and transmit it to the complex recognition and situation judgment unit (300).

[0050] The data receiving unit (200) can separate multiple objects within a frame, assign a tracking ID, and enable tracking of information for each object. The data receiving unit (200) can temporarily store raw detection data received from the detection unit (100) in a local storage device and use it as backup data for subsequent analysis.

[0051] The data receiving unit (200) may include a data normalization engine that converts detection data of various formats into a standardized internal format. The data receiving unit (200) may perform a timeline mapping function that can tag information such as the direction of movement, speed, and dwell time of an object in chronological order.

[0052] The complex recognition and situation judgment unit (300) can generate object identification information and movement history information by processing information extracted through the data receiving unit (200) with a pre-trained deep learning module.

[0053] The complex recognition and situation judgment unit (300) can determine an abnormal situation when the identification information does not match the previously registered identification information or when the movement history information differs from the previously defined normal movement history information.

[0054] Specifically, the complex cognition and situation judgment unit (300) can embed multiple deep learning models capable of performing object age estimation, companion composition judgment, entry order and path analysis, etc.

[0055] The complex perception and situation judgment unit (300) may include an LSTM-based algorithm that processes time series information and can analyze the continuity and consistency of a long-term movement path.

[0056] The complex cognitive and situation judgment unit (300) is linked with a condition database that stores situation judgment criteria and can be compared with various judgment rules such as entry time conditions, age conditions, and entry order conditions.

[0057] The complex perception and situation judgment unit (300) can reconstruct the entire movement path and behavior history of an object by integrating information obtained from multiple locations within the surveillance area. The complex perception and situation judgment unit (300) can reduce the probability of misrecognition by comprehensively analyzing walking speed, body shape, clothing color, etc., for distinguishing between similar objects.

[0058] The complex perception and situation judgment unit (300) may include a risk level scoring function for visually representing abnormal signs for each object. The complex perception and situation judgment unit (300) may database the situation judgment results to form a feedback loop structure capable of continuous learning and improvement.

[0059] The control unit (400) can output a voice or visual warning, control an access device, or communicate with a remote control system based on the judgment result of the complex perception and situation judgment unit (300).

[0060] The control unit (400) may provide a voice or visual warning through an output device placed within the monitoring area when an abnormal situation is determined. Additionally, the control unit (400) may control access devices such as electronic door locks, gate openers, and elevator call restrictors to restrict the entry and exit of objects.

[0061] The control unit (400) can transmit report data including identification information, time, location, reason for situation determination, etc. regarding abnormal situations to a remote control system through a communication module.

[0062] The control unit (400) may include a control policy engine capable of selectively activating a warning means, an access blocking means, and a log recording means according to a judgment result. The control unit (400) may include a setting function to apply different response methods and control means to each component according to a security level specified by the user.

[0063] The control unit (400) can automatically activate detection data backup when an abnormal situation occurs and send real-time notifications to control personnel. The control unit (400) may include a dual authentication structure to enable forced release or manual intervention through administrator authentication in case of an emergency.

[0065] FIG. 2 is a diagram illustrating a process of extracting face information, appearance information, movement path information and biometric information by processing object detection data according to one embodiment of the present disclosure.

[0066] As illustrated in FIG. 2, raw detection data obtained from the detection unit (100) is input to the data receiving unit (200), and then a preprocessing step in which it is divided into frames can be performed first.

[0067] During the preprocessing stage, basic image quality enhancement tasks such as noise removal, brightness and contrast adjustment, resolution correction, and image stabilization can be performed on each frame. If objects within the image are blurry or lighting is uneven, Gaussian filtering or histogram equalization techniques may be applied to ensure normalized image quality.

[0068] The preprocessed video frame can then be branched into multiple recognition modules (201, 202, 203, 204, 205) to perform specialized analysis on different information items. The branched paths may include a face detection module (201), an appearance analysis module (202), a movement tracking module (203), a biosignal analysis module (204), a metadata generation module (205), etc.

[0069] For example, the face detection module (201) can detect feature points such as eyes, nose, and mouth, as well as facial contours, using a CNN-based face recognition algorithm, and identify the face of an object based on this. The face detection module (201) can estimate the age range, gender, and whether glasses are worn by the object, and can be used as basic information for determining whether the object is a minor.

[0070] In addition, multi-frame recognition considering facial orientation and changes in expression can be performed by integrating and analyzing frames captured from multiple angles. When an object moves while rotating from the front to the side, recognition accuracy can be ensured by extracting a normalized image aligned with the facial rotation angle at each viewpoint.

[0071] The appearance analysis module (202) can analyze visual characteristics such as the height, body shape, clothing color, length of upper and lower garments, and presence or absence of personal belongings of an object, and can be used as a reference value for re-identification of the same person.

[0072] For example, if the same person reappears at a different time or location, identification accuracy can be improved by comparing with appearance analysis information. Appearance information is corrected based on angle, lighting conditions, and background elements, then converted into a feature vector that can be used for detection by comparing it with an internal database. At this time, color analysis is standardized based on the HSV color space, and a clothing pattern recognition algorithm may be additionally applied.

[0073] The movement tracking module (203) calculates the movement path of an object from the position change between frames and can extract data such as the direction of movement, average speed, time of stay, and whether the path is repeated.

[0074] Movement data can reconstruct behavioral paths based on the sequence between entry and exit locations and compare them with normal entry routes. It can be used as a warning criterion in cases of excessive dwell time, entry via abnormal routes, or repeated movement between entrances. For example, behaviors such as lingering in front of an elevator for a long time or returning after passing an entrance can be tagged as abnormal signs.

[0075] The biosignal analysis module (204) can analyze the object's heart rate, body temperature, respiratory cycle, etc. based on data obtained through a depth camera or infrared sensor.

[0076] For example, biological changes such as an abnormal rise in body temperature or irregular breathing may be judged as signs of a state of tension or an emergency situation and transmitted to a complex perception and situation judgment unit (300). The biological information is analyzed as a graph based on a time axis, and can be tagged as an abnormal pattern if it exceeds a predefined range. In addition, simple temporary changes and persistent abnormal states can be distinguished by analyzing the intercorrelation between biological signals.

[0077] Each information item can be converted into a structured data structure and stored through a metadata generation module (205), and subsequently transmitted to a complex perception and situation judgment unit (300) to be used as basic information for object identification and abnormal situation judgment. The metadata may include object ID, extraction time, location information, reliability score, etc., and can be used for log recording and visualization configuration later. The metadata is composed of a structured format such as JSON or XML, and event-oriented real-time log-based stream transmission may also be possible.

[0078] Meanwhile, FIG. 2 illustrates an example in which the aforementioned modules (201, 202, 203, 204, 205) are separated to process each information item in parallel, but the present invention is not limited thereto.

[0079] That is, the above detection data can also be implemented by dividing it into frames and sequentially processing the frames through a single recognition module to integrally extract at least one of the object's face information, appearance information, movement path information, and biometric information. This sequential analysis structure is applicable even in environments with limited computational resources and can be advantageous for integrally interpreting changes at different time points within each frame.

[0081] FIG. 3 is a diagram illustrating a complex perception and situation judgment process according to one embodiment of the present disclosure.

[0082] Referring to FIG. 3, the complex recognition and situation judgment unit (300) receives at least one of the following data received from the data receiving unit (200): face information, appearance information, movement path information, and biometric information. By integrating and analyzing the data, it generates identification information and movement history information of an object, and can determine whether there is an abnormal situation by comparing it with a predefined judgment condition.

[0083] The complex cognition and situation judgment unit (300) may include a plurality of deep learning modules, and each deep learning module may be independently designed with a model structure specialized for a specific information item, but the present invention is not limited thereto.

[0084] That is, the complex cognition and situation judgment unit (300) may be composed of a single integrated deep learning module. Hereinafter, it will be described assuming that the complex cognition and situation judgment unit (300) includes multiple deep learning modules.

[0085] The deep learning module included in the complex cognition and situation judgment unit (300) can be subdivided as follows.

[0086] The deep learning module (301) for face recognition can, for example, adopt a ResNet-based CNN structure to extract unique feature vectors from the face information of an object and then calculate the similarity with pre-registered identification information.

[0087] The external appearance analysis deep learning module (302) can, for example, extract external appearance information of an object, such as clothing color, body shape, and presence or absence of belongings, through a neural network of the Vision Transformer family, and combine this with time series data to be used for object tracking.

[0088] The movement path information analysis deep learning module (303) can generate structured movement history information by analyzing the movement direction, dwell time, and interval between locations over time, for example, by utilizing a model based on LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit) based on the change in the position of an object.

[0089] The bio-information analysis deep learning module (304) utilizes infrared camera and depth image-based inputs, and can extract body temperature, heart rate, respiratory cycle, etc. and determine abnormal signs through a CNN-based thermal detection and time-based signal analysis algorithm.

[0090] The complex cognition and situation judgment unit (300) may include a rule judgment engine that collects information from each of the above deep learning modules (301, 302, 303, 304) in an integrated manner and compares it with a predefined judgment condition. The engine may be implemented not only with explicit rules based on IF-THEN, but also with a learned rule-based model based on a Decision Tree or Rule, and judgment under complex conditions is possible.

[0091] For example, the following abnormal situations can be identified.

[0092] (Example 1) If the object's facial information does not match the registered identification information, the age estimation result is under 19 years, and the object moves alone along the entry path without a companion, it may be determined as "suspected entry of a minor alone."

[0093] (Example 2) If an object that is not a registered guest operates the access device in the elevator area, and at that time the existing guest is not at the front desk or in the guest room, it may be determined as an "attempted unauthorized access."

[0094] (Example 3) If the facial information matches but the movement path information follows a path different from the predefined normal movement history (e.g., bypassing emergency stairs, unauthorized passage through the front), it can be determined as "abnormal movement path."

[0095] (Example 4) If the same object repeatedly attempts to enter or exit during a specific time period (e.g., 2:00 AM to 4:00 AM) or wanders around a limited area for a long time, it can be classified as "abnormal dwelling behavior."

[0096] (Example 5) To identify whether objects are linked, the similarity of facial information, redundancy of appearance information, movement paths, and time intervals of two objects are analyzed to determine if they are a 'group of linked objects'. If one of the objects is determined to be a minor, it can be determined as a 'group suspected of distributed infiltration'.

[0097] The complex perception and situation judgment unit (300) can calculate a risk score based on the results of such complex analysis. The risk score can be calculated as a weighted average by quantifying indicators such as facial information reliability, appearance information consistency, movement path consistency, time zone weighting, and connectivity between objects.

[0098] For example, if an object presumed to be a minor attempts to enter a room during the early morning hours by following the same entry route as an adult, the risk score is determined to be above a threshold, and an automatic sanction process may be activated.

[0099] The complex perception and situation judgment unit (300) can achieve continuous performance improvement based on learning. The semi-supervised learning method can lower the false positive rate of the algorithm by re-incorporating cases identified as abnormal situations into the training data after review by the manager.

[0100] The reinforcement learning-based structure modifies weights based on the frequency of security events and the effectiveness of follow-up actions, and can gradually improve the accuracy of recurring pattern recognition.

[0101] Verification of the consistency of the object's identification information may also be one of the main functions of the complex recognition and situation judgment unit (300).

[0102] If the same object reappears with a different appearance after a certain period of time, the possibility of disguise is evaluated by analyzing the changes in appearance and facial information; if identification reliability drops sharply, it can be determined as an "attempted entry under disguise." Even minor changes in appearance, such as wearing glasses, a hat, hair length, or a hood, are evaluated through vector distance-based quantitative analysis.

[0103] Additionally, the complex perception and situation judgment unit (300) can identify interaction patterns between objects. If a pattern is detected where two objects repeatedly pass through the same area at similar times or move simultaneously and enter and exit a room, they can be classified as a 'linked entry group' through a clustering algorithm.

[0104] This method enables early detection of group entry involving minors or entry disguised as a guardian, and allows for the gradual upward adjustment of the warning level if repeated more than a certain number of times.

[0105] Additionally, the complex perception and situation judgment unit (300) can combine and analyze complex conditions such as time difference movement between existing guests and unauthorized objects, separation of entry and exit paths, and discrepancy in external information.

[0106] This enables situational context analysis based on multiple features, going beyond simple rule-based determination, and allows for active responses to various scenarios such as dispersed entry of minors, entry bypassing the front desk, and retries after repeated rejections.

[0107] The complex recognition and situation judgment unit (300) automatically logs the event when an abnormal situation occurs and transmits it in real time to a visualization UI so that an administrator or remote monitoring personnel can check it immediately.

[0108] In this process, the reasoning for the judgment, participating objects, and time and location information are attached as metadata and can be used as base data for the future generation of audit logs and reports.

[0110] FIG. 4 is a diagram illustrating the flow of a control operation performed by a control unit according to the result of determining an abnormal situation according to one embodiment of the present disclosure.

[0111] Referring to FIG. 4, in a complex cognitive-based intelligent access control and situation management system (1) according to one embodiment of the present invention, the control unit (400) can control the access of an object in real time by sequentially performing a plurality of control procedures based on an abnormal situation judgment result received from the complex cognitive and situation judgment unit (300).

[0112] The control unit (400) can comprehensively analyze the type of abnormal situation judgment information, risk score, location information of objects, etc., and perform multi-stage response scenarios such as voice and visual warnings, physical access restrictions, video backup, remote reporting, and requests for administrator intervention according to a predefined control policy.

[0113] The control unit (400) receives a judgment result packet transmitted from the complex perception and situation judgment unit (300) and can analyze the risk score included in the result.

[0114] The risk score is calculated based on the analysis of the object's face information, appearance information, movement path information, and biometric information, and can be expressed as a quantified value. If this score is greater than or equal to a preset threshold, the control unit (400) determines that an abnormal situation has occurred with respect to the object and can initiate an immediate control sequence.

[0115] The control unit (400) can search for the output device closest to the location based on the current location of the object and output an automatic warning message through the device.

[0116] The output device may be, for example, a digital signage at the front, an elevator interior display, a speaker mounted on the hallway ceiling, a floor warning panel, a warning light on the wall of the guest room entrance, etc. The control unit (400) may selectively transmit a customized message according to the object's current behavior scenario.

[0117] For example, voice or text warnings such as "Customer who has not checked in, please proceed to the information desk," "Minors are not permitted to stay," or "Access to the room is restricted. Please contact the manager" can be immediately output to enable the object to recognize them.

[0118] The message is automatically converted into multiple languages ​​to be applicable to foreign customers, and may also include voice guidance for the visually impaired and flashing LED warnings for the hearing impaired.

[0119] The control unit (400) can perform access restriction measures. This is a method of blocking further movement of an object by controlling access control devices linked to the system (1).

[0120] Specifically, this can be done by maintaining the locked state of the smart door lock, nullifying elevator calls and blocking movement between floors, or disabling the authentication input window in front of the guest room door.

[0121] For example, for an object that calls the elevator without going through the front procedure, physical access itself can be restricted by ignoring the operation of the call button or blocking the door opening signal.

[0122] The control unit (400) can automatically record detection data immediately after an abnormal situation occurs in conjunction with a video backup module to a storage. The backup range may be a period of time before and after the abnormality detection point (e.g., 30 seconds before to 90 seconds after), and the data can be transmitted to a separate local storage or cloud server in an encrypted state and used as evidence.

[0123] The detection data includes time tags, object identification numbers, and related judgment result metadata, enabling quick identification during future queries.

[0124] The control unit (400) can transmit the execution history and results of the above control operation to a remote monitoring system in real time. The transmitted data may include items such as object identification information, determined abnormal situation type, risk score, time and location of occurrence, performed control measures, and control results, and encrypted communication may be applied to ensure security.

[0125] The control unit (400) can also support interaction with a remote manager. The manager can access a dedicated web-based control dashboard or mobile application from outside the hotel, for example, through an integrated monitoring center or a mobile device, to check real-time alarm history and related videos and perform manual control intervention.

[0126] For example, an administrator can receive an immediate notification upon the occurrence of an abnormal situation, review the video clip displayed on the screen, and select options such as 'Allow entry,' 'Block entry,' or 'Repeat voice warning.' This interface can serve as a core technology that enables security response with minimal personnel in unmanned front-end operation environments.

[0127] The control unit (400) may include a control policy engine that can preconfigure situational control priority, response method, control target device, etc.

[0128] Administrators can freely configure response scenarios for each type of anomaly through the web-based administrator console and create sophisticated response rules by combining various logical conditions, such as time zone conditions, location conditions, and warning repetition count conditions.

[0129] For example, you can define advanced policies such as "immediately restrict entry without a voice warning in the case of 'abnormal movement' detected during the early morning hours (00:00–05:00)."

[0130] In this way, the control unit (400) can control the movement of objects in real time based on the analysis results of the complex perception and situation judgment unit (300), and by executing a series of automated control procedures that enable rapid security response even in an unmanned environment, the security level and operational efficiency of the accommodation facility can be significantly improved.

[0132] FIG. 5 is a drawing illustrating an example of the configuration of a monitoring area within a facility where a system according to one embodiment of the present disclosure is installed.

[0133] Referring to FIG. 5, a complex cognitive-based intelligent access control and situation management system (1) according to one embodiment of the present invention may be installed in a plurality of surveillance zones within a building, and each surveillance zone may be configured to include the main movement path and dwelling point of an object.

[0134] As illustrated in FIG. 5, the system (1) may install a plurality of detection units (100) and terminals corresponding to a monitoring target space consisting of an external entrance / exit of a building, a front desk area, an elevator boarding / alighting point, a floor corridor, a guest room entrance, an emergency stair entrance, etc., and may deploy a hardware infrastructure capable of collecting and outputting detection data at each point.

[0135] A fixed or rotating high-resolution camera is installed at the external entrance (510), and can be configured to stably collect facial information, appearance information, clothing, direction of movement, etc., at the moment an object first enters, in a fixed lighting environment.

[0136] An IR sensor or a thermal imaging camera may be installed at the external entrance (510) to enable accurate recognition even at night or in low-light environments. Additionally, precise timing-based data collection of the time of object entry is possible through an object detection sensor linked to the automatic door.

[0137] In the front area (520), by installing a terminal with an upward-facing camera along with a stand-type display or kiosk placed in the center of the lobby, it can be designed to naturally recognize facial information while performing non-face-to-face check-in or customer service, and to collect various biometric-based information such as age estimation, gender estimation, speech tone and response analysis.

[0138] The kiosk can be additionally equipped with a biometric recognition module (fingerprint, iris, etc.) along with a voice guidance function, and the reservation authentication and access verification procedures are automated.

[0139] The elevator area (530) is one of the key points for access control, and a video sensor of the detection unit (100) is installed above the elevator call button, and at the same time, a separate camera is also embedded in the ceiling inside the elevator to check the movement path and companion information during the elevator ride.

[0140] For example, if a minor enters along the same corridor and gets off at the same floor within 30 seconds immediately after an adult who checked in first uses the elevator, the system (1) may determine this as a 'dispersed infiltration attempt'. The elevator is linked with the control unit (400) and can disable the call button or block the opening of the door for unauthenticated objects.

[0141] In the floor corridor (540), multiple wall-mounted sensors are installed at regular intervals to identify connections between zones, and continuous tracking can be performed to recognize the movement path and dwell time of an object, abnormal patterns of movement path, etc.

[0142] At the end point of each corridor (540), a heat detector or distance sensor may be additionally installed to automatically identify an object that has stopped moving or to detect a long stay. Additionally, if an object repeatedly circulates on the same floor or stays in front of a room door for a long time, a warning message may be sent through the warning panel in the area or a procedure to call a controller may be automatically executed.

[0143] A tablet or smart lock for access authentication may be installed in the room area (550), and video recording may be automatically initiated via a proximity sensor or weight detection sensor when an object approaches.

[0144] At this time, the system (1) can compare the facial information of the registered guest with real-time video information and perform an automatic voice warning or block entry in case of abnormal access.

[0145] For example, if an individual presumed to be young attempts to enter a guest room without an adult guest present, the device can play a message such as "Access to this room is restricted to registered guests only." The device is equipped with an emergency call button and a QR code scanning function, allowing for immediate notification to be sent to an administrator or guardian.

[0146] Since the emergency stair entrance (560) is typically an area where camera blind spots are likely to form, by installing a depth camera or a 360-degree rotating device of the detection unit (100), unauthorized access through a bypass route can be monitored in real time.

[0147] According to one embodiment of the present invention, if multiple objects approaching a specific floor via emergency stairs without using an elevator are detected, it can be automatically reported to the manager.

[0148] In addition, location and status information of objects in all monitoring zones is synchronized and transmitted to the central control unit (400) and the complex perception and situation judgment unit (300), and a real-time situation judgment and control sequence can be performed based on this information. At this time, each detection point is equipped with an independent module to perform local judgment first and transmit to the central complex perception and situation judgment unit (300) according to importance.

[0149] The configuration illustrated in Fig. 5 is designed to enable real-time response even in an unmanned operating environment, and can provide the remote manager with real-time object identification results, access control status, whether warning messages are sent, and whether video backups are performed in conjunction with a remote control system.

[0150] For example, an integrated control center located in Seoul can detect and control the situation of accommodation facilities on Jeju Island in real time. Remote operators can receive notifications via mobile apps, web platforms, or smartwatch-based systems and determine whether to prioritize intervention based on the situation.

[0151] The example of the surveillance zone configuration shown in Fig. 5 is merely one embodiment, and when applying the actual system, the location and equipment specifications of each surveillance zone can be adjusted by considering the structure, number of floors, security level, interior environment, etc. of the accommodation facility.

[0152] For example, in the case of luxury hotels, additional ceiling-mounted cameras can be installed in every hallway, while smaller facilities such as motels may maintain a minimal configuration centered around guest room entrances. Additionally, features such as a dedicated night shooting mode, enhanced surveillance mode for specific time periods, and congestion-based surveillance priority adjustment can be applied according to user settings.

[0153] As such, the surveillance zone configuration according to Fig. 5 is designed to enable linked monitoring of multiple points along the main movement path of an object, and can detect irregular movement, unauthorized entry, and linked behavior of the object in an advanced manner, and based on this, can determine abnormal situations and perform automatic control.

[0155] The drawings and detailed description of the invention referenced so far are merely exemplary of the invention and are used only for the purpose of explaining the invention, not to limit the meaning or the scope of the invention as defined in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the invention should be determined by the technical spirit of the appended claims.

[0156] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a Digital Signal Processor (DSP), a microcomputer, a Field Programmable Gate Array (FPGA), a Programmable Logic Unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions.

[0157] The processing unit may execute an operating system and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, the processing unit may be described as being used as a single unit, but a person of ordinary skill in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements.

[0158] For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as a parallel processor, are also possible. Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure the processing unit to operate as desired or command the processing unit independently or collectively.

[0159] Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by a processing device or to provide instructions or data to a processing device. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0160] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software.

[0161] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiments, and vice versa.

[0162] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims are also included within the scope of the claims set forth below. Explanation of the symbols

[0163] 1: Complex Cognition-Based Intelligent Access Control and Situation Management System 100: Detector 200 : Data receiving unit 300: Complex Cognition and Situation Judgment Unit 400 : Control unit

Claims

Claim 1 It includes at least one sensor installed at multiple detection points along the entry and exit path of an object, wherein the at least one sensor is a detection unit that acquires detection data of the object within a surveillance area; a data receiving unit that receives the detection data and extracts face information, appearance information, movement path information, and biometric information of the object; and a complex perception and situation judgment unit that processes the information using a pre-trained deep learning module to generate identification information and movement history information of the object, and determines whether an abnormal situation exists by comparing it with pre-defined judgment conditions. and a control unit that outputs a voice or visual warning, controls an access device, or performs communication with a remote control system according to the above judgment result; wherein the data receiving unit includes a face detection module, an appearance analysis module, a movement tracking module, and a biometric analysis module; wherein the data receiving unit divides the detection data into frames and inputs the detection data divided into frames into the face detection module, the appearance analysis module, the movement tracking module, and the biometric analysis module in parallel to simultaneously extract the face information, the appearance information, the movement path information, and the biometric information, or inputs them sequentially to integrally extract the face information, the appearance information, the movement path information, and the biometric information; wherein the complex perception and situation judgment unit includes a deep learning module for face recognition, a deep learning module for appearance analysis, a deep learning module for movement path information analysis, and a deep learning module for biometric information analysis; wherein the deep learning module for face recognition, the deep learning module for appearance analysis, the deep learning module for movement path information analysis, and the deep learning module for biometric information analysis utilize the extracted face information, the appearance information, A complex cognitive-based intelligent access control and situation management system that receives the above movement path information and the above biometric information in parallel and determines whether there is an abnormal situation. Claim 2 delete Claim 3 delete Claim 4 In claim 1, the complex cognition and situation determination unit utilizes the deep learning module to determine as an abnormal situation cases where the identification information does not match the previously registered identification information or the movement history information differs from the previously defined normal movement history information, in a complex cognition-based intelligent access control and situation management system. Claim 5 In claim 4, the control unit performs the operation of providing a voice or visual warning through an output device placed within the monitoring area when the abnormal situation is determined, controlling the access device to restrict the entry and exit of the object, and transmitting information regarding the abnormal situation to a remote control system, a complex cognitive-based intelligent access control and situation management system. Claim 6 A step of acquiring detection data of an object within a surveillance area by a detection unit comprising at least one sensor; a step of receiving the detection data by a data receiving unit and extracting face information, appearance information, movement path information, and biometric information of the object; a step of processing the information using a pre-trained deep learning module by a complex perception and situation judgment unit to generate identification information and movement history information of the object, and determining whether an abnormal situation exists by comparing it with pre-defined judgment conditions; The method includes the step of outputting a voice or visual warning, controlling an access device, or communicating with a remote control system according to the judgment result by the control unit; wherein the data receiving unit includes a face detection module, an appearance analysis module, a movement tracking module, and a biometric analysis module; the data receiving unit divides the detection data into frames and inputs the detection data divided into frames into the face detection module, the appearance analysis module, the movement tracking module, and the biometric analysis module in parallel to simultaneously extract the face information, the appearance information, the movement path information, and the biometric information, or inputs them sequentially to integrally extract the face information, the appearance information, the movement path information, and the biometric information; and the complex perception and situation judgment unit includes a deep learning module for face recognition, a deep learning module for appearance analysis, a deep learning module for movement path information analysis, and a deep learning module for biometric information analysis, wherein the deep learning module for face recognition, the deep learning module for appearance analysis, the deep learning module for movement path information analysis, and the deep learning module for biometric information analysis are the extracted face information, the appearance A method for operating a complex cognitive-based intelligent access control and situation management system that receives information, the movement path information, and the biometric information in parallel to determine whether there is an abnormal situation. Claim 7 A computer-readable recording medium having a program recorded thereon for executing the method of operating a complex cognitive-based intelligent access control and situational management system of claim 6.

Images