Selective masking of surveillance video

Abstract

The present specification relates to an apparatus and a method for selective masking of surveillance video. The method for selective masking of surveillance video may comprise the steps of: receiving a surveillance video obtained by capturing a surveillance area by means of a surveillance camera; analyzing the surveillance video and, if a first object is recognized in the surveillance video, applying masking to the first object in a first state; changing the masking of the first object from the first state to a second state if a predetermined object is detected within the first object; and releasing the masking of the person object masked in the second state if an event related to the predetermined object is detected in the surveillance video.

Description

Optional masking of surveillance footage

[0001] This specification relates to a technique for masking surveillance images acquired from a surveillance camera.

[0002] In surveillance systems utilizing surveillance camera footage, protecting personal information related to the privacy of individuals contained in the captured video data is essential. Computer vision technology can detect personal information areas, such as facial areas, within surveillance footage, and then privacy mask (e.g., mosaic-processing or blurring) these areas to protect areas where personal information may be compromised.

[0003] However, if a crime occurs in the surveillance footage or a person identified as a criminal appears, selective privacy masking technology is needed, such as not applying privacy masking to the person involved.

[0004] The above-described content is only intended to help understand the background technology for the technical ideas of the present invention, and therefore cannot be understood as content corresponding to prior art known to those skilled in the art in the technical field of the present invention.

[0005] The present specification is intended to solve the aforementioned problem, and one embodiment of the present specification aims to selectively apply masking by determining in real time an area to which privacy masking is to be applied and an area to which privacy masking is not to be applied in a surveillance video.

[0006] In addition, one embodiment of the present specification aims to release the masking of an area when it is determined that a crime is suspected or a dangerous situation has occurred while privacy masking is being applied to a surveillance video.

[0007] In addition, one embodiment of the present specification aims to solve the problem of applying privacy masking to a person related to a crime or a dangerous situation by automatically not applying privacy masking to the object or object related to the situation when a dangerous object or dangerous situation is detected in the surveillance video.

[0008] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] The present specification proposes a selective masking method for surveillance video. The method may include the steps of: receiving a surveillance video acquired by photographing a surveillance area from a surveillance camera; analyzing the surveillance video and, if a first object is recognized in the surveillance video, applying a masking to the first object in a first state; detecting a pre-designated object in the first object, changing the masking of the first object from the first state to a second state; and detecting an event related to the pre-designated object in the surveillance video, releasing the masking of the first object masked in the second state.

[0010] The above-described selective masking method for surveillance video and other embodiments may include the following features.

[0011] In an embodiment, further, the step of unmasking the first object masked in the second state when an event related to the pre-designated object is detected in the surveillance video may include the step of generating an alarm notifying that an unmasking target event has occurred when an event related to the pre-designated object is detected in the surveillance video; and the step of unmasking the first object masked in the second state when there is a user's approval in response to the alarm.

[0012] According to an embodiment, the selective masking method for the surveillance video may further include a step of generating an alarm notifying that a masking release event has occurred when the masking of the first object in the surveillance video is released.

[0013] In an embodiment, further, when an event related to the pre-designated object is detected in the surveillance video, the step of unmasking the first object masked in the second state may include: when the pre-designated object is a blunt object or a weapon, when an action of the first object swinging the pre-designated object or an action of stabbing another object with the pre-designated object is detected in the surveillance video, the step of unmasking the first object masked in the second state; when the pre-designated object is a firearm, when a scream or a gunshot is detected in the surveillance video, the step of unmasking the first object masked in the second state; and when the pre-designated object is an object around the first object, when an action of the first object harming another object with the pre-designated object is detected in the surveillance video, the step of unmasking the first object masked in the second state may include:

[0014] Meanwhile, the present specification proposes a selective masking device for surveillance video. The device comprises: a communication unit that receives a surveillance video acquired by a surveillance camera photographing a surveillance area; an object recognition unit that analyzes the surveillance video based on artificial intelligence to recognize an object in the surveillance video; a situation judgment unit that analyzes the surveillance video based on artificial intelligence to determine the behavior of a first object in the surveillance video and determines a sound generated in the surveillance video; and a masking unit that applies masking according to a masking criterion to the object recognized by the object recognition unit; wherein, when the object recognition unit recognizes a first object in the surveillance video, the masking unit applies masking to the first object in a first state; when the object recognition unit detects a pre-designated object in the first object, the masking for the first object is changed from the first state to a second state; and when the situation judgment unit detects an event related to the pre-designated object in the surveillance video, the masking for the first object masked in the second state can be released.

[0015] The optional masking device for the above surveillance video and other embodiments may include the following features:

[0016] According to an embodiment, the masking unit may generate an alarm notifying that an unmasking target event has occurred when the situation judgment unit detects an event related to the pre-designated object in the surveillance video, and, in response to the alarm, may unmask the first object masked in the second state when there is a user's approval.

[0017] In addition, according to an embodiment, the masking unit may, when the object recognition unit recognizes a second object in the surveillance video, apply masking to at least a portion of the second object in the first state, and when the situation judgment unit determines that the first object, whose masking has been released from the second object, has performed an action related to the second object, release the masking to at least a portion of the masked object in the first state.

[0018] In addition, according to an embodiment, the masking unit may release the masking of the first object masked in the second state when the object recognition unit detects a blunt object or a weapon among the pre-designated objects in the surveillance video, and the situation judgment unit detects an action of the first object swinging the pre-designated object or an action of stabbing another object with the pre-designated object in the surveillance video; release the masking of the first object masked in the second state when the object recognition unit detects a firearm among the pre-designated objects in the surveillance video, and the situation judgment unit detects a scream or a gunshot in the surveillance video; and release the masking of the first object masked in the second state when the object recognition unit detects an object around the first object among the pre-designated objects in the surveillance video, and the situation judgment unit detects an action of the first object harming another object with the pre-designated object in the surveillance video.

[0019] On the other hand, the present specification proposes a selective masking method for surveillance video. The method may include the steps of: receiving a surveillance video acquired by photographing a surveillance area from a surveillance camera; analyzing the surveillance video to recognize a plurality of first objects in the surveillance video, and outputting a portion of the plurality of first objects by masking them in a first state; detecting a pre-designated object from the plurality of first objects in the surveillance video, and changing the masking of a first object from the first state to a second state and outputting the first object in which the pre-designated object is detected among the plurality of first objects; and analyzing a behavior of the first object masked in the second state in the surveillance video, and detecting an event related to the pre-designated object, and unmasking the first object masked in the second state and outputting the first object.

[0020] Additionally, the selective masking method and device for the surveillance video, and other embodiments, may include the following features.

[0021] In some embodiments, the event associated with the pre-specified object may be an action or sound associated with the pre-specified object.

[0022] In some embodiments, the pre-designated objects may also include firearms, weapons, and blunt weapons.

[0023] In some embodiments, the sound associated with the pre-designated object may include a gunshot or a scream, and the action associated with the pre-designated object may include an action of holding a gun or shooting a gun, an action of stabbing or swinging a weapon, and an action of swinging a blunt object.

[0024] In accordance with an embodiment, the selective masking method for the surveillance video may further include: when a second object is recognized in the surveillance video, applying masking to at least a portion of the second object in the first state; and when it is determined that the first object, from which the masking is released, is on board the second object, releasing the masking to at least a portion of the masked in the first state.

[0025] The embodiments disclosed herein have the effect of selectively applying masking by determining in real time the areas to which privacy masking is to be applied and areas to which it is not to be applied in a surveillance video.

[0026] In addition, the embodiments disclosed in this specification have the effect of being able to release the masking of an area when it is determined that a crime is suspected or a dangerous situation has occurred while privacy masking is being applied to a surveillance video.

[0027] In addition, the embodiment disclosed in this specification has the effect of solving the problem of applying privacy masking to a person related to a crime or a dangerous situation by automatically not applying privacy masking to an object or an object related to a dangerous situation when a dangerous object or a dangerous situation is detected in applying privacy masking to a surveillance video.

[0028] Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0029] The following drawings attached to this specification illustrate preferred embodiments of the present invention and, together with specific details for carrying out the invention, serve to further understand the technical idea of ​​the present invention. Therefore, the present invention should not be interpreted as being limited to matters described in such drawings.

[0030] FIG. 1 illustrates a configuration of a selective masking system for surveillance video according to one embodiment.

[0031] Figure 2 is a block diagram schematically showing the internal configuration of a surveillance camera that constitutes a selective masking system for surveillance video.

[0032] FIG. 3 is a drawing illustrating the configuration of an image management device that operates as an optional masking device for surveillance images according to one embodiment.

[0033] Figure 4 is a block diagram of the AI ​​processing unit of Figure 3.

[0034] FIG. 5 is a drawing illustrating a selective masking method for surveillance video according to one embodiment.

[0035] Figure 6 illustrates an example of applying privacy masking when a human object is recognized in a surveillance video.

[0036] Figure 7 illustrates an example of changing the masking state when a pre-specified object is detected from a recognized human object.

[0037] Figure 8 illustrates an example of changing the masking state when an event occurs in a surveillance video.

[0038] Figure 9 is a drawing explaining a method of applying masking when an unmasked human object gets into a car.

[0039] FIG. 10 is a diagram illustrating the configuration of a selective masking device for surveillance video according to an embodiment as a functional block.

[0040] The technology disclosed herein can be applied to selective masking techniques for surveillance video. However, the technology disclosed herein is not limited to this, and can be applied to any device or method to which the technical principles of the technology can be applied.

[0041] It should be noted that the technical terms used in this specification are merely used to describe specific embodiments and are not intended to limit the scope of the technology disclosed herein. Furthermore, unless specifically defined otherwise herein, the technical terms used herein should be interpreted as having a meaning generally understood by a person of ordinary skill in the art to which the technology disclosed herein pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used herein is an incorrect technical term that does not accurately express the scope of the technology disclosed herein, it should be replaced with a technical term that can be correctly understood by a person of ordinary skill in the art to which the technology disclosed herein pertains. Furthermore, general terms used herein should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0042] While terms including ordinal numbers, such as "first" and "second," used herein may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0043] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components are given the same reference numbers and redundant descriptions thereof will be omitted.

[0044] Additionally, when describing the technology disclosed in this specification, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the technology disclosed in this specification. Furthermore, it should be noted that the attached drawings are intended solely to facilitate understanding of the concepts of the technology disclosed in this specification and should not be construed as limiting the scope of the technology.

[0045] An image or video according to one embodiment of the present invention includes both still images and moving images unless there is a special limitation.

[0046] Throughout the specification, a device or terminal includes a communication terminal or communication device capable of wired or wireless communication with a server or other device. The device or terminal may take various forms, such as a mobile phone, smartphone, smart pad, laptop computer, desktop computer, smart TV, or wearable device. Wearable devices may take various forms, such as a watch-type terminal, a glasses-type terminal, or a head-mounted display (HMD). Furthermore, the terminal is not limited to these forms and may be implemented as a variety of electronic devices.

[0047]

[0048] Hereinafter, embodiments are described in detail with reference to the attached drawings.

[0049] FIG. 1 illustrates a configuration of a selective masking system for surveillance video according to one embodiment.

[0050] Referring to FIG. 1, a selective masking system (1000) for surveillance video can be configured to include a surveillance camera (100), a video management device (200), and a video storage (300).

[0051] A surveillance camera (100) photographs a surveillance area to obtain surveillance images of the surveillance area. The surveillance camera (100) is connected to an image management device (200) via a network, and can transmit image data regarding the surveillance images obtained via the network to the image management device (200).

[0052] The video management device (200) may include a video security solution such as a DVR, CMS, NVR, VMS, or a display device. The video management device (200) may receive video data acquired by filming with a surveillance camera (100) through a network and output the data to a user (administrator) through a display, and may also transmit the received video data to a video storage (300). The entirety or at least a portion of the configuration of the video management device (200) may function as a selective masking device for surveillance video according to an embodiment.

[0053] Accordingly, the video management device (200) can selectively apply masking to objects detected based on surveillance images received from the surveillance camera (100) according to predetermined criteria. Data regarding the surveillance images to which selective masking has been applied can be transmitted to the video storage (300) and stored temporarily or permanently. The video data stored in the video storage (300) can be encrypted and protected for security purposes.

[0054] The image storage (300) may store image data received from the image management device (200) by including at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The image data may include an image to which privacy masking (de-identification) is applied for an object requiring specific privacy protection, such as a person or a vehicle.

[0055] Meanwhile, the entire configuration of the surveillance camera (100) or at least a portion thereof may also function as a selective masking device for surveillance images according to an embodiment. That is, the surveillance camera (100) itself, i.e., objects detected in images captured by the edge device, may be selectively masked according to predetermined criteria. Hereinafter, the image management device (200) will be described as an example of a selective masking device according to an embodiment, but the related configuration and functions may be equally applied to the surveillance camera (100).

[0056] Figure 2 is a block diagram schematically showing the internal configuration of a surveillance camera that constitutes a selective masking system for surveillance video.

[0057] Referring to FIG. 2, a surveillance camera (100) may be configured to include an image sensor (110), an encoder (120), a storage unit (130), a communication unit (140), an AI processing unit (150), and a control unit (160).

[0058] The image sensor (110) performs the function of capturing a surveillance area and obtaining a surveillance image, and can be implemented as, for example, a CCD (Charge-Coupled Device) sensor, a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, etc.

[0059] The encoder (120) performs an operation of encoding a surveillance image acquired through an image sensor (110) into a digital signal, which may follow, for example, H.264, H.265, MPEG (Moving Picture Experts Group), M-JPEG (Motion Joint Photographic Experts Group) standards, etc.

[0060] The storage unit (130) can store a program for the operation of the control unit (160) and temporarily store input / output data and generated data. In addition, the storage unit (130) can store video data, audio data, still images, metadata, etc. The metadata may be data including object detection information (movement, sound, intrusion into a designated area, etc.) captured in the surveillance area, object identification information (person, car, face, hat, clothing, etc.), and detected location information (coordinates, size, etc.).

[0061] In addition, the still image is generated together with the metadata and stored in the storage unit (130), and can be generated by capturing image information for a specific analysis area among the image analysis information. For example, the still image can be implemented as a JPEG image file. For example, the still image can be generated by cropping a specific area of ​​the image data determined to be an identifiable object among the image data of the surveillance area detected in a specific area and for a specific period of time, and this can be transmitted in real time together with the metadata.

[0062] The communication unit (140) can transmit the video data, audio data, still images, and / or metadata to the video management device (200 of FIG. 1). According to one embodiment, the communication unit (140) can transmit the video data, audio data, still images, and / or metadata to the video management device in real time. The communication unit (140) can perform at least one communication function among wired / wireless Local Area Network (LAN), Wi-Fi, ZigBee, Bluetooth, and Near Field Communication.

[0063] The AI ​​processing unit (150) is for processing images based on artificial intelligence, and performs object detection, object identification, and object tracking algorithms based on deep learning learned from images acquired through a surveillance camera (100) of an AI image analysis-based camera automatic setting system (1000) according to one embodiment of the present specification. The AI ​​processing unit (150) may be implemented as one module with a control unit (160) that controls the entire system, or may be implemented as an independent module. The AI ​​processing unit (150) may also be implemented as a separate module or device separated from the surveillance camera (100).

[0064] The control unit (160) can control all operations related to the function of the surveillance camera (100) and all other components (image sensor (110), encoder (120), storage unit (130), communication unit (140), and AI processing unit (150)).

[0065] FIG. 3 is a drawing illustrating the configuration of an image management device that operates as an optional masking device for surveillance images according to one embodiment.

[0066] Referring to FIG. 3, the video management device (200) may be configured to include a communication unit (210), a decoder (220), a control unit (230), a storage unit (240), and an AI processing unit (250). The illustrated components are not essential, and a video management device (200) having more or fewer components may be implemented. These components may be implemented in hardware or software, or through a combination of hardware and software.

[0067] The communication unit (210) can receive video data, audio data, still images, and / or metadata from the surveillance camera in real time by communicating with the communication unit (140 of FIG. 2) of the surveillance camera via a network. The communication unit (210) can perform at least one communication function among the networks described below.

[0068] The network disclosed herein may be, but is not limited to, a wireless network, a wired network, a public network such as the Internet, a private network, a Global System for Mobile communication network (GSM) network, a General Packet Radio Network (GPRN), a Local Area Network (LAN), a Wide Area Network (WAN), a Metropolitan Area Network (MAN), a cellular network, a Public Switched Telephone Network (PSTN), a Personal Area Network, Bluetooth, Wi-Fi Direct, Near Field Communication, Ultra-Wide Band, a combination thereof, or any other network.

[0069] The decoder (220) can decode video data, audio data, still images and / or metadata transmitted and received in an encoded state from a surveillance camera.

[0070] The storage unit (240) can store a program for the operation of the control unit (230) and temporarily store input / output data and generated data. In addition, the storage unit (240) can store video data, audio data, still images, metadata, etc. The metadata may be data including object information detected in the surveillance video (movement, sound, intrusion into a designated area, dangerous materials, dangerous behavior, etc.), object identification information (person, car, face, hat, clothing, vehicle license plate, weapon, weapon, etc.), and detected location information (coordinates, size, etc.).

[0071] The control unit (230) can control all operations related to the function of the video management device (200) and all other components (communication unit (210), decoder (220), storage unit (240), and AI processing unit (250)). In particular, the control unit (230) can cooperate with the AI ​​processing unit (250) to detect objects in surveillance images received from surveillance cameras and apply or release masking to objects according to predetermined criteria.

[0072] The AI ​​processing unit (250) is for processing images based on artificial intelligence, and performs object detection, object identification, object behavior judgment, situation judgment, sound detection and situation judgment related to detected sound, and object tracking algorithm based on deep learning learned from images acquired through the surveillance camera (100) of the selective masking system (1000) for surveillance images according to one embodiment of the present specification. The AI ​​processing unit (250) may be implemented as one module with the control unit (230) that controls the entire system, or may be implemented as an independent module. The AI ​​processing unit (250) may also be implemented as a separate module or device separated from the image management device (200).

[0073] The AI ​​processing unit (250) may be configured to functionally include an object recognition unit that recognizes an object in a surveillance video and a situation judgment unit that judges a situation occurring in the surveillance video, and may operate as the object recognition unit and the situation judgment unit. The object recognition unit may include an object recognition model that can recognize or track an object in the video based on deep learning. The situation judgment unit may include a situation judgment model that can judge a situation related to an object recognized in the video based on deep learning, or recognize a situation related to a sound occurring in the video by detecting the sound. The object recognition model and the situation judgment model may be included in a deep learning model (256 of FIG. 4) described below.

[0074] Figure 4 is a block diagram of the AI ​​processing unit of Figure 3.

[0075] The AI ​​processing unit (250) may include an electronic device including an AI module capable of performing AI processing, or a server including the AI ​​module. In addition, the AI ​​processing unit (250) may be included as part of at least a portion of the image management device (200) illustrated in FIG. 3 and may be configured to perform at least a portion of the AI ​​processing performed by the image management device (200).

[0076] The AI ​​processing of the AI ​​processing unit (250) may include all operations related to the control of the control unit (230) illustrated in FIG. 3 and all operations for image recognition through artificial intelligence learning. For example, zero data for a surveillance area acquired by an image sensor of a surveillance camera may be processed by an AI model based on the YOLO model to perform an operation for recognizing the appearance of a vehicle or person in a surveillance video. The AI ​​processing unit (250) may also be included as a component of the control unit (230) illustrated in FIG. 3.

[0077] The above AI processing unit (250) may include an AI processor (251), a memory (255), and / or a communication unit (257).

[0078] The above AI processing unit (250) is a computing device capable of learning a neural network, and can be implemented as various electronic devices such as a server, desktop PC, notebook PC, tablet PC, etc., or can be implemented as a single chip.

[0079] The AI ​​processor (251) can learn a neural network using a program stored in the memory (255). In particular, the AI ​​processor (251) can learn a neural network for recognizing device-related data. Here, the neural network for recognizing device-related data can be designed to simulate the structure of a human brain on a computer, and can include a plurality of network nodes having weights that simulate neurons of a human neural network. The plurality of network modes can each exchange data according to a connection relationship so as to simulate the synaptic activity of neurons that exchange signals through synapses. Here, the neural network can include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes can be located in different layers and exchange data according to a convolution connection relationship. Examples of neural network models include various deep learning techniques such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), and deep Q-networks, and can be applied to fields such as computer vision (CV), speech recognition, natural language processing, and speech / signal processing.

[0080] Meanwhile, the processor performing the functions described above may be a general-purpose processor (e.g., CPU), but may also be an AI-specific processor for artificial intelligence learning (e.g., GPU).

[0081] The memory (255) can store various programs and data required for the operation of the AI ​​processing unit (250). The memory (255) can be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), a solid state drive (SDD), etc. The memory (255) is accessed by the AI ​​processor (251), and data reading / writing / modifying / deleting / updating, etc. can be performed by the AI ​​processor (251). In addition, the memory (255) can store a neural network model (e.g., a deep learning model (256)) generated through a learning algorithm for data classification / recognition according to one embodiment of the present invention.

[0082] Meanwhile, the AI ​​processor (251) may include a data learning unit (252) that learns a neural network for data classification / recognition. The data learning unit (252) may learn criteria regarding which learning data to use to determine data classification / recognition and how to classify and recognize data using the learning data. The data learning unit (252) may acquire learning data to be used for learning and apply the acquired learning data to the deep learning model, thereby learning the deep learning model.

[0083] The data learning unit (252) may be manufactured in the form of at least one hardware chip and mounted on the AI ​​processing unit (250). For example, the data learning unit (252) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics processor (GPU) and mounted on the AI ​​processing unit (250). In addition, the data learning unit (252) may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable medium that can be read by a computer. In this case, at least one software module may be provided by an operating system (OS) or an application (application program).

[0084] The data learning unit (252) may include a learning data acquisition unit (253) and a model learning unit (254).

[0085] The learning data acquisition unit (253) can acquire learning data required for a neural network model for classifying and recognizing data. For example, the learning data acquisition unit (253) can acquire image data and / or sample data for transmitters on road infrastructure for input into the neural network model as learning data.

[0086] The model learning unit (254) can use the acquired learning data to learn the neural network model to have a judgment criterion on how to classify a given data. At this time, the model learning unit (254) can train the neural network model through supervised learning that uses at least some of the learning data as a judgment criterion. Alternatively, the model learning unit (254) can train the neural network model through unsupervised learning that discovers a judgment criterion by learning on its own using the learning data without guidance. In addition, the model learning unit (254) can train the neural network model through reinforcement learning that uses feedback on whether the result of the situation judgment according to the learning is correct. In addition, the model learning unit (254) can train the neural network model using a learning algorithm including error back-propagation or gradient descent.

[0087] Once the neural network model is trained, the model training unit (254) can store the trained neural network model in memory. The model training unit (254) can also store the trained neural network model in the memory of a server connected to the AI ​​processing unit (250) via a wired or wireless network.

[0088] The data learning unit (252) may further include a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or to save resources or time required for creating the recognition model.

[0089] The learning data preprocessing unit can preprocess the acquired data so that it can be used for learning to determine the situation. For example, the learning data preprocessing unit can process the acquired data into a preset format so that the model learning unit (254) can use the acquired learning data for learning to recognize image data for the transmitter.

[0090] Additionally, the learning data selection unit can select data required for learning from among the learning data acquired by the learning data acquisition unit (253) or the learning data preprocessed by the preprocessing unit. The selected learning data can be provided to the model learning unit (254). For example, the learning data selection unit can select only data included in a specific field as learning data by recognizing a specific field among the data sets collected through the network.

[0091] Additionally, the data learning unit (252) may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

[0092] The model evaluation unit inputs evaluation data into the neural network model, and if the analysis results output from the evaluation data do not satisfy a predetermined standard, it can cause the model learning unit (252) to relearn. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluation unit can evaluate that the predetermined standard is not satisfied if the number or ratio of evaluation data with inaccurate analysis results among the analysis results of the learned recognition model for the evaluation data exceeds a preset threshold.

[0093] Meanwhile, the AI ​​processing unit (250) illustrated in FIG. 4 is functionally divided into an AI processor (251), a memory (255), a communication unit (257), etc., but it should be noted that the aforementioned components may be integrated into one module and referred to as an AI module, an AI-based image analysis module, an AI-based object recognition module, or an AI-based situation recognition module.

[0094] Hereinafter, a selective masking method for surveillance images according to an embodiment will be described with reference to FIGS. 3 to 5.

[0095] FIG. 5 is a drawing illustrating a selective masking method for surveillance video according to one embodiment.

[0096] Referring to FIGS. 3 to 5, first, the image management device (200) receives a surveillance image captured from a surveillance camera (S110). The image data for the surveillance image is received from the communication unit (140 of FIG. 2) of the surveillance camera via the communication unit (210), and the control unit (230) transmits the image data for the received surveillance image to the AI ​​processing unit (250) to analyze the surveillance image based on artificial intelligence.

[0097] The control unit (230) analyzes the input surveillance video with the object recognition model of the AI ​​processing unit (250) and, if it recognizes a human object to which privacy masking should be applied within the surveillance video, applies privacy masking to a certain area including the recognized human object so that the object cannot be identified (S120). Here, the masking application area for the recognized human object may be set to a certain area including the entire human object according to a preset value, or may be set to only the face area, which is an area where the human cannot be identified.

[0098] At this time, if the object recognition model of the AI ​​processing unit (250) additionally detects a pre-designated object from a human object to which privacy masking has been applied, the control unit (230) changes the masking for the human object to which privacy masking has been applied to a state different from the privacy masking and applies it (S130). For example, the pre-designated object may be a firearm, a weapon (e.g., a knife, etc.), a dangerous object (e.g., a chair, a club, etc.) around the human object as a condition for determining a violent or criminal situation. At this time, if the control unit (230) detects a pre-designated object such as a firearm or a weapon from the recognized human object, the control unit (230) may change the masking state by adjusting the color, intensity, shape, range / size of the area, etc. of the masking in the privacy masking applied to the human object so that the user (administrator) or an official of a criminal investigation agency can easily recognize the human object holding the firearm or weapon. In some embodiments, images of objects whose masking status has changed may be displayed separately within the overall user viewer screen as a pop-up or picture-in-picture (PIP) for easy observation by the user.

[0099] Fig. 6 illustrates an example of applying privacy masking when a human object is recognized in a surveillance video, and Fig. 7 illustrates an example of changing the masking status when a pre-specified object is detected from the recognized human object.

[0100] Fig. 6(a) illustrates a state in which the AI ​​processing unit (250) analyzes the input surveillance video and recognizes human objects (OB1, OB2) to which privacy masking should be applied within the surveillance video. Fig. 6(b) illustrates a state in which the control unit (230) applies privacy masking (M1) to a certain area including the human objects (OB1, OB2) recognized by the AI ​​processing unit (250) so that the objects cannot be identified.

[0101] FIG. 7(a) illustrates a state in which the AI ​​processing unit (250) detects a pre-designated object (OB3), such as a firearm or a weapon, in a specific object (OB1) among human objects to which privacy masking should be applied. In this case, the control unit (230), as illustrated in FIG. 7(b), changes the state of the privacy masking (M1) already applied to the human object (OB1) possessing the pre-designated object (OB3) from the first state (M1) to the second state (M2) so that the user (administrator) or a criminal investigation agency official can easily recognize it. At this time, the masking of the human object (OB2) in which the pre-designated object is not detected maintains the previous state (M1).

[0102] Referring to FIGS. 3 to 7, when the situation judgment model of the AI ​​processing unit (250) detects an event related to a pre-designated object (OB3) in a surveillance video, the control unit (230) releases the masking of a human object (OB1 in FIG. 7) whose masking status has been changed so that a user (administrator) or a criminal investigation agency official can easily recognize it (S140). Here, the event related to the pre-designated object (OB3) may be the occurrence of a sound or action related to the pre-designated object (OB3), which may be an action or sound determined to be a criminal act committed by the human object (OB1 in FIG. 7) possessing the pre-designated object (OB3). The situation judgment model of the AI ​​processing unit (250) is trained to detect the actions and sounds generated by the human object in the surveillance video to determine a situation related to a crime or violence. The sound related to the pre-designated object may be, for example, the sound of a gunshot or a human scream. Actions related to pre-specified objects can be actions such as picking up a gun, shooting a gun, stabbing with a weapon, swinging with a weapon, and swinging with a blunt object.

[0103] Specifically, if the pre-designated object is a blunt object or a weapon, if an action of a human object swinging the pre-designated object or an action of stabbing another human object with the pre-designated object is detected in the surveillance video, the masking of the human object (OB1 in FIG. 7) that has been masked in a changed state (M2) can be released. In addition, if the pre-designated object is a firearm, if a human scream or gunshot is detected in the surveillance video, the masking of the human object (OB1 in FIG. 7) that has been masked in a changed state (M2) can be released. In addition, if the pre-designated object is an object around the human object, if an action of a human object swinging a nearby object (e.g., a chair, a blunt object, etc.) or performing a harmful action toward another human object is detected in the surveillance video, the masking of the human object (OB1 in FIG. 7) that has been masked in a changed state (M2) can be released.

[0104] According to an embodiment, the process of unmasking a human object (OB1) in which a pre-designated object (OB3) is detected in a second state (M2) may be performed as follows. First, when the control unit (230) detects an event related to a pre-designated object (OB3) in a surveillance video, the control unit (230) notifies the user of an alarm notifying that an unmasking target event has occurred in the human object (OB1) masked in the second state (M2), and if the user approves in response to the alarm, the masking of the human object masked in the second state (M2) may be unmasked. Here, the event related to the pre-designated object (OB3) may mean the occurrence of a sound or action related to the pre-designated object (OB3). The control unit (230) may unmask an area preset by the user according to the user's option in a masked object such as a face or a human upper body.

[0105] Figure 8 illustrates an example of changing the masking state when an event occurs in a surveillance video.

[0106] FIG. 8(a) illustrates a state in which a masking (M2) is applied to a human object (OB1) holding a pre-designated object (OB3) in the same manner as FIG. 7(b) so that the user (administrator) or a criminal investigation agency official can easily recognize the human object. As shown in FIG. 8(b), when the situation judgment model of the AI ​​processing unit (250) detects an action (ST1) and a sound (ST2) that are judged to be events corresponding to masking release conditions such as criminal acts, the control unit (230) notifies the user by alarm that an event corresponding to the masking release conditions has occurred, and releases the masking already applied to the human object (OB1) holding the pre-designated object (OB3) upon the user's approval for masking release. When the control unit (230) determines that a criminal act or suspected criminal act has occurred in the surveillance video, it releases the masking of the object that performed the act, thereby allowing the user (administrator) to observe the video in real time or review the stored video later to confirm the identity of the human object.

[0107] Again, referring to FIGS. 3 to 8, if the object recognition model of the AI ​​processing unit (250) recognizes a vehicle object in the surveillance video, the control unit (230) applies basic privacy masking to some areas of the vehicle object, such as the area where the vehicle registration plate is located and the area where the driver is located, so that they are not identified (S150). The vehicle object refers to a means of transportation such as a motorbike or automobile.

[0108] However, if the situation judgment model of the AI ​​processing unit (250) determines that a human object whose masking was released in the aforementioned S140 is on board the vehicle object, the control unit (230) releases the privacy masking applied to the vehicle object (S160).

[0109] Figure 9 is a drawing explaining a method of applying masking when an unmasked human object gets into a car.

[0110] FIG. 9(a) illustrates a state in which basic privacy masking (M1) is applied to the vehicle registration plate area (OB5) and driver area (OB6) of a vehicle object (OB4) recognized in a surveillance video so as not to be identified. FIG. 9(b) illustrates a situation in which the masking applied to the vehicle object (OB4) is released when a human object (OB1) to which the masking has been released performs an action related to the vehicle object (OB4) to which the privacy masking has been applied, for example, when the human object (OB1) is detected to be getting into the vehicle object (OB4). Here, the control unit (230) can release the masking of an area preset by the user according to the user's option among the masked objects, such as the vehicle license plate, the vehicle window, and the entire vehicle. At this time, if the user's option setting is automatic, the control unit (230) can automatically determine and release the masking area according to the type of object associated with the human object (OB1). For example, when a human object (OB1) is riding in a vehicle, the unmasking area can be determined based on the identified vehicle, such as the driver's seat area of ​​the vehicle, the license plate area of ​​the vehicle, or the entire area of ​​the vehicle.

[0111] Meanwhile, object re-identification (Re-ID) technology can be applied in this process. Object re-identification technology refers to a technology for identifying objects in cases where an object that has already been identified in a scene goes out of the scene and then reappears and the video management device identifies it as the same object, or an object that has already been identified in a scene is covered by another object in the scene and then loses tracking and then reappears and the video management device identifies it as the same object, or a specific object passes through the surveillance areas of multiple surveillance cameras and the video management device identifies and tracks the object passing through the surveillance areas in the images of the relevant surveillance areas as the same object. At this time, the video management device stores vector data on the features of objects identified in the video captured in the surveillance area of ​​each surveillance camera in the metadata of the identified object, and then, when a new object enters from outside the scene, when an object that was covered by another object appears, or when a new object appears in the surveillance area of ​​neighboring surveillance cameras, the feature vector data of the detected object is compared with the vector data stored in the metadata of the object that has disappeared, and if the match is the same or higher than a predetermined value, the detected object can be identified as the same object. Therefore, when an object that has already been identified in a scene leaves the scene and then reappears, when an object that has already been identified in a scene is covered by another object within the scene and then reappears, or when a specific object passes through the surveillance areas of multiple surveillance cameras, the detected object can be identified as the same object again. In this specification, the identity of the feature vector data of the objects is used as the criterion for identifying them as the same object, but various other methods may also be used.

[0112] In the case of detecting that the human object (OB1) is riding in the vehicle object (OB4), vector data on the characteristics of the human object (OB1) carrying a pre-designated object (OB3) is calculated and stored in the metadata of the human object (OB1) to assign an object identifier, and when the human object (OB1) passes through multiple surveillance areas, vector data on the characteristics of objects detected in surveillance images for each surveillance area is calculated, and then the value is compared with the vector data stored in the metadata of the human object (OB1), and an object identifier identical to that of the human object (OB1) is assigned to objects that are recognized as identical, thereby enabling the tracking of the human object (OB1) as it moves to the vehicle object (OB4). Thereafter, when the human object (OB1) overlaps with the vehicle object (OB4) within a scene (Scene) and disappears within the scene, or is identified again in a part of the vehicle object (OB4), it can be determined that the human object (OB1) is riding in the vehicle object (OB4). Through this process, the control unit (230) can determine whether the human object (OB1) has boarded the vehicle object (OB4), and if it is determined that the human object (OB1) has boarded the vehicle object (OB4), the privacy masking applied to the vehicle object (OB4) is released, thereby allowing a user or a criminal investigation agency to confirm the identification information of the vehicle object (OB4), thereby enabling tracking of the escape means of the human object (OB1), who is a criminal suspect. Meanwhile, when the masking of the human object in the surveillance video is released, the control unit (230) can generate an alarm notifying that a masking release event has occurred. The alarm can be generated audibly or can be visually displayed on the user's (administrator's) monitor.

[0113] Image data to which masking has been applied or image data to which masking has been removed in the aforementioned manner is stored in the aforementioned image storage (300 in FIG. 1), and then the image can be reviewed according to a set procedure.

[0114] In the above description, the steps, processes, or operations may be further divided into additional steps, processes, or operations, or combined into fewer steps, processes, or operations, depending on the implementation of the present invention. Furthermore, some steps, processes, or operations may be omitted as needed, or the order of the steps or operations may be switched. Furthermore, each step or operation included in the selective masking method for surveillance video described above may be implemented as a computer program and stored on a computer-readable recording medium, and each step, process, or operation may be executed by a computer device.

[0115] Below, a device for performing a selective masking method for the aforementioned surveillance video is set up.

[0116] FIG. 10 is a diagram illustrating the configuration of a selective masking device for surveillance video according to an embodiment as a functional block.

[0117] Referring to FIG. 10, a selective masking device (400) for surveillance video may be configured to include a communication unit (410), an object recognition unit (420), a situation judgment unit (430), and a masking unit (440). The illustrated components are not essential, and a selective masking device for surveillance video may be implemented with more or fewer components. These components may be implemented in hardware or software, or through a combination of hardware and software.

[0118] The communication unit (410) can receive surveillance images obtained by a surveillance camera photographing the surveillance area.

[0119] The object recognition unit (420) is an artificial intelligence-based object recognition model that analyzes surveillance footage received through the communication unit (410) and can recognize objects such as people, firearms, weapons, and blunt instruments in the surveillance footage.

[0120] The situation judgment unit (430) is an artificial intelligence-based situation judgment model that analyzes the surveillance video received through the communication unit (410) to judge the behavior of a human object in the surveillance video and to judge sounds generated in the surveillance video.

[0121] The masking unit (440) can apply masking to an object recognized by the object recognition unit (420) according to masking criteria.

[0122] Specifically, the masking unit (440) can mask the recognized human object in an unidentifiable state when the object recognition unit (420) recognizes the human object in the surveillance video. Masking in an unidentifiable state means privacy masking, and means masking to a degree that can protect the personal information and private life of the object.

[0123] Meanwhile, the masking unit (440) changes the masking of the human object masked in the non-identifiable state to the identified state when the object recognition unit (420) detects a pre-designated object among the human objects masked in the non-identifiable state. Masking in the identified state means masking in which the privacy masking is released, but the color, intensity, shape, and area / size of the masking are changed so that the object can be easily recognized. The pre-designated object may mean a firearm, a weapon, or a blunt instrument.

[0124] Meanwhile, the masking unit (440) releases the masking of a human object masked in an identification state when the situation judgment unit (430) detects an event related to a pre-designated object in the surveillance video. Here, the occurrence of the event related to the pre-designated object may mean the occurrence of a sound and an action related to the pre-designated object, and the sound and action related to the pre-designated object may be an action determined to be a criminal act committed by the human object and a sound caused by the action. The sound related to the pre-designated object may include the sound of a gun or the sound of a person screaming. The action related to the pre-designated object may include an action of raising a gun or shooting a gun, an action of stabbing or swinging a weapon, and an action of swinging a blunt object.

[0125] Specifically, when the object recognition unit (420) detects a blunt object or a weapon among the objects designated in advance in the surveillance video, and the situation judgment unit (430) detects an action of a human object swinging the pre-designated object or an action of stabbing another human object with the pre-designated object in the surveillance video, the masking unit (440) releases the masking of the human object masked in the identification state.

[0126] In addition, the masking unit (440) releases the masking of a human object masked in an identification state when the object recognition unit (420) detects a firearm among pre-designated objects in the surveillance video and the situation judgment unit (430) detects a human scream or gunshot in the surveillance video.

[0127] In addition, the masking unit (440) detects an object (e.g., a chair, a blunt object, etc.) around a human object among the objects designated in advance in the surveillance video by the object recognition unit (420), and when the situation judgment unit (430) detects that the human object in the surveillance video is performing a harmful action toward another human object as a designated object, the masking unit (440) releases the masking of the human object masked in the identification state.

[0128] Meanwhile, when the object recognition unit (420) recognizes a vehicle object in the surveillance video, the masking unit (440) masks at least a portion of the recognized vehicle object (such as the vehicle passenger area, vehicle registration plate area, etc.) to an unidentifiable state. However, when the situation judgment unit (430) determines that a human object whose masking has been unmasked is riding in the vehicle object masked in an unidentifiable state, the masking unit (440) unmasks the vehicle parts that have been masked in an unidentifiable state. Thereafter, the image data with or without masking is stored in a separate storage device.

[0129]

[0130] The term "unit" as used herein (e.g., control unit, etc.) can mean a unit that includes one or a combination of two or more of hardware, software, or firmware, for example. "Unit" can be used interchangeably with terms such as unit, logic, logical block, component, or circuit, for example. "Unit" can be the smallest unit of an integrally formed component or a part thereof. "Unit" can also be the smallest unit that performs one or more functions or a part thereof. "Unit" can be implemented mechanically or electronically. For example, "unit" can include at least one of an application-specific integrated circuit (ASIC) chip, field-programmable gate array (FPGA), or programmable-logic device that performs certain operations, which are known or will be developed in the future.

[0131] At least a portion of a device (e.g., modules or functions thereof) or a method (e.g., operations) according to various embodiments may be implemented as instructions stored on a computer-readable storage medium, for example, in the form of a program module. When the instructions are executed by a processor, the one or more processors may perform a function corresponding to the instructions. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. The computer-readable storage medium / computer-readable recording medium may include a hard disk, a floppy disk, a magnetic media (e.g., a magnetic tape), an optical media (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc (DVD), a magneto-optical media (e.g., a floptical disk), a hardware device (e.g., a read only memory (ROM), a random access memory (RAM), or a flash memory), etc.). In addition, the program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The above-described hardware devices may be configured to operate as one or more software modules to perform operations of various embodiments, and vice versa.

[0132] Modules or program modules according to various embodiments may include at least one or more of the aforementioned components, some of which may be omitted, or may further include other additional components. Operations performed by modules, program modules, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically. Additionally, some operations may be executed in a different order, omitted, or other operations may be added.

[0133] The term "a" as used herein is defined as one or more than one. Furthermore, the use of introductory phrases such as "at least one" and "one or more" in the claims, even if the same claim includes introductory phrases such as "at least one" and "one or more" and the ambiguous phrase "a," should not be construed to mean that the introduction of another claim element by the ambiguous phrase "a" limits any particular claim containing the introduced claim element to an invention containing only one such element.

[0134] In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of the items listed together.

[0135] Unless otherwise specified, terms such as "first" and "second" are used arbitrarily to distinguish between the elements they describe. Therefore, these terms are not necessarily intended to indicate temporal or other priority of such elements, nor does the mere fact that certain measures are recited in different claims indicate that a combination of such measures cannot be advantageously employed. Therefore, these terms are not necessarily intended to indicate temporal or other priority of such elements. The mere fact that certain actions are recited in different claims does not indicate that a combination of such actions cannot be advantageously employed.

[0136] The arrangement of components to achieve the same function is effectively "related" to achieve the desired function. Therefore, any two components combined to achieve a specific functionality can be considered "related" to achieve the desired function, regardless of the structural or intermediary components. Similarly, two components thus associated can be considered "operably connected" or "operably coupled" to achieve the desired function.

[0137] Furthermore, those skilled in the art will recognize that the functional boundaries between the aforementioned operations are merely exemplary. Multiple operations may be combined into a single operation, a single operation may be divided into additional operations, and operations may be executed with at least partial temporal overlap. Furthermore, alternative embodiments may include multiple instances of a particular operation, and the order of the operations may be altered in various other embodiments. However, other modifications, variations, and alternatives are also possible. Accordingly, the detailed description and drawings should be considered in an illustrative rather than a restrictive sense.

[0138] The phrase "may be X" indicates that condition X may be satisfied. It also indicates that condition X may not be satisfied. For example, a reference to a system that includes a particular component must also include scenarios where the system does not include the particular component. For example, a reference to a method that includes a particular action must also include scenarios where the method does not include the particular component. However, to take another example, a reference to a system that is configured to perform a particular action must also include scenarios where the system is not configured to perform the particular task.

[0139] The terms "comprising," "having," "consisting of," "consisting of," and "consisting essentially of" are used interchangeably. For example, any method may include at least the acts described in the drawings and / or specification, or may include only the acts described in the drawings and / or specification. Furthermore, the word "comprising" does not exclude the presence of elements or acts listed in a claim.

[0140] Those skilled in the art will recognize that the boundaries between logical blocks are merely exemplary, and that alternative embodiments may merge logical blocks or circuit elements or impose alternative decompositions of functionality across various logical blocks or circuit elements. Therefore, it should be understood that the architecture depicted herein is merely exemplary, and that many other architectures that achieve the same functionality may be implemented.

[0141] Furthermore, for example, in one embodiment, the illustrated examples may be implemented as circuits located on a single integrated circuit or within the same device. Alternatively, the examples may be implemented as any number of individual integrated circuits or individual devices interconnected in any suitable manner, and other variations, modifications, variations, and alternatives are also possible. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0142] Additionally, for example, the examples described above or portions thereof may be implemented as software or code representations of physical circuits or logical representations convertible to physical circuits, such as in any suitable type of hardware description language.

[0143] Furthermore, the present invention is not limited to physical devices or units implemented with non-programmable hardware, but may also be applied to programmable devices or units capable of performing desired device functions by operating in accordance with appropriate program code, such as mainframes, minicomputers, servers, workstations, personal computers, notepads, personal digital assistants (PDAs), electronic games, automobiles and other embedded systems, mobile phones and various other wireless devices, generally referred to herein as 'computer systems'.

[0144] The system, apparatus or device referred to in this specification includes at least one hardware component.

[0145] Connections as described herein may be any type of connection suitable for transmitting signals from or to each node, unit, or device, for example, via an intermediate device. Accordingly, unless explicitly stated otherwise, a connection may be, for example, a direct connection or an indirect connection. A connection may be described or illustrated with reference to a single connection, multiple connections, a unidirectional connection, or a bidirectional connection. However, different embodiments may vary the implementation of the connection. For example, separate unidirectional connections may be used instead of a bidirectional connection, or vice versa. Furthermore, multiple connections may be replaced by a single connection that transmits multiple signals sequentially or in a time-multiplexed manner. Similarly, a single connection transmitting multiple signals may be split into multiple connections that transmit subsets of those signals. Thus, numerous options exist for transmitting signals.

[0146] Preferred embodiments of the technology of this specification have been described above with reference to the attached drawings. The terms and words used in this specification and claims should not be construed as limited to their conventional or dictionary meanings, but rather should be interpreted in their meanings and concepts consistent with the technical spirit of the present invention. The scope of the present invention is not limited to the embodiments disclosed in this specification, and the present invention may be modified, altered, or improved in various forms within the spirit of the present invention and the scope of the claims.

[0147] The embodiments disclosed in this specification can be applied to surveillance cameras and surveillance camera systems, and to fields of service provision using the same.

Claims

1. A step of receiving a surveillance image obtained by photographing a surveillance area from a surveillance camera; A step of analyzing the above surveillance video and recognizing a first object in the surveillance video, and then applying masking to the first object as a first state; A step of changing the masking for the first object from the first state to the second state when detecting a pre-specified object in the first object; and A step of unmasking the first object masked to the second state when an event related to the pre-designated object is detected in the surveillance video; A selective masking method for surveillance footage.

2. In paragraph 1, the event related to the above-mentioned pre-designated object is, characterized by actions and sounds related to the above pre-designated objects; A selective masking method for surveillance footage.

3. In paragraph 2, the pre-designated object is, Characterized by the inclusion of firearms, weapons and blunt instruments; A selective masking method for surveillance footage.

4. In paragraph 3, Sounds associated with the above pre-designated objects include gunshots or screams, The actions related to the above-mentioned pre-designated object are characterized by actions including actions of holding a gun or shooting a gun, actions of stabbing with a weapon or swinging a weapon, and actions of swinging a blunt object. A selective masking method for surveillance footage.

5. In the first paragraph, the step of unmasking the first object masked in the second state when an event related to the pre-designated object is detected in the surveillance video is, A step of generating an alarm notifying that an unmasking target event has occurred when an event related to the above-mentioned pre-designated object is detected in the above-mentioned surveillance video; and characterized in that it comprises a step of unmasking the first object masked in the second state if there is a user's approval in response to the alarm; A selective masking method for surveillance footage.

6. In paragraph 1, Further comprising a step of generating an alarm notifying that a masking release event has occurred when the masking of the first object in the surveillance video is released; A selective masking method for surveillance footage.

7. In paragraph 1, When recognizing a second object in the surveillance video, a step of applying masking to at least a part of the second object with the first state; and If it is determined that the unmasked first object has performed an action associated with the second object, the step of unmasking at least a portion of the masked first state is further included; A selective masking method for surveillance footage.

8. In paragraph 1, The step of unmasking the first object masked to the second state when an event related to the pre-designated object is detected in the surveillance video is, If the above-mentioned pre-designated object is a blunt weapon or a weapon, a step of unmasking the first object masked in the second state when an action of the first object swinging the pre-designated object or an action of stabbing another object with the pre-designated object is detected in the surveillance video; If the above-mentioned pre-designated object is a firearm, a step of unmasking the first object masked in the second state when a scream or gunshot is detected in the surveillance video; and In the case where the above-mentioned pre-designated object is an object around the first object, if it is detected in the surveillance video that the first object is performing an action harmful to another object as the above-mentioned pre-designated object, a step of unmasking the first object masked in the second state is included; A selective masking method for surveillance footage.

9. A communication unit that receives surveillance images obtained by a surveillance camera capturing the surveillance area; An object recognition unit that analyzes the surveillance video based on artificial intelligence to recognize objects in the surveillance video; A situation judgment unit that analyzes the surveillance video based on artificial intelligence to determine the behavior of the first object in the surveillance video and determines the sound generated in the surveillance video; A masking unit that applies masking according to masking criteria to an object recognized by the object recognition unit; including: The above masking part, When the above object recognition unit recognizes a first object in the surveillance video, it applies masking to the first object as a first state, When the object recognition unit detects a pre-designated object from the first object, the masking for the first object is changed from the first state to the second state, and When the above situation judgment unit detects an event related to the above pre-designated object in the above surveillance video, the masking of the first object masked in the second state is released. Optional masking device for surveillance footage.

10. In paragraph 9, an event related to the pre-designated object is characterized by actions and sounds related to the above pre-designated objects; Optional masking device for surveillance footage.

11. In the 10th paragraph, the pre-designated object is, Characterized by the inclusion of firearms, weapons and blunt instruments; Optional masking device for surveillance footage.

12. In paragraph 11, Sounds associated with the above pre-designated objects include gunshots or screams, The actions related to the above-mentioned pre-designated object are characterized by actions including actions of holding a gun or shooting a gun, actions of stabbing with a weapon or swinging a weapon, and actions of swinging a blunt object. Optional masking device for surveillance footage.

13. In paragraph 9, the masking part, When the above situation judgment unit detects an event related to the above-mentioned pre-designated object in the above-mentioned surveillance video, it generates an alarm notifying that an event to be unmasked has occurred. In response to the alarm, if there is a user's approval, the masking of the first object masked in the second state is unmasked. Optional masking device for surveillance footage.

14. In the 9th paragraph, the masking part, When the object recognition unit recognizes a second object in the surveillance video, masking is applied to at least a portion of the second object in the first state, and If the above situation judgment unit determines that the first object whose masking has been released has performed an action related to the second object, the masking of at least a portion of the first state is released. Optional masking device for surveillance footage.

15. In paragraph 9, the masking part, If the object recognition unit detects a blunt object or a weapon among the pre-designated objects in the surveillance video, and the situation judgment unit detects an action of the first object swinging the pre-designated object or an action of stabbing another object with the pre-designated object in the surveillance video, the masking of the first object masked in the second state is released, When the object recognition unit detects a firearm among the pre-designated objects in the surveillance video, and the situation judgment unit detects a scream or gunshot in the surveillance video, the masking of the first object masked in the second state is released, and The object recognition unit detects objects around the first object among the pre-designated objects in the surveillance video, and the situation judgment unit detects that the first object performs a harmful action toward another object as the pre-designated object in the surveillance video, and then releases the masking of the first object masked in the second state. Optional masking device for surveillance footage.

16. A step of receiving a surveillance image obtained by photographing a surveillance area from a surveillance camera; A step of analyzing the above surveillance video and recognizing a plurality of first objects in the surveillance video, and then outputting a portion of the plurality of first objects by applying masking to the first state; A step of detecting a pre-designated object from the plurality of first objects of the surveillance video, changing the masking for the first object from the first state to the second state and outputting the first object from among the plurality of first objects in which the pre-designated object is detected; and A step of analyzing the behavior of the first object masked in the second state in the surveillance video, detecting an event related to the pre-designated object, and then unmasking and outputting the first object masked in the second state; A selective masking method for surveillance footage.

17. In paragraph 16, the event related to the above-mentioned pre-designated object is, characterized by actions and sounds related to the above pre-designated objects; A selective masking method for surveillance footage.

18. In paragraph 17, the pre-designated object is, Characterized by the inclusion of firearms, weapons and blunt instruments; A selective masking method for surveillance footage.

19. In paragraph 18, Sounds associated with the above pre-designated objects include gunshots or screams, The actions related to the above-mentioned pre-designated object are characterized by actions including actions of holding a gun or shooting a gun, actions of stabbing with a weapon or swinging a weapon, and actions of swinging a blunt object. A selective masking method for surveillance footage.

20. In paragraph 16, When recognizing a second object in the surveillance video, a step of applying masking to at least a part of the second object with the first state; and If it is determined that the unmasked first object has performed an action associated with the second object, the step of unmasking at least a portion of the masked first state is further included; A selective masking method for surveillance footage.

Images