System for auto-masking front door shape images using deep learning object information and auto-masking method using the same

Abstract

To prevent a door lock password from being exposed. [Solution] The system for automatically masking front door shape images includes: an image capture device that captures the front door and transfers it to a server; a server that stores image information 1 received from the image capture device and generates a machine learning algorithm or deep learning algorithm based on the front door shape and product image attached to the front door contained in the image information 1, thereby extracting feature map information of the front door; sets the extracted feature map information of each front door area as an encryption key; masks each front door area in the image information 1 using a pre-stored algorithm; stores the masked image information 2; and, upon user request, transfers the masked front door image information 2 to a user smart terminal; and a user smart terminal that requests information about the front door from the server, receives and provides the image information with the front door masked.

Description

[Technical Field]

[0001] The present invention relates to a method for masking a front door of an apartment or office in video data captured by the front door. Generally, a user arrives at the front door and unlocks the door by entering a password using a keypad attached to a door lock module. However, if the video data of the password entry is stored in an image capture device, the door lock password may be exposed, which needs to be prevented. [Background technology]

[0002] Prior art related to the present invention is disclosed in Korean Patent Registration No. 10-2551230 (published July 5, 2023). FIG. 1 is a configuration diagram of a conventional masking processing system for automatically masking and unmasking surveillance camera images in real time using deep learning object information. As shown in FIG. 1, the conventional masking processing system for automatically masking and unmasking surveillance camera images in real time using deep learning object information includes a video imaging device 100, a server 200, and a user terminal 300. In one embodiment, the video imaging device 100 may be a fixed CCTV that captures a pre-designated zone in real time, or may be installed in the pre-designated zone. In one embodiment, the video imaging device 100 may generate first video data for the pre-designated zone. In one embodiment, the video imaging device 100 may include a communication module that transmits and receives data to and from a server 200 via wired or wireless communication. In another embodiment, the server 200 may include an artificial intelligence analysis unit, a masking unit, a storage unit, a communication unit, and an unmasking unit. In one embodiment, the AI ​​analysis unit may generate an AI learning model using a pre-stored algorithm based on a human shape image and facial images of men and women of all ages and races. In one embodiment, the pre-stored algorithm may be a machine learning algorithm or a deep learning algorithm. In one embodiment, the deep learning algorithm may be a convolutional neural network (CNN) algorithm. The convolutional neural network may include a feature extraction region and a classification region configured by stacking multiple convolutional layers and max pooling layers. The convolutional neural network may perform convolutional operations on input data through a loop of filters to extract image features, and may extract feature map information using the operation results.The feature map information may be information expressed as a matrix. In one embodiment, the AI ​​analysis unit 210 may identify a person using an AI learning model that uses the first image data as an input variable and detect a facial region of the identified person. In one embodiment, when the first image data includes multiple people, the AI ​​analysis unit 210 may identify the multiple people using an AI learning model and extract feature map information for the facial region of each identified person. The feature map information for each person's facial region extracted from the first image data, which is the original unmasked image data, may have different values. In one embodiment, the masking unit 220 may set the feature map information for each person's facial region extracted by the AI ​​analysis unit 210 as an encryption key and mask each person's facial region in the first image data using a pre-stored algorithm. In one embodiment, the masking method may be an encryption method. In one embodiment, the pre-stored algorithm may be a scrambling encryption algorithm. The storage unit 230 may store, in one embodiment, feature map information of each person's facial region extracted through the artificial intelligence analysis unit 210 and second image data in which each person's facial region is masked. The communication unit 240 may receive facial image data to be restored from the user terminal 300. When the unmasking unit 250 receives the facial image data to be restored from the user terminal 300, it may restore, in the masked second image data, a facial region having feature map information that matches the feature map information extracted from the facial image data to be restored. Specifically, when the facial image data to be restored is received from the user terminal 300 through the communication unit 240, the artificial intelligence analysis unit 210 may extract feature map information from the facial image data to be restored using a pre-stored artificial intelligence learning model. The unmasking unit 250 may determine, in one embodiment, whether feature map information of each person's facial region extracted through the artificial intelligence analysis unit 210 matches the feature map information extracted from the facial image data to be restored.In one embodiment, when the unmasking unit 250 determines that feature map information of each person's facial region extracted by the artificial intelligence analysis unit 210 matches feature map information extracted from the facial image data to be restored, the unmasking unit 250 may restore (decrypt) the facial region using the feature map information extracted from the facial image data to be restored as an encryption key. In one embodiment, the pre-stored algorithm may be a scrambling encryption algorithm. In one embodiment, the storage unit 230 may store third image data, which is image data from which only specific faces have been restored (decrypted). In this way, only the faces of specific people can be restored from masked images to be used as evidence in a specific case, while the faces of the remaining people can remain masked. By selectively restoring only the faces of specific people, the personal information of the remaining people can be protected. In one embodiment, the unmasking unit 250 may transmit a restoration impossible message to the user terminal 300 via the communication unit 240 if it is determined that there is no feature map information matching the feature map information extracted from the restoration target face image data among the feature map information of each person's facial region extracted by the artificial intelligence analysis unit 210. In one embodiment, the user terminal 300 may connect to the server 200 via an app or the web and output on a screen first image data, which is the original image data; second image data, which is data in which the facial regions of all people in the image are masked; and third image data, which is data in which only the face requested by the user has been unmasked (restored). In one embodiment, the user terminal 300 may be a digital device equipped with a memory means and a microprocessor and having computing capabilities, such as a mobile communication terminal, a desktop PC, a laptop computer, a workstation, a palmtop PC, a personal digital assistant (PDA), a web pad, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Korean Patent No. 10-2551230 (Announced July 5, 2023) Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional technology configured as described above can protect the privacy of individuals whose faces are not restored by masking the faces of people included in camera video data transmitted by a camera installed in a specific area and restoring the faces of specific people requested by the user. However, if a video of a person entering or exiting an apartment building or front door pressing a password on a keyboard included in the door lock is hacked, it cannot prevent the leakage of information such as the password and apartment number. Another object of the present invention is to mask the specific front door area and make the rest of the video visible, thereby eliminating the problem of invasion of others' privacy and eliminating the possibility of criminal misuse. [Means for solving the problem]

[0005] The system for automatically masking a front door shape image using deep learning object information of the present invention, which has the above-mentioned objectives, is characterized by comprising: an image capturing device that captures an image of a front door and transmits it to a server; a server that stores image information 1 received from the image capturing device, generates a machine learning algorithm or a deep learning algorithm based on the shape of the front door and a product image attached to the front door contained in the image information 1, extracts feature map information of the front door, sets the extracted feature map information of each front door area as an encryption key, masks each front door area in the image information 1 using a pre-stored algorithm, stores the masked image information 2, and transmits the masked image information 2 of the corresponding front door to a corresponding user's smart device upon a user request; and an administrator's smart device that requests information about the corresponding front door from the server, receives the masked image information of the corresponding front door from the server, and provides it. [Effects of the Invention]

[0006] The present invention configured as described above has the effect of preventing the door lock password from being hacked and used for criminal purposes. In addition, the present invention is free from the problem of violating the privacy of others by masking a specific area of ​​the front door and leaving the rest of the image visible, and has the effect of eliminating the possibility of misuse for criminal purposes. [Brief explanation of the drawings]

[0007] [Figure 1] This is a configuration diagram of a masking processing system that automatically masks and unmasks surveillance camera footage in real time using conventional deep learning object information. [Figure 2] This is a configuration diagram of a system for automatically masking front door shape images using deep learning object information of the present invention. [Figure 3] FIG. 2 is a detailed configuration diagram of a server applied to the present invention. [Figure 4]1 is a control flowchart of a method for automatically masking a front door feature image using deep learning object information of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] The system for auto-masking a front door shape image using deep learning object information according to the present invention having the above-mentioned objectives and the auto-masking method using the same will be described below with reference to Figures 2 to 4.

[0009] FIG. 2 is a configuration diagram of a system for automatically masking front door shape images using deep learning object information of the present invention. As shown in FIG. 2, the system for automatically masking a front door shape image using deep learning object information of the present invention includes an image capturing device 10 that captures an image of a front door and transmits it to a server; a server 20 that stores image information 1 (image captured by the image capturing device) received from the image capturing device, generates a machine learning model or deep learning model based on the shape of the front door and a product image attached to the front door included in the image information 1, detects the front door area in the image information 1 using the image capturing device, extracts feature map information of the front door area from the detected front door area, sets the extracted feature map information of the front door area as an encryption key, masks the front door area included in the image information 1 using a pre-stored machine learning algorithm or deep learning algorithm, stores masked image information 2 (image with masking), and transmits the masked image information 2 of the front door to a user's smart device upon request; and an administrator smart device 30 that requests information about the front door from the server, receives from the server and provides the image information in which the front door area is masked. In the above, the machine learning model or deep learning model may be a machine learning algorithm or a deep learning algorithm. In the above, the algorithm used to extract a feature map of the detected front door area may be a convolutional neural network algorithm among deep learning algorithms, and the convolutional neural network algorithm may include a feature extraction region and a classification region configured in a form in which a plurality of convolutional layers and pooling layers are overlapped.In addition, in the above, extracting feature map information of the detected front door area, setting the extracted feature map information of the front door area as an encryption key, and masking the front door area included in the video information 1 using a pre-stored machine learning algorithm or deep learning algorithm means applying a feature point extraction algorithm to the detected front door area to extract feature points of the front door area, saving the calculation result of the front door area as image map information, applying an encryption algorithm to the saved image map to perform a masking calculation, and saving the masked image information that is the calculation result.

[0010] 3 is a detailed configuration diagram of a server applied to the present invention. As shown in FIG. 3, the server 20 applied to the present invention includes a storage unit 22 that stores video information 1 received from a video capture device, transfers the stored video information 1 to an artificial intelligence analysis unit, receives and stores video information 2 in which a corresponding entrance door is masked from an artificial intelligence masking unit, an artificial intelligence analysis unit 24 that receives video information 1 from the storage unit and stores it in a memory, generates a machine learning model or a deep learning model based on the shape of the entrance door and the product image attached to the entrance door included in the video information 1, stores it in a memory, detects the corresponding entrance door area using the generated model, and transfers the video information 1 including the detected entrance door area information to the artificial intelligence masking unit, and The system is characterized by including an artificial intelligence masking unit 26 that extracts feature map information of the corresponding front door area based on the detected information on the corresponding front door area, sets the extracted feature map information of the corresponding front door area as an encryption key, masks the corresponding front door area included in the video information 1 using a previously stored machine learning algorithm or deep learning algorithm, and transfers video information 2 in which the corresponding front door area has been masked to the storage unit 22, and a request determination unit 28 that determines whether or not there is an information request for the corresponding front door area from the user terminal, and if there is an information request, receives the video information 2 in which the corresponding front door area has been masked from the storage unit 22 and transfers it to the user terminal.

[0011] 4 is a control flowchart of a method for automatically masking a front door shape image using deep learning object information of the present invention. As shown in FIG. 4, the method for automatically masking a front door shape image using deep learning object information of the present invention includes step S11 in which an image capture device captures an image of an apartment or office entrance and transfers the captured image information to a server, step S12 in which the server detects one or more front door areas using an artificial intelligence learning model previously constructed and stored from the image information of the apartment or office entrance received from the image capture device, step S13 in which the server extracts feature map information from the detected one or more front door areas, and step S14 in which the server extracts feature map information from the extracted one or more corresponding front door areas. The method includes step S14 of setting the map as an encryption key and masking the front door area in the video information using a pre-stored algorithm by applying the set encryption key; step S15 of the server storing the video information with the masked front door area; step S16 of the user's smart device requesting video review from the server; step S17 of the server receiving the video review request transmitting the video information with the masked front door area to the user's smart device; and step S18 of the user's smart device providing the video information with the masked front door area(s). In the method, step S12 of detecting the front door area(s) using a pre-built and stored AI learning model from the video information of the apartment or office entrance involves generating an AI learning model using a pre-stored algorithm based on the shape of the front door and image information of accessories attached to the front door, and using the AI ​​learning model to detect the front door area(s). The pre-stored algorithm may be a machine learning algorithm or a deep learning algorithm.In step S13, the server extracts feature map information for the detected single or multiple front door areas, and the algorithm used to extract the feature map information for the single or multiple front door areas may be a convolutional neural network algorithm among deep learning algorithms, and the convolutional neural network algorithm may include a feature extraction region and a classification region configured by overlapping multiple convolutional layers and pooling layers.In step S14, the server sets the feature map for the extracted single or multiple front door areas as an encryption key, and masks the front door areas in the image information using a pre-stored algorithm by applying the set encryption key.The server stores the result of the feature point extraction operation for the corresponding front door areas as image map information, performs a masking operation by applying an encryption algorithm to the stored image map, and stores the masked image information that is the calculation result. [Explanation of symbols]

[0012] 10...Image acquisition device 20...Server 30...User smart device

Claims

1. This system automatically masks front door shape images using deep learning object information, which can prevent door lock passwords from being hacked and used for criminal purposes. The system for automatically masking a front door shape image using the deep learning object information includes: an image capturing device 10 that captures an image of the entrance door and transfers it to a server; a server 20 that stores image information 1 received by an image capturing device, generates a learning model based on the shape of the front door and a product image attached to the front door included in the image information 1, detects a corresponding front door area from the image information 1 using the generated learning model, extracts feature map information of the corresponding front door area from the detected corresponding front door area, sets the extracted feature map information of the corresponding front door area as an encryption key, masks the corresponding front door area included in the image information 1 using a pre-stored machine learning algorithm or deep learning algorithm, stores the masked image information 2, and transmits the masked image information 2 of the corresponding front door to a corresponding user smart terminal; A system for automatically masking a front door shape image using deep learning object information, which is characterized by being configured with an administrator smart terminal 30 that requests information about the front door from a server and receives and provides image information in which the front door area is masked from the server.

2. The learning model of the server is The system for automatically masking a front door shape image using deep learning object information according to claim 1, which can be a machine learning algorithm or a deep learning algorithm.

3. The algorithm used to extract the feature map of the detected entrance door area of ​​the server is: The system for automatically masking a front door shape image using deep learning object information according to claim 1, wherein the deep learning algorithm may be a convolutional neural network algorithm.

4. The server The system for automatically masking a front door shape image using deep learning object information according to claim 1, characterized in that it applies a feature point extraction algorithm to the detected front door area to extract feature points of the front door area, saves the calculation result of the front door area as image map information, applies an encryption algorithm to the saved image map to perform a masking calculation, and saves the masked image information that is the calculation result.

5. The server a storage unit (22) that stores image information (1) received from the image capture device, transfers the stored image information (1) to an artificial intelligence analysis unit, and receives and stores image information (2) in which the corresponding entrance door is masked from the artificial intelligence masking unit; an artificial intelligence analysis unit (24) that receives image information (1) from the storage unit, stores it in a memory, generates a machine learning model or a deep learning model based on the shape of the entrance door included in the image information (1) and a product image attached to the entrance door, stores it in a memory, detects the corresponding entrance door area using the model, and transfers the image information (1) including the detected entrance door area information to an artificial intelligence masking unit; 2. The system for automatically masking a front door shape image using deep learning object information according to claim 1, further comprising an artificial intelligence masking unit (26) that extracts feature map information of the front door area based on the detected front door area information received from the artificial intelligence analysis unit, sets the extracted feature map information of the front door area as an encryption key, masks the front door area included in the image information (1) using a previously stored machine learning algorithm or deep learning algorithm, and transfers the image information (2) with the masked front door area to the storage unit (22).

6. The convolutional neural network algorithm: The system for auto-masking front door shape images using deep learning object information according to claim 3, characterized in that it can include a feature extraction region and a class classification region configured in the form of multiple overlapping convolutional layers and pooling layers.

7. The server The system for automatically masking a front door shape image using deep learning object information described in claim 5, further comprising a request determination unit 28 that determines whether or not there is a request for information on the front door area from the user terminal, and if there is an information request, receives image information 2 in which the front door area is masked from the storage unit 22 and transfers it to the user terminal.

8. A method for automatically masking a front door shape image using deep learning object information that can prevent a door lock password from being hacked and used for criminal purposes, The method for automatically masking a front door shape image using the deep learning object information includes: Step S11 in which the image capturing device captures an image of the entrance of the apartment building or office and transfers the captured image information to the server; Step S12: The server detects one or more entrance door areas using an artificial intelligence learning model that has been previously constructed and stored based on the image information of the entrance of the apartment or office received from the image capturing device; Step S13: The server extracts feature map information from the detected entrance door area or areas; Step S14: The server sets the extracted feature map of the corresponding one or more corresponding front door areas as an encryption key, and masks the front door areas in the image information using a pre-stored algorithm by applying the set encryption key; Step S15: The server stores the image information in which the entrance door area is masked; Step S16: the user's smart terminal requests the server to check the video; Step S17: The server that has received the video confirmation request information transmits the masked video information of the corresponding entrance door area to the corresponding user's smart terminal. A method for automatically masking a front door shape image using deep learning object information, characterized in that it includes a step S18 in which a user smart terminal provides image information in which the corresponding one or more front door areas are masked.

9. Step S12 detects one or more entrance door areas using an artificial intelligence learning model that is pre-constructed and stored based on the image information of the entrance of the apartment or office, A method for automatically masking a front door shape image using deep learning object information as described in claim 8, characterized in that an artificial intelligence learning model is generated using a pre-stored algorithm based on the shape of the front door and image information of the attached products attached to the front door, and this is used to detect one or more front door areas.

10. In step S13, the server extracts feature map information of the detected entrance door area or areas, and the algorithm used to extract the feature map information of the entrance door area or areas is: The method for auto-masking a front door shape image using deep learning object information according to claim 8, wherein the deep learning algorithm may be a convolutional neural network algorithm.

11. Step S14 in which the server sets the extracted feature map of the corresponding one or more corresponding front door areas as an encryption key, and masks the front door area from the video information using a pre-stored algorithm by applying the set encryption key; A method for automatically masking a front door shape image using deep learning object information as described in claim 8, characterized in that the result of the feature point extraction calculation for the corresponding front door area is saved as image map information, an encryption algorithm is applied to the saved image map to perform a masking calculation, and the masked image information resulting from the calculation is saved.

Images