Automasking System of Shape Image for Front Door by Using Deep Learning Object data and Method thereof

Abstract

The system for auto-masking an image of a front door shape using deep learning object information according to the present invention is characterized by comprising: an image acquisition device that captures a front door and transmits it to a server; a server that stores image information 1 received from the image acquisition device and can generate a machine learning algorithm or a deep learning algorithm based on the shape of the front door included in the image information 1 and an image of a product attached to the front door, thereby extracting feature map information of the front door and setting the extracted feature map information of each front door area as an encryption key, and using a pre-stored algorithm to mask the area of ​​each front door in the image information 1 and store the masked image information 2, and transmits the masked image information 2 of the corresponding front door to the corresponding user's smart terminal when there is a user request; and an administrator smart terminal that requests information about the corresponding front door from the server and receives and provides the image information of the corresponding front door that is masked from the server.

Description

Technology Field

[0001] The present invention relates to masking a front door in video data captured of a front door of an apartment or office. Generally, a user arrives at a front door and unlocks the door lock by entering a password using a keypad mounted on a door lock module. However, if video data of the password being entered exists in a video acquisition device, the door lock password may be exposed, so it is necessary to prevent this. Background Technology

[0003] The prior art related to the present invention is disclosed in Korean Registered Patent No. 10-2551230 (published July 5, 2023). FIG. 1 is a configuration diagram of a masking processing system that auto-masks and de-masks surveillance camera images in real time using the prior deep learning object information. The masking processing system in FIG. 1 that auto-masks and de-masks surveillance camera images in real time using the prior deep learning object information includes an image capturing device (100), a server (200), and a user terminal (300). As one embodiment, the image capturing device (100) may be a fixed CCTV that captures a pre-designated area in real time and may be placed in the pre-designated area. As one embodiment, the image capturing device (100) may generate first image data for the pre-designated area. Additionally, as one embodiment, the image capturing device (100) may include a communication module that transmits and receives data with the server (200) using wired or wireless communication. In addition, the server (200) may include an artificial intelligence analysis unit, a masking unit, a storage unit, a communication unit, and a masking release unit. As one embodiment, the artificial intelligence analysis unit may generate an artificial intelligence learning model using a pre-stored algorithm based on a human shape image and face images of men, women, and children of various races. As one embodiment, the pre-stored algorithm may be a machine learning algorithm or a deep learning algorithm. As one embodiment, the deep learning algorithm may be a convolutional neural network (CNN) algorithm. The convolutional neural network may include a feature extraction area and a class classification area configured by stacking multiple layers of convolutional layers and max pooling layers.A convolutional neural network can perform convolution operations on input data by traversing filters to extract image features, and can extract feature map information using the result of the operation. The feature map information may be information expressed as a matrix. As one embodiment, the artificial intelligence analysis unit (210) can identify a person using an artificial intelligence learning model that takes the first image data as an input variable and can detect the face region of the identified person. As one embodiment, if the first image data contains multiple people, the artificial intelligence analysis unit (210) can identify multiple people using an artificial intelligence learning model and extract feature map information of the face region of each identified person. The feature map information of the face region of each person extracted from the first image data, which is the original image data that is not masked, may have different values. As one embodiment, the masking unit (220) can set the feature map information of the face region of each person extracted through the artificial intelligence analysis unit (210) as an encryption key and mask the face region of each person in the first image data using a pre-stored algorithm. As one embodiment, the masking method may be an encryption method. As one embodiment, the pre-stored algorithm may be a scrambling encryption algorithm. As one embodiment, the storage unit (230) may store feature map information of each person's face region extracted through the artificial intelligence analysis unit (210) and second image data in which each person's face region is masked. As one embodiment, the communication unit (240) may receive face image data to be restored from the user terminal (300). As one embodiment, the unmasking unit (250) may, upon receiving face image data to be restored from the user terminal (300), restore a face region from the masked second image data that has feature map information matching the feature map information extracted from the face image data to be restored.Specifically, when restoration target face image data is received from a user terminal (300) through a communication unit (240), the artificial intelligence analysis unit (210) can extract feature map information from the restoration target face image data using a pre-stored artificial intelligence learning model. As one embodiment, the masking removal unit (250) can determine whether there is feature map information among the feature map information of each person's face region extracted through the artificial intelligence analysis unit (210) that matches the feature map information extracted from the restoration target face image data. As one embodiment, if the masking removal unit (250) determines that there is feature map information among the feature map information of each person's face region extracted through the artificial intelligence analysis unit (210) that matches the feature map information extracted from the restoration target face image data, the face region can be restored (decrypted) using the feature map information extracted from the restoration target face image data as an encryption key. As one embodiment, the pre-stored algorithm may be a scrambling encryption algorithm. As one embodiment, the storage unit (230) can store third image data, which is image data in which only a specific face has been restored (decoded). Through this, only the face of a specific person can be restored from a masked image used as evidence for a specific event, while the faces of the remaining people remain masked, and by selectively restoring only the face of a specific person, the personal information of the remaining people can be protected. As one embodiment, the masking release unit (250) can transmit a message that restoration is impossible to the user terminal (300) through the communication unit (240) if it is determined that there is no feature map information among the feature map information of each person's face region extracted through the artificial intelligence analysis unit (210) that matches the feature map information extracted from the face image data to be restored.In one embodiment, the user terminal (300) can be connected to the server (200) via an app or the web and can display on the screen a first image data which is original image data, a second image data in which the face area of ​​all people in the image is masked, and a third image data in which only the face requested by the user to be restored is unmasked (restored). In one embodiment, the user terminal (300) may be a digital device equipped with memory means and equipped with a microprocessor to have computational capabilities, such as a mobile communication terminal, a desktop computer, a laptop computer, a workstation, a palmtop computer, a personal digital assistant (PDA), a web pad, etc. The problem to be solved

[0005] The conventional technology configured as described above masks the faces of people included in the video data transmitted by a camera installed in a specific area and restores the faces of specific people requested by the user, thereby protecting the personal privacy of those whose faces are not restored; however, it cannot prevent the leakage of information such as passwords and apartment numbers by hacking the video of an entrant entering an apartment or front door and pressing a password on a keyboard included in the door lock. Furthermore, another objective of the present invention is to mask a specific front door area while allowing the remaining video to be visible, thereby freeing oneself from the issue of invasion of privacy by others and eliminating the potential for criminal use. means of solving the problem

[0007] The present invention, a system for auto-masking an image of a front door shape using deep learning object information having the above-mentioned purpose, is characterized by comprising: an image acquisition device that captures a front door and transmits it to a server; a server that stores image information 1 received from the image acquisition device and can generate a machine learning algorithm or a deep learning algorithm based on the shape of the front door included in the image information 1 and an image of a product attached to the front door, thereby extracting feature map information of the front door and setting the extracted feature map information of each front door area as an encryption key, and using a pre-stored algorithm to mask the area of ​​each front door in the image information 1 and store the masked image information 2, and transmits the masked image information 2 of the corresponding front door to the corresponding user's smart terminal when there is a user request; and an administrator smart terminal that requests information about the corresponding front door from the server and receives and provides the image information of the corresponding front door that is masked from the server. Effects of the invention

[0009] The present invention, configured as described above, has the effect of preventing the password of a door lock from being hacked and used for criminal purposes. In addition, by applying masking to a specific area of ​​the front door and allowing the rest to display the image, the present invention has the effect of being free from the problem of invasion of privacy by others and eliminating the potential for misuse for criminal purposes. Brief explanation of the drawing

[0011] FIG. 1 is a configuration diagram of a masking processing system that auto-masks and de-masks surveillance camera images in real time using conventional deep learning object information. FIG. 2 is a system configuration diagram for auto-masking a front door shape image using deep learning object information of the present invention. FIG. 3 is a detailed configuration diagram of a server applied to the present invention, Figure 4 is a control flowchart for an auto-masking method for an entrance door shape image using deep learning object information according to the present invention. Specific details for implementing the invention

[0012] The system for auto-masking a front door shape image using deep learning object information and the method for auto-masking using the same, having the above-mentioned purpose according to the present invention, are described as follows based on FIGS. 2 to 4.

[0014] FIG. 2 is a system configuration diagram for auto-masking an image of a front door shape using deep learning object information of the present invention. A system for auto-masking a front door shape image using deep learning object information according to the present invention in upper respiratory tract 2 comprises: an image acquisition device (10) that captures a front door and transmits it to a server; a server (20) that stores image information 1 (image acquired by the image acquisition device) received from the image acquisition device and can generate a machine learning model or a deep learning model based on the shape of the front door included in the image information 1 and an image of a product attached to the front door, and using this to detect the corresponding front door area in the image information 1, extract feature map information of the corresponding front door area from the detected corresponding front door area, set the extracted feature map information of the corresponding front door area as an encryption key, and uses a pre-stored machine learning algorithm or deep learning algorithm to mask the corresponding front door area included in the image information 1, stores the masked image information 2 (masked image), and transmits the masked image information 2 of the corresponding front door to the user's smart terminal when there is a user request; and a server that requests information regarding the corresponding front door from the server and receives from the server the image of the corresponding front door area that is masked It is characterized by being composed of an administrator smart terminal (30) that receives and provides video information. In the above, the machine learning learning model or deep learning learning model may be a machine learning learning algorithm or a deep learning learning algorithm. The algorithm used to extract a feature map of the detected front door area may be a Convolutional Neural Network algorithm among deep learning algorithms, and the Convolutional Neural Network algorithm may include a feature extraction area and a class classification area configured in a form in which a synthesis layer and pooling are stacked multiple times.In addition, extracting feature map information of the front door area detected above and setting the extracted feature map information of the front door area as an encryption key, and masking the front door area included in the image information 1 using a pre-stored machine learning algorithm or deep learning algorithm, involves applying a feature point extraction algorithm to the detected front door area to extract feature points of the front door area, storing the operation result of the front door area as image map information, applying an encryption algorithm to the stored image map to perform a masking operation, and storing the masked image information which is the result of the operation.

[0016] Figure 3 is a detailed configuration diagram of a server applied to the present invention. In the upper section 3, the server (20) to which the present invention is applied comprises: a storage unit (22) that stores image information 1 received from an image acquisition device and transmits the stored image information 1 to an artificial intelligence analysis unit, and receives and stores image information 2 in which the corresponding front door is masked from an artificial intelligence masking unit; an artificial intelligence analysis unit (24) that receives image information 1 from the storage unit and stores it in memory, generates a machine learning model or a deep learning model based on the shape of the front door included in the image information 1 and the product image attached to the front door, stores it in memory, detects the corresponding front door area using the model, and transmits image information 1 including the detected front door area information to the artificial intelligence masking unit; and extracts feature map information of the corresponding front door area based on the detected front door area information received from the artificial intelligence analysis unit, sets the extracted feature map information of the corresponding front door area as an encryption key, masks the corresponding front door area included in the image information 1 using a pre-stored machine learning algorithm or deep learning algorithm, and the image information 2 in which the corresponding front door area is masked. It is characterized by being composed of an artificial intelligence masking unit (26) that transmits to a storage unit (22) and a request determination unit (28) that determines whether there is a request for information of the corresponding front door area from a user terminal and, if there is a request for information, receives image information 2 with the corresponding front door area masked from the storage unit (22) and transmits it to the user terminal.

[0018] Figure 4 is a control flowchart for an auto-masking method for an entrance door shape image using deep learning object information of the present invention. The method for auto-masking an entrance door shape image using deep learning object information according to the present invention in Figure 4 comprises the steps of: an image acquisition device photographing an apartment or office entrance and transmitting the captured image information to a server (S11); a server detecting a single or multiple entrance door areas using an artificial intelligence learning model that is pre-built and stored in the image information of the apartment or office entrance received from the image acquisition device (S12); a server extracting feature map information from the detected single or multiple entrance door areas (S13); a server setting the extracted feature map of the single or multiple entrance door areas as an encryption key and masking the entrance door areas in the image information using a pre-stored algorithm by applying the set encryption key (S14); a server storing image information in which the entrance door areas are masked (S15); a user smart terminal requesting image verification from the server (S16); a server receiving the image verification request information transmitting the masked entrance door area image information to the user smart terminal (S17); and the user smart terminal [retrieves] the single or multiple The method is characterized by including the step (S18) of providing image information in which the front door area is masked. The step (S12) of detecting a single or multiple front door areas using an artificial intelligence learning model that is pre-built and stored in the image information of an apartment or office entrance is to create an artificial intelligence learning model using an algorithm that is already stored based on the shape of the front door and image information of an attached product mounted on the front door, and to detect a single or multiple front door areas using this model. Additionally, the algorithm that is already stored may be a machine learning learning algorithm or a deep learning learning algorithm.In addition, in the step (S13) of extracting feature map information of the single or multiple front door areas detected by the server, the algorithm used to extract the feature map information of the single or multiple front door areas may be a convolutional neural network algorithm among deep learning algorithms, and the convolutional neural network algorithm may include a feature extraction area and a class classification area configured in a form in which a synthesis layer and a pooling layer are stacked multiple times. In addition, the step (S14) of setting the feature map of the single or multiple front door areas extracted by the server as an encryption key and applying the set encryption key to mask the front door areas in the image information using a pre-stored algorithm involves storing the result of the feature point extraction operation of the front door areas as image map information, applying an encryption algorithm to the stored image map to perform a masking operation, and storing the masked image information which is the result of the operation. Explanation of the symbols

[0020] 10 : Image acquisition device, 20 : Server, 30: User smart terminal

Claims

Claim 1 In a system for auto-masking an image of a front door shape using deep learning object information that can prevent a door lock password from being hacked and used for criminal purposes, the system for auto-masking an image of a front door shape using deep learning object information comprises: an image acquisition device (10) that photographs a front door and transmits it to a server; a system that stores image information 1 received from the image acquisition device and can generate a learning model based on the shape of the front door included in the image information 1 and an image of a product attached to the front door, and can use this to detect the corresponding front door area in the image information 1, extract feature map information of the corresponding front door area from the detected corresponding front door area, set the extracted feature map information of the corresponding front door area as an encryption key, and can use a pre-stored Convolutional Neural Network algorithm that includes a feature extraction area and a class classification area configured by stacking multiple layers of synthesis layers and pooling layers to mask the corresponding front door area included in the image information 1, stores the masked image information 2, and sends the masked image information 2 of the corresponding front door to the user's smart terminal. A storage unit (22) that stores image information 1 received from an image acquisition device by transmitting and transmits the stored image information 1 to an artificial intelligence analysis unit, and receives and stores image information 2 in which the front door is masked from an artificial intelligence masking unit, and receives image information 1 from the storage unit and stores it in memory, and the shape of the front door included in the image information 1,A server (20) comprising: an artificial intelligence analysis unit (24) that generates a Convolutional Neural Network algorithm based on a product image attached to a front door, stores it in memory, uses it to detect the front door area, and transmits image information 1 containing the detected front door area information to an artificial intelligence masking unit; an artificial intelligence masking unit (26) that extracts feature map information of the front door area based on the detected front door area information received from the artificial intelligence analysis unit, sets the extracted feature map information of the front door area as an encryption key, uses a pre-stored Convolutional Neural Network algorithm to mask the front door area included in the image information 1, and transmits image information 2 with the front door area masked to a storage unit (22); and a request determination unit (28) that determines whether there is a request for information of the front door area from a user terminal, and if there is a request for information, receives image information 2 with the front door area masked from the storage unit (22) and transmits it to the user terminal; and information regarding the front door to the server A system for auto-masking a front door shape image using deep learning object information, characterized by comprising an administrator smart terminal (30) that requests and receives and provides image information 2 in which the front door area is masked from a server. Claim 2 delete Claim 3 delete Claim 4 A system for auto-masking a front door shape image using deep learning object information according to claim 1, wherein the server applies a feature point extraction algorithm to a detected front door area to extract feature points of the front door area, stores the operation result of the front door area as image map information, applies an encryption algorithm to the stored image map to perform a masking operation, and stores the masked image information which is the operation result. Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 A method for auto-masking an image of a front door shape using deep learning object information to prevent a door lock password from being hacked and used for criminal purposes, wherein the method for auto-masking an image of a front door shape using deep learning object information comprises: a step (S11) in which an image acquisition device photographs an apartment or office entrance and transmits the captured image information to a server; a step (S12) in which the server creates an artificial intelligence learning model using an algorithm already stored based on the shape of the front door and image information of an attached product mounted on the front door, and detects a single or multiple front door area using the model; a step (S13) in which the server extracts feature map information from the detected single or multiple front door areas; a step (S14) in which the server stores the result of the feature point extraction operation of the front door area as image map information, performs a masking operation by applying an encryption algorithm to the stored image map, and stores the masked image information which is the result of the operation; a step (S15) in which the server stores the image information in which the front door area is masked; and a step in which a user smart terminal requests image verification from the server. A method for auto-masking a front door shape image using deep learning object information, characterized by comprising: a step (S16); a step (S17) in which a server receiving image verification request information transmits image information of the masked front door area to the user's smart terminal; and a step (S18) in which the user's smart terminal provides image information in which a single or multiple front door areas are masked. Claim 9 delete Claim 10 A method for auto-masking a front door shape image using deep learning object information, characterized in that, in the step (S13) of extracting feature map information of a single or multiple front door area detected by the server, the algorithm used to extract the feature map information of the single or multiple front door area may be a convolutional neural network algorithm among deep learning algorithms. Claim 11 delete

Images