Method and system for de-identification processing of personal information on basis of mobile device

Abstract

A system for de-identification processing of personal information on the basis of a mobile device comprises: an image collection device; a mobile device which detects a personal information area from an original image obtained from the image collection device and performs de-identification processing on the personal information area to generate a de-identified image; a server which is connected to the mobile device through a communication network, receives the de-identified image from the mobile device and stores same, and provides the de-identified image according to a request of a user device; and the user device which accesses the server through the communication network to receive and decode the de-identified image.

Description

Method and System for De-identification of Personal Information Based on Mobile Devices

[0001] The present invention relates to a method and system for processing personal information anonymization, and more specifically, to a mobile device having an embedded anonymization module and a method and system for processing personal information anonymization based on the mobile device.

[0002] Videos generated and collected by various video collection devices, such as portable video recorders, mobile phones, car dashcams, CCTVs, body cams, smart home cameras, drones, action cams, and autonomous vehicle cameras, contain personal information. Users acquire videos directly or indirectly from these devices as needed and utilize them as important data in various fields, including marketing, big data, and artificial intelligence.

[0003] However, since the aforementioned videos contain personal information, they must be stored, managed, transmitted, and used after undergoing de-identification processing, such as masking, so that others cannot identify them, in accordance with legal requirements for the protection of personal information. As such, the de-identification of video data is a very important process required by law, and as the fields in which personal information is utilized increase, the risk of personal information leakage also increases, so it is protected under the Personal Information Protection Act.

[0004] Storing video containing personal information in the local storage of a video collection device is not a violation of the Personal Information Protection Act. However, distributing video content stored in local storage, or transmitting and saving it to another remote storage, constitutes a violation of the Personal Information Protection Act. Personal information within the video, such as people's faces and vehicle license plates, must be anonymized to render it unidentifiable before distribution or transmission.

[0005] Traditionally, video collection devices stored un-anonymized original footage, allowing only authorized administrators to access it. Alternatively, upon a request for footage from an external device, an administrator would individually access the requested footage offline to perform anonymization processing before providing it. However, for systems such as standard CCTVs or autonomous vehicles, cameras operate continuously due to their operational characteristics, making it difficult to store data on local storage. Consequently, a method is employed where video data is transmitted in real-time to a server or cloud for storage.

[0006] As such, video generated and collected by various video collection devices contains personal information; however, these devices lack the technical means to perform anonymization, making it difficult to carry out efficient anonymization. Furthermore, in the case of CCTVs, data must be transmitted to servers for anonymization processing while bearing the risk of personal information exposure, which makes the transmission process vulnerable to hacking. In other words, the inability to efficiently and safely perform anonymization on existing scattered video collection devices has resulted in problems involving reduced costs and compromised security regarding personal information protection.

[0007] (Patent Document 1) KR 2020-0036656 A

[0008] The present invention aims to solve the above problems and provide a method for performing a de-identification operation in real time on an image acquisition device, storing the de-identified image as an image that can be restored using a related key value, and protecting image content from illegal access from the outside.

[0009] The present invention aims to provide a method for managing video data cost-effectively while enabling the protection of personal information.

[0010] The present invention also aims to provide a method for managing image data that enables efficient access and use of the image data by persons with legitimate authority.

[0011] To solve the above problem, a method and system for processing personal information de-identification based on a mobile device according to one aspect of the present invention are provided.

[0012] The above-described mobile device-based personal information de-identification processing system comprises: an image collection device; a mobile device that detects a personal information area in an original image acquired from the image collection device and performs de-identification processing on the personal information area to generate a de-identified image; a server connected to the mobile device via a communication network, receives and stores the de-identified image from the mobile device, and provides it upon a request from a user device; and a user device that accesses the server through an authentication procedure to receive and decrypt the de-identified image.

[0013] The mobile device further includes an encryption module that encrypts the metadata of the de-identified image using an encryption key, and

[0014] The mobile device transmits the de-identified image, the metadata file encrypted by the encryption module, the encryption key, and the master key to the server.

[0015] The user device provides login information and a related key value to the server to download the de-identified image. The server performs validation using the login information and the related key value, and upon completion of validation, transmits the de-identified image and the encrypted metadata file to the user device in response to the image lookup or restoration request queue.

[0016] The above server includes an authentication unit, and the authentication unit performs validity verification by checking whether a relevant key value matches a request queue from a user device.

[0017] It is preferable that the above-mentioned related key value includes an encryption key for encrypting the metadata of the anonymized image data and a master key for anonymization.

[0018] The detected personal information area information includes coordinate information of the personal information area per frame of the original image, and the metadata includes the coordinate information of the personal information area and a master key for de-identification.

[0019] The above personal information area is de-identified using a shuffling method with a shuffling table, and the above master key is an index of the above shuffling table.

[0020] The mobile device may be positioned adjacent to the image acquisition device and acquires the original image generated from the image acquisition device in real time.

[0021] According to another aspect of the present invention, a mobile device is provided that generates a de-identified image by de-identifying an original image and transmits it to a server.

[0022] The mobile device comprises an image collection module that collects and stores the original image from the image collection device, a de-identification module that detects personal information areas in the original image and performs de-identification processing to generate a de-identified image, an encryption module that encrypts the metadata of the de-identified image using an encryption key, a communication module for communicating with external devices through a communication network, and a control module that controls the components of the mobile device.

[0023] The mobile device is positioned adjacent to the image acquisition device and acquires the original image from the image acquisition device, and transmits the de-identified image, a metadata file encrypted by the encryption module, the encryption key of the metadata file, and a master key for de-identification to the server.

[0024] According to another aspect of the present invention, a method for processing an image is provided for execution in a mobile device-based personal information de-identification processing system. The method comprises: a step in which a mobile device receives an original image from an image collection device; a de-identification step in which the mobile device detects a personal information area in the original image and de-identifies it to generate a de-identified image; an encryption step in which the mobile device encrypts the meta-information of the de-identified image with an encryption key to generate a meta-information file; a step in which the mobile device transmits the de-identified image, the meta-information file, the encryption key, and a master key to a server; and a step in which the server stores the de-identified image and the meta-information file in memory and stores the encryption key and the master key in a database.

[0025] The above master key is used to de-identify the above original image into the above de-identified image, and the above metadata includes de-identification information.

[0026] According to one aspect of the present invention, since an image in which personal information has been anonymized is transmitted from a mobile device to a server, the load on the server can be reduced, and since the anonymized image is stored on the server, the requirements of the Personal Information Protection Act are satisfied.

[0027] According to another aspect of the present invention, efficient and active de-identification processing can be performed locally regardless of the status of the image generation device, and authorized users can restore the image to its original state, thereby providing excellent field adaptability and enhanced security.

[0028] Figure 1 is a conceptual diagram schematically illustrating the structure and data flow of the mobile device-based image personal information de-identification system of the present invention.

[0029] FIG. 2 is a block diagram showing the structure of a mobile device and a server of a mobile device-based image personal information de-identification system according to an embodiment of the present invention.

[0030] FIG. 3 is a schematic diagram illustrating an image processing method in a de-identification module and an encryption module of a mobile device according to an embodiment of the present invention.

[0031] FIG. 4 is a schematic diagram showing image processing steps in a mobile device-based image personal information de-identification system according to an embodiment of the present invention.

[0032]

[0033] Hereinafter, in order to enable a person skilled in the art to easily practice the present invention, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0034] Terms such as "first," "second," etc., used in the description of this specification are merely identifiers to distinguish one component from another. When a component is referred to in this specification as being "connected" or "connected" to another component, it should be understood that the component may be directly connected to or directly connected to the other component, but unless otherwise specifically stated, it may also be connected or connected through another component in between.

[0035] Among the terms used in this specification, "module," "part," "interface," etc., generally refer to computer-related objects and may refer, for example, to hardware, software, and combinations thereof. In this specification, an information processing device is a device capable of processing data and communicating with external devices, comprising a processor, memory, and a communication module, and may be a computer, PC, set-top box, smartphone, etc.

[0036] Referring to FIG. 1, a mobile device-based personal information de-identification processing system according to a preferred embodiment of the present invention includes an image collection device (300), a mobile device (100) connected to the image collection device (300) via wired or wireless communication, a server (200), and a user device (400).

[0037] The above image collection device (300) is an information processing device that generates or collects image data, and may be a portable video recording device, mobile phone, car black box, CCTV, body cam, smart home camera, drone camera, action cam, autonomous vehicle camera, etc. that generates or collects original image data.

[0038] The mobile device (100) detects a personal information area in the original image acquired from the image collection device (300) and performs de-identification processing on the personal information area to generate a de-identified image. The server (200) is connected to the mobile device via a communication network, receives and stores the de-identified image from the mobile device, and provides it upon the request of the user device. The user device (400) is an information processing device used by a user who wishes to receive and use the de-identified image, and accesses the server via a communication network to receive and decode the de-identified image.

[0039] The mobile device (100) may be positioned adjacent to a local image acquisition device (300) and may acquire original images generated from the image acquisition device in real time via wired or wireless communication. The mobile device (100) is a device separate from the image acquisition device (300), such as a camera, and may be, for example, a smartphone, a set-top box connected to a CCTV to collect image data acquired from the CCTV, but is not limited thereto. The mobile device (100) is a movable information processing device and, in terms of hardware, includes a processor, memory, an input / output interface, and a communication module.

[0040] Referring to FIG. 2, the mobile device (100) includes an image collection module (101), a de-identification module (102), an encryption module (103), a control module (107), a storage module (104), and a communication module (105), and each of the above components may be implemented by software and / or hardware.

[0041] The image acquisition module may be a component that collects and stores original images from an image acquisition device.

[0042] The de-identification module (102) detects personal information areas, which are areas subject to de-identification, for each image frame in the collected original image and performs de-identification processing. To detect personal information areas, the de-identification module (102) may include various algorithms such as an artificial intelligence learning model, a face recognition engine, or a number recognition engine. The target for de-identification processing is a personal information area such as a face or a license plate number in an image frame, and the detection of personal information areas is performed by various algorithms such as an artificial intelligence learning model, a face recognition engine, or a number recognition engine. Meanwhile, when a personal information area such as a human face or a license plate number is detected, de-identification processing is performed by a processor, which can be done in various ways such as masking, scrambling, shuffling, or blurring. Here, for convenience, a shuffling method is adopted, but the present invention is not limited thereto.

[0043] The de-identification module (102) stores the de-identified image and metadata in the storage module. The metadata is de-identification information, such as de-identification area information (coordinates or pixel information of the personal information area de-identified per frame, etc.) and a master key. Here, the master key is a key value capable of restoring the de-identified image, and may be, for example, a shuffle index, but is not limited thereto. The shuffle index is information of the shuffle order (table) used when shuffling the de-identification target area by shuffling. According to one embodiment of the present invention, de-identification processing can be performed using a shuffling method in which the personal information area of ​​an image frame is divided into a predetermined shuffle area, and pixel values ​​within the shuffle area are mixed together using a shuffle table to create a fake copy. At this time, the selected shuffle table is extracted as an index calculated using the de-identification key value. That is, the index is calculated using the de-identification key value, and shuffling is performed by the shuffle table according to this index. There can be various shuffling orders, which are tabulated, and each order is indexed, so if you know this index, you can restore the original.

[0044] Referring to FIG. 3, the encryption module (103) encrypts the meta information using an encryption key and stores it as a meta information file. The control module transmits the encrypted meta information file and the de-identified image to the server through the communication module. At this time, the encryption key and the master key can be transmitted to the server together.

[0045] It is preferable that the encryption key value and the anonymization master key value of the above meta-information file be entered or specified by the user in advance when the original image data is generated, produced, or acquired, but is not limited thereto and may be automatically generated or entered during the anonymization process, and the user may acquire them.

[0046] The server (200) is connected to one or more mobile devices (100) and one or more user devices (400) via a communication network and may be an information processing device that processes and stores data from one or more mobile devices (100) and one or more user devices (400) and manages access and use of the user devices (400).

[0047] According to FIG. 2, the server (200) includes a control unit (207), an authentication unit (202) that performs login and key validity checks from a user device, a database (203), a memory (207), and a communication unit (205).

[0048] Referring to FIGS. 1, 2, and 4, a server (200) receives a de-identified image, an encrypted metadata file, an encryption key, and a master key from a mobile device through a communication module (205). The server stores the de-identified image and the encrypted metadata file in memory, and stores the encryption key and the master key in a database.

[0049] Through the aforementioned process, de-identified video and metadata files, rather than the original video, are transmitted from the mobile device to the server, so there is no risk of the original video being exposed.

[0050] Meanwhile, the user can access the server through the user device (400) to download the de-identified image and metadata file. First, the user performs a login procedure to the server through the user device (400). The login procedure follows a general authentication verification procedure and is therefore not described in detail. After logging in, when the encryption key and master key are transmitted from the user device to the server, the authentication unit of the server performs a validity check. That is, to download the de-identified image and metadata file from the server, the authorized user first inputs the master key and encryption key from the user device and transmits them to the server. The authentication unit of the server checks whether the master key and encryption key stored in the database match the received master key and encryption key, and if the validity check is successful, the control unit (207) transmits the de-identified image and the corresponding metadata file to the user device through the communication unit. If the validity check fails, the server does not provide the de-identified image and the corresponding metadata file to the user device.

[0051] Meanwhile, the user device (400) can perform original image restoration using a de-identified image and a corresponding metadata file.

[0052] The user device (400) may be an information processing device used by an administrator or user who has access and processing rights to the image.

[0053] A method for managing image data according to an embodiment of the present invention will be described in detail with reference to FIG. 4.

[0054] Referring to Fig. 4, the user proceeds with the management service subscription process after undergoing an authentication process to use the video data management service provided by the operating server through the user device. Since this identity verification and subscription process is the same as the general identity verification and membership registration process, a detailed explanation is omitted.

[0055] The user device extracts and restores the encrypted metadata file for each image frame of the received video using an encryption key. The restored metadata file contains metadata including de-identification information. The user device restores the de-identified video frame by frame using a list of de-identified regions and a master key. At this time, the list of de-identified regions includes coordinate information of the de-identified regions for each frame, and the master key includes a shuffling index.

[0056] A mobile device-based de-identification method is described with reference to Fig. 4.

[0057] The mobile device acquires the original image from the image acquisition device (Step 1).

[0058] The mobile device detects personal information areas in the image and performs de-identification processing (Step 2). At this time, de-identification processing is performed using a master key, and a metadata file containing the de-identification information is encrypted with an encryption key (Step 3). The original image may be stored separately.

[0059] The mobile device transmits the de-identified image, the encrypted metadata file, the encryption key, and the master key to the server. The server stores the received de-identified image and the encrypted metadata file, and stores the encryption key and the master key in the database (Step 4).

[0060] The user device accesses the server through a login procedure. The server receives a request from the user device for a specific video lookup or restoration containing a related key value. When the encryption key and the de-identification master key of the metadata file are entered through the user device, the server performs validation (step 5).

[0061] The authentication unit of the server determines whether the aforementioned related key value matches, and if the related key value matches, creates a request queue containing device information regarding the image acquisition device and the related key value. That is, it checks whether the key value associated with specific de-identified image data received from the user device matches the related key value stored in the database; if they match, it creates a request queue, and if they do not match, it sends a message to the user device rejecting the request for viewing or restoring the specific image.

[0062] The aforementioned related key values ​​include the meta file key value and the anonymization key value of the anonymized image data. Here, the encryption key is the key value used to encrypt the meta information file. The meta information file includes personal information area information detected by the image acquisition device (coordinates and pixel information of the personal information area per frame of the original image data).

[0063] Regarding the generated request queue, the server authentication unit re-verifies the relevant key value included in the request queue. That is, it checks the relevant key value and the unique ID of the anonymized image data (or original image data) that match the relevant key value included in the request queue and the unique ID of the image data. If a complete match is confirmed, the control unit requests and downloads the corresponding anonymized image data; if there is no match, it notifies of the mismatch again, and the restoration of the original image data is stopped.

[0064] Once validation is complete, the server transmits the de-identified image and metadata files to the user device.

[0065] The user device restores the de-identified image using the metadata file.

[0066] Therefore, users with administrative privileges can easily download the video remotely, restore it to the original, and view it.

[0067] The above method for restoring an unidentified image includes the steps of decrypting a meta-information file using a meta-file encryption key, obtaining unidentified information including personal information area information from the decrypted meta-information, and restoring the original image from the unidentified image using the unidentified information and a master key.

[0068] However, the de-identification processing of the present invention is not limited to a shuffling method using a shuffling table, and can be performed using various other recoverable methods, such as masking and blurring.

[0069] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented in software executed by hardware, or implemented by a combination thereof. The software may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, hard disk, removable disk, CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0070] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive.

[0071] The present invention relates to image processing and is applicable in the field of information and communication, thus having industrial applicability.

Claims

1. As a mobile device-based personal information anonymization processing system, Image acquisition device; A mobile device that detects a personal information area in an original image acquired from the above-mentioned image collection device and performs de-identification processing on the said personal information area to generate a de-identified image; A server connected to the mobile device via a communication network, receiving and storing the de-identified image from the mobile device and providing it upon a request from the user device; and It includes a user device that accesses the server via a communication network to receive and decrypt the anonymized image, and The mobile device above is, An image acquisition module that collects and stores the original image from the image acquisition device, A de-identification module that detects personal information areas in the above original image data and generates a de-identified image by performing de-identification processing using a master key, A control module that controls each component of the above-mentioned mobile device, and A mobile device-based personal information anonymization processing system characterized by including a communication module for communicating with external devices through a communication network.

2. In Paragraph 1, The mobile device further includes an encryption module that encrypts the metadata of the de-identified image using an encryption key, and A mobile device-based personal information de-identification processing system characterized by the mobile device transmitting the de-identified image, a metadata file encrypted by the encryption module, the encryption key, and the master key to the server.

3. In Paragraph 1 or 2, The user device provides login information and related key values ​​to the server to download the de-identified image, and A mobile device-based personal information de-identification processing system characterized by the above server performing validation using the above login information and related key values, and when validation is completed, transmitting the above de-identified image and encrypted metadata file to a user device in response to a video viewing or restoration request queue.

4. In Paragraph 3, The above server includes an authentication unit, and The above authentication unit performs validity verification by checking whether a relevant key value matches the request queue from the user device, and A mobile device-based personal information de-identification processing system characterized by the above-mentioned related key value including an encryption key for encrypting metadata of de-identified image data and a master key for de-identification.

5. In Paragraph 2, The above-mentioned detected personal information area information includes coordinate information of the personal information area for each frame of the original image, and The above meta-information includes coordinate information of the above personal information area and a master key for anonymization, and A mobile device-based personal information de-identification processing system characterized in that the above personal information area is de-identified by a shuffling method using a shuffling table, and the above master key is an index of the above shuffling table.

6. In Paragraph 1, The mobile device may be positioned adjacent to the image collection device, and A mobile device-based personal information anonymization processing system characterized by acquiring original images generated from the above-mentioned image collection device in real time.

7. A mobile device that generates a de-identified image by processing the original image to be de-identified and transmits it to a server, An image acquisition module that collects and stores the original image from the image acquisition device, A de-identification module that detects personal information areas in the above original image and performs de-identification processing to generate a de-identified image, An encryption module that encrypts the metadata of the above-mentioned de-identified image using an encryption key, A communication module for communicating with external devices through a communication network and It includes a control module that controls the components of the mobile device, and A mobile device characterized by being positioned adjacent to a video acquisition device, acquiring the original video from the video acquisition device, and transmitting the de-identified video, a metadata file encrypted by the encryption module, an encryption key for the metadata file, and a master key for de-identification to the server.

8. A method for processing images executed in a mobile device-based personal information anonymization processing system of paragraph 1, A step in which a mobile device receives an original video from a video collection device; A de-identification step in which a mobile device detects a personal information area in the original image and de-identifies it to generate a de-identified image; An encryption step in which a mobile device encrypts the metadata of the de-identified image using an encryption key to generate a metadata file; A step in which a mobile device transmits the de-identified image, the metadata file, the encryption key, and the master key to a server; and The method includes the step of the server storing the de-identified image and the metadata file in memory and storing the encryption key and master key in a database; The above master key is used to de-identify the above original image into the above de-identified image, and An image processing method characterized in that the above meta-information includes de-identification information.

9. In Paragraph 8, An authentication step in which the above server receives the encryption key and master key entered into the user device and performs validation; A step of transmitting the de-identified image and metadata file to a user device when the verification is completed on the server above; and An image processing method characterized by further including the step of decoding the metadata file on the user device to obtain metadata and restoring the de-identified image using the metadata.

10. In Paragraph 9, The mobile device may be positioned adjacent to the image collection device, and An image processing method characterized by acquiring an original image generated from the above image acquisition device in real time.

Images