System and method for managing de-identified image data of personal information

Abstract

This system for managing de-identified image data of personal information comprises: an image collection device for detecting a personal information area from collected original-image data, and generating de-identified image data by de-identifying the personal information area; and an operating server, which is connected to the image collection device through a communication network, receives the de-identified image data from the image collection device and stores same, and provides same according to a request of the image collection device, wherein the image collection device is an information processing device for collecting image data, and includes: an image collection unit for generating or collecting original-image data; a personal information area detection unit for detecting a personal information area from the original-image data; a de-identification unit for generating de-identified image data by de-identifying the personal information area; a restoration unit for restoring the original-image data from the de-identified image data by using a key value; a control unit for controlling each constituent element of the image collection device; and a communication unit for communicating with external devices through the communication network.

Description

Personal information de-identification video data management system and method

[0001] The present invention relates to a method for managing personal information de-identified image data, and more particularly, to a system and method for managing personal information de-identified image data and an image restoration method.

[0002] Personal information is being used as crucial data in diverse fields, including marketing, big data, and artificial intelligence. As the scope of personal information utilization expands, the risk of personal information leakage also increases, and thus, it is protected by the Personal Information Protection Act. Video data, in particular, requires special attention, as its exploitation using deepfake technology poses significant human rights violations.

[0003] Images and video information can be acquired through various video collection devices, such as CCTV and smartphones. Storing this collected video content on the local storage of the video collection device does not violate the Personal Information Protection Act. However, distributing the video content stored on the local storage device or transmitting and storing it from the collection device to another remote storage device is a violation of the Personal Information Protection Act. Personal information (such as human faces or license plates) contained in the content must be de-identified before distribution or transmission.

[0004] Accordingly, video data containing personal information is stored in anonymized form in the video collection device at the edge, and only the administrator can access and download the video. When there is a request for video from an external device, the requested video is de-identified and provided. In cases where video acquired from CCTV in an apartment or kindergarten needs to be exported externally, the administrator (an authorized person) and the de-identification worker are present together, and the original video from the storage device is de-identified and copied to a removable storage device before being exported externally. Another method, which is to store on a remote server, is to de-identify the streamed video and store and manage both the original and de-identified video.

[0005] The above methods have the problem that workers must always work on site and that management costs increase because both original and de-identified images are stored.

[0006] Meanwhile, autonomous vehicles are agricultural vehicles such as tractors and rice transplanters, or vehicles capable of indoor and outdoor cleaning, capable of autonomous driving and operation. Due to the nature of autonomous vehicles, they continuously acquire and collect video footage, which may contain personal information. Video footage captured from edge devices such as autonomous vehicles has limitations in being stored on local storage due to the constant operation of the camera. Therefore, data must be directly deleted from local storage or stored on remote storage, and must be de-identified before being stored / transmitted. However, because de-identified footage is transmitted instead of the original footage, and the original footage must be deleted, the problem arises where the original footage cannot be viewed when required.

[0007] (Patent Document 1) KR 2020-0036656 A

[0008] The present invention aims to solve the above problems by providing a method for performing an anonymization operation in real time in an image collection device, storing the anonymized image as an image that can be restored using a related key value, and protecting the image content from illegal access from the outside.

[0009] The present invention seeks to provide a method for managing image data cost-effectively while protecting personal information.

[0010] The present invention also seeks to provide a method for managing image data that allows a person with legitimate authority to efficiently access and use the image data.

[0011] To solve the above problem, a personal information de-identification image data management system according to one aspect of the present invention is provided. The de-identification image data management system includes an image collection device that detects a personal information area from collected original image data and performs de-identification processing on the personal information area to generate de-identified image data; and an operation server that is connected to the image collection device via a communication network, receives the de-identified image data from the image collection device, stores it, and provides it upon request from the image collection device.

[0012] The above image collection device is an information processing device that collects image data, and preferably includes an image collection unit that generates or collects original image data, a personal information area detection unit that detects a personal information area in the original image data, a de-identification unit that de-identifies the personal information area to generate de-identified image data, a restoration unit that restores the original image data from the de-identified image data using a key value, a control unit that controls each component of the image collection device, and a communication unit that communicates with external devices through a communication network.

[0013] The above image collection device transmits device information and related key values ​​together with the de-identified image data to the operation server, and deletes the de-identified original image data and the de-identified image data.

[0014] The above device information is generated based on the MAC address and serial number of the image collection device. The operating server includes a data storage unit, a control unit, an authentication unit, an input / output interface, and a communication module, and the data storage unit includes a program storage unit and a database. The database stores the device information and related key values ​​together with the de-identified image data. It is preferable that the authentication unit performs authentication using the device information and related key values.

[0015] The above image data management system may further include a user device. The user device is an information processing device used by a user with management or access rights to the image collection device, and provides device information and related key values ​​to the operation server to retrieve the original image data.

[0016] The above operation server executes authentication using the above device information, and when authentication is completed, receives a specific image search or restoration request including a related key value from the user device, determines whether the key value matches, and if the key value matches, creates a request queue including device information and related key value for the image collection device.

[0017] The above image collection device receives a request queue matching the device information, checks whether the related key value matches, receives the matching de-identified image data from the operation server, and restores the original image data from the de-identified image data using the related key value.

[0018] The above device information is the MAC address and serial number of the encrypted image collection device, and the related key value includes the meta file key value of the de-identified image data and the de-identification key value.

[0019] The above image collection device may further include an authentication verification unit.

[0020] The above authentication verification unit checks whether the relevant key value matches the request queue from the operation server, and if it matches, the control unit downloads the de-identified image data from the operation server and controls the restoration unit to restore the original image data.

[0021] The above-detected personal information area information includes coordinates and pixel information of the personal information area for each frame of the original image data, and the coordinates and pixel information of the personal information area are stored in a metafile, and the metafile is encrypted using a metafile key value. The personal information area is preferably anonymized using a shuffling table in a shuffling manner, and the anonymization key value is preferably an index of the shuffling table.

[0022] The above operation server further includes a device registration unit that registers device information of one or more image collection devices to which a user has access authority, in response to a request from a user device.

[0023] The above device information is generated based on the MAC address and serial number of the image collection device, and is an encrypted value obtained by inputting the value obtained by adding the MAC address and serial number (combineString=Mac address+serial Number) into a hash algorithm.

[0024] According to another aspect of the present invention, a method for managing de-identified image data is provided, which is executed by an operating server that receives and stores de-identified image data in which personal information is de-identified from an image collection device.

[0025] The above-described method for managing de-identified image data includes a step of receiving and storing de-identified image data together with device information and related key values ​​from an image collection device; an authentication step of receiving an authentication request including device information from a user device and executing authentication; a step of receiving a specific image search or restoration request including related key values ​​from the user device when the authentication is completed; and a step of determining whether the related key values ​​match, and if the related key values ​​match, generating a request queue including device information and related key values ​​for the image collection device.

[0026] The above-described method for managing de-identified image data may further include a step of providing the de-identified image data to the image collection device in response to a request for de-identified image data based on confirmation of a match between related key values ​​of the image collection device; and a step of receiving restored original image data from the image collection device and providing the restored original image data to the user device.

[0027] The above device information is the MAC address and serial number of the encrypted image collection device, and the related key value includes the meta file key value and the de-identification key value of the de-identified image data. The meta file of the de-identified image data, in which the personal information is de-identified using the meta file key value, is encrypted. The meta file includes the coordinates and pixel information of the personal information area for each frame of the original image data.

[0028] The above personal information area is preferably anonymized by shuffling using a shuffling table, and the anonymization key value is preferably an index of the shuffling table.

[0029] According to one aspect of the present invention, an efficient and privacy-enhanced image data management method is provided.

[0030] According to another aspect of the present invention, an image data management method and system that are efficient and have enhanced security are provided by enabling the storage of de-identified image data that can be restored using device information and related key values ​​of an image collection device in a remote storage, and enabling the image to be restored from the device that originally acquired the image upon a restoration request from the image owner.

[0031] Figure 1 is a schematic structural diagram of the image data management system of the present invention.

[0032] FIG. 2 is a block diagram showing the structure of an image collection device according to one embodiment of the present invention.

[0033] Figure 3 is a block diagram showing the structure of an operating server according to one embodiment of the present invention.

[0034] FIG. 4 is a schematic diagram showing a pre-device information registration process in a video data management method according to one embodiment of the present invention.

[0035] FIG. 5 is a schematic diagram showing the sequence and data transmission process of an image data management method according to one embodiment of the present invention.

[0036] FIG. 6 is a diagram illustrating a method for de-identifying image data according to one embodiment of the present invention.

[0037]

[0038] Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

[0039] Terms such as "first," "second," etc. used throughout the description of this specification are merely identifiers used to distinguish one component from another. When a component is referred to as "connected" or "connected" to another component in this specification, it should be understood that the component may be directly connected or connected to the other component, but unless otherwise specifically stated, it should also be understood that the component may be connected or connected via another component in between.

[0040] The terms "module," "part," and "interface" used herein generally refer to computer-related objects, and may refer to hardware, software, and combinations thereof, for example. In this specification, an information processing device includes a processor, memory, and a communication module, and may be a computer, PC, set-top box, smartphone, or the like, capable of processing data and communicating with external devices.

[0041] Typically, video image data acquired from cameras and other devices undergoes de-identification processing before being transmitted to other devices, including external servers, or stored and managed in memory. Video data that undergoes frame-by-frame detection and de-identification processing is then encrypted and encoded before being stored, managed, utilized, and transmitted. De-identification targets personal information areas such as faces and vehicle license plates, and detection of these areas is accomplished using various algorithms, such as artificial intelligence learning models, facial recognition engines, or number recognition engines. Meanwhile, when personal information areas such as human faces or vehicle license plates are detected, de-identification is performed by the processor, which can utilize various methods, such as masking, scrambling, blurring, categorization, substitution, and deletion.

[0042] Referring to FIG. 1, an image data management system according to a preferred embodiment of the present invention includes an image collection device (100), a user device (200), and an operation server (300) that are connected to each other via a communication network.

[0043] The image collection device (100) is an information processing device that collects image data, and in terms of hardware, includes a processor, memory, an input / output interface, and a communication module. The image collection device (100) may include an image acquisition device such as a camera, but is not limited thereto, and may also be a device that is connected to a separate image acquisition device and collects image data from it. For example, the image collection device may be a smartphone, a set-top box that is connected to a CCTV and collects image data acquired from the CCTV, or an ECU system of an autonomous vehicle.

[0044] Referring to FIG. 2, the structure of the image collection device (100) includes an image collection unit, a personal information area detection unit, an anonymization unit, a restoration unit, an authentication confirmation unit, a control unit, and a communication unit, and each of the above components can be implemented by software and / or hardware.

[0045] The video collection unit collects video data, and may be a camera, but is not limited thereto, and may also be a component that collects and stores video data from a separate device. The personal information area detection unit detects personal information areas, which are areas subject to de-identification, from the collected video data, and may include various algorithms such as an artificial intelligence learning model, a facial recognition engine, or a number recognition engine. The detected personal information area information (coordinates of the personal information area for each frame, pixel information, etc.) is stored in a metafile, and the metafile is encrypted using a metafile key value.

[0046] Metafile keys and de-identification keys can be specified by the user, who is the image data manager. While it is preferable for the security key to be entered or specified by the user in advance when the original image data is created, produced, or acquired, it is not limited to this and can also be automatically generated or entered during the de-identification process, and the user can obtain it.

[0047] The de-identification unit performs de-identification processing on the personal information area of ​​the image data detected by the personal information area detection unit using a de-identification key value. However, the de-identified image data can be restored using the de-identification key value. According to one embodiment of the present invention, de-identification processing can be performed using a shuffling method in which the personal information area is divided into predetermined shuffle areas as shown in FIG. 6 and the pixel values ​​within the shuffle areas are shuffled using a shuffle table to create a fake copy. At this time, the selected shuffle table is extracted with an index calculated using the de-identification key value. In other words, an index is calculated using the de-identification key value, and shuffling is performed using a shuffle table according to this index. The shuffling method will be described in detail later.

[0048] The authentication verification unit verifies the authentication and key value matching for the request queue from the operation server. If there is a match, the control unit downloads the de-identified video data from the operation server and enables the restoration unit to restore the original video data. The communication unit communicates with external devices via a communication network.

[0049] The user device (200) may be an information processing device used by an administrator or user who has access and processing authority for the image data of the image collection device (100).

[0050] The operation server (300) is connected to one or more image collection devices (100) and one or more user devices (200) through a communication network, and may be an information processing device that processes and stores data from one or more image collection devices (100) and one or more user devices (200) and manages access to and use of the image collection devices (100) by the user devices (200).

[0051] According to FIG. 3, an operating server (300) according to one embodiment of the present invention includes a data storage unit, a control unit, an authentication unit, an input / output interface, a communication module, and a device registration unit. The data storage unit includes a program file storage unit and a database, and the database stores de-identified image data, related device information, and key values.

[0052] The above device registration unit registers device information of an image collection device (100) to which a user has access authority upon a request from a user device. The device information is generated based on the Mac address and serial number of the image collection device (100). The device information may be, for example, a value obtained by adding the MAC address and serial number of the image collection device (100) (combineString = Mac address + serial Number) and inputting it into a hash algorithm and encrypting it (licenseString = hash algorithm (combinString)), but is not limited thereto.

[0053] The authentication unit performs authentication processing based on the device information in response to an authentication request from a user device.

[0054] Hereinafter, a method for managing image data according to one embodiment of the present invention will be described in detail with reference to FIGS. 4 and 5.

[0055] Referring to Figure 4, a user uses the user authentication service to access the video data management service provided by the operating server via their device. After authenticating themselves, they then proceed with the management service sign-up process. These authentication and sign-up procedures are identical to the standard user authentication and membership registration process, so detailed explanations are omitted.

[0056] Afterwards, the user proceeds with the procedure of registering the management device (video collection device) for which he / she has management authority. When registering the management device, the MAC address and serial number of the video collection device are registered so that the device can be distinguished. However, the MAC address and serial number are encrypted on the user device and transmitted to the operation server. The encryption is performed using the hash algorithm as described above. If the user has multiple video collection devices for which he / she has management authority, he / she can proceed with the registration procedure for each video collection device. That is, the MAC address and serial number of each video collection device can be encrypted and a request for device registration can be made to the operation server. The device registration section of the operation server stores the device information of the video collection device (100) for which the user has access / management authority according to the registration request from the user device. The key value can be composed of 16 digits of numbers + uppercase / lowercase alphabets using an advanced standard encryption technique based on the AES (Advanced Encryption Standard) algorithm.

[0057] Referring to FIG. 5 below, a method for managing image data according to an embodiment of the present invention will be described in detail.

[0058] The image collection device (100) detects personal information areas in the collected images and performs de-identification processing to generate de-identified image data. The generated de-identified image data, along with device information (the MAC address and serial number of the encrypted image collection device) and related key values ​​(meta file key value and de-identification key value), are transmitted to the operation server. Since the image collection device (100) continuously collects image data, it has limitations in storage capacity at the edge. Therefore, if necessary, both the de-identified image data and the original image data can be deleted, and only the related key value and the de-identified image data (or original image data) unique number can be stored. New image data is continuously collected, the personal information areas are extracted, de-identified, and transmitted to the operation server.

[0059] According to FIG. 5, the operation server receives de-identified image data and device information (encrypted MAC address and serial number of the image collection device) and related key values ​​(meta file key value and de-identified key value) from the image collection device and stores them as structured data in the data storage unit.

[0060] Meanwhile, a user with legitimate authority over a specific video collection device can access the original video collected by the video collection device or request the original video data through the user device. In other words, the user can request video viewing or restoration from the operating server through the user device. In this case, an authentication procedure is first performed. The authentication procedure involves the user device transmitting authentication information containing device information to the operating device, and the authentication unit of the operating server performs authentication using the authentication information. During this authentication step, the authentication unit first checks whether the device information transmitted from the user device (i.e., the MAC address and serial number of the encrypted video collection device) matches the device information in the database. If the matching device information is found, the authentication unit then performs a second check to see if the user device has registered the device information stored in the device registry. However, this second check is not mandatory and is optional, further enhancing security. This is because management authority is granted only if the user device has previously registered the device information of the video collection device managed by the user.

[0061] Once authentication is complete, the operation server receives a specific image search or restoration request including the relevant key value from the user device. The authentication unit of the operation server determines whether the relevant key value matches, and if the relevant key value matches, it creates a request queue including device information and the relevant key value for the image collection device. That is, it checks whether the key value related to the specific anonymized image data received from the user device matches the relevant key value stored in the database, and if it matches, it creates a request queue. If it does not match, it sends a message to the user device rejecting the specific image search or restoration request.

[0062] The above-mentioned related key value includes a meta file key value of the de-identified image data and a de-identification key value. Here, the meta file key value is a key value used by the image collection device to encrypt the meta file of the de-identified image data. The meta file includes personal information area information (coordinates and pixel information of the personal information area for each frame of the original image data) detected by the image collection device. The de-identification key value is a de-identification key value used by the de-identification unit of the image collection device to de-identify the personal information area using a shuffling method.

[0063] For the above-mentioned generated request queue, the image collection device only receives request queues with matching device information (), and the authentication confirmation unit re-verifies the related key value included in the request queue. That is, the related key value included in the request queue and the related key value that matches the unique number of the image data and the unique number of the de-identified image data (or the original image data) are checked. If a complete match is confirmed, the control unit requests the corresponding de-identified image data from the operation server and downloads it. If there is no match, the mismatch is notified to the operation server again, and the restoration of the original image data is stopped. Since the restoration work is performed only in the image collection device, and the related key value is double-checked in the operation server and the image collection device, security is likely to be improved more than when the image data is restored in the operation server.

[0064] Once the authentication verification unit of the video collection device reconfirms the relevant key value, the control unit receives the de-identified video data from the operation server and controls the restoration unit to restore the original video data using the relevant key value. The restored original video data from the video collection device is then transmitted to the user device that requested the video view or restoration via the operation server. Therefore, users with administrative authority can easily view or obtain the original video data remotely.

[0065] Below, anonymization and restoration using the shuffling method are described with reference to Fig. 6. The shuffling method is an anonymization processing method that creates fake copies using a shuffling table.

[0066] The shuffling method includes a step of calculating a shuffle index using a de-identified key value, a step of selecting a shuffle table corresponding to the shuffle index, a shuffling step of shuffling the personal information area using the selected shuffle table, and a step of encrypting de-identified information including the coordinates and number of pixels of the personal information area using a meta file key value and storing the encrypted information as a meta file.

[0067] The method for restoring the above-described de-identified image data includes a step of decrypting a metafile of the de-identified image data using a metafile key value, a step of obtaining personal information area information from the decrypted metafile, a step of separating a personal information area of ​​the de-identified image data using the personal information area information, a step of restoring the separated personal information area using a de-identification key value, and a step of combining the restored personal information area with the de-identified image data to generate original image data.

[0068] However, the anonymization processing of the present invention is not limited to a shuffling method using a shuffling table, and may be performed in various ways, such as other restorable methods, such as masking and blurring.

[0069] The steps of a method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented in software executed by hardware, or implemented by a combination thereof. The software may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present invention pertains.

[0070] While the embodiments of the present invention have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive.

[0071] The present invention relates to an image data management system and method, and is applicable to the information and communication industry, and thus has industrial applicability.

Claims

1. As a video data management system, An image collection device that detects a personal information area from collected original image data and performs anonymization processing on the personal information area to generate anonymized image data; and An operation server, which is connected to the image collection device through a communication network, receives and stores the de-identified image data from the image collection device, and provides it upon request from the image collection device; The above image collection device is an information processing device that collects image data. An image collection unit that generates or collects original image data; A personal information area detection unit that detects personal information areas in the above original image data, A de-identification unit that de-identifies the above personal information area to create de-identified image data. A restoration unit that restores original image data from de-identified image data using key values. A control unit that controls each component of the above image collection device, and An image data management system characterized by including a communication unit that communicates with external devices through a communication network.

2. In paragraph 1, The above image collection device transmits the device information and related key values together with the de-identified image data to the operation server, and deletes the de-identified original image data and the de-identified image data. The above device information is generated based on the MAC address and serial number of the image collection device. The above operation server includes a data storage unit, a control unit, an authentication unit, an input / output interface, and a communication module, and the data storage unit includes a program storage unit and a database, and the database stores the device information and related key values together with the de-identified image data. An image data management system characterized in that the authentication unit performs authentication using the above device information and related key values.

3. In paragraph 1 or 2, The above image data management system further includes a user device, The above user device is an information processing device used by a user who has management or access rights to the image collection device, and provides device information and related key values to the operation server to search the original image data. The above operation server executes authentication using the above device information, and when authentication is completed, it receives a specific image search or restoration request including a related key value from the user device and determines whether the key value matches, and if the key value matches, it creates a request queue including device information and related key value for the image collection device, An image data management system characterized in that the image collection device receives a request queue matching the device information, checks whether the related key value matches, receives the matching de-identified image data from the operation server, and restores the original image data from the de-identified image data using the related key value.

4. In paragraph 2 or 3, The above device information is the MAC address and serial number of the encrypted video collection device. The above related key values include the meta file key value of the de-identified image data and the de-identified key value, The above image collection device further includes an authentication verification unit, A video data management system characterized in that the above authentication verification unit checks whether the related key value matches the request queue from the operation server, and if it matches, the control unit downloads the de-identified video data from the operation server and controls the restoration unit to restore the original video data.

5. In any one of paragraphs 1 to 4, The above-detected personal information area information includes coordinates and pixel information of the personal information area for each frame of the original image data, and the coordinates and pixel information of the personal information area are stored in a meta file, and the meta file is encrypted using a meta file key value. An image data management system, characterized in that the above personal information area is anonymized in a shuffling manner using a shuffling table, and the anonymization key value is an index of the shuffling table.

6. In any one of paragraphs 2 to 5, The above operation server further includes a device registration section for registering device information of each video collection device to which the user has access rights according to a request from the user device. An image data management system characterized in that the above device information is generated based on the MAC address and serial number of the image collection device, and is an encrypted value obtained by inputting the value obtained by adding the MAC address and the serial number (combineString=Mac address+serial Number) into a hash algorithm.

7. A method for managing de-identified image data executed by an operating server that receives and stores de-identified image data from an image collection device, A step of receiving and storing device information and related key values together with de-identified image data from an image collection device; An authentication step that receives an authentication request including device information from a user device and executes authentication; Upon completion of the above authentication, a step of receiving a specific image search or restoration request including a relevant key value from the user device; and A method for managing de-identified image data, comprising: a step of determining whether the related key values match, and if the related key values match, generating a request queue including device information and related key values for the image collection device.

8. In paragraph 7, A step of providing the de-identified image data to the image collection device in response to a request for de-identified image data based on confirmation of the match between the relevant key values of the image collection device; and A method for managing de-identified image data, comprising: a step of receiving original image data restored from the image collection device and providing the same to the user device.

9. In paragraph 8, The above device information is the MAC address and serial number of the encrypted video collection device. A method for managing de-identified image data, characterized in that the above-mentioned related key value includes a meta file key value of de-identified image data and a de-identified key value.

10. In paragraph 9, The meta file of the de-identified image data, which de-identifies the personal information using the above meta file key value, is encrypted, and the meta file includes coordinates and pixel information of the personal information area for each frame of the original image data, A method for managing de-identified image data, characterized in that the above personal information area is de-identified in a shuffling manner using a shuffling table, and the de-identified key value is an index of the shuffling table.

Images