Apparatus and method for de-identifying personal information

Abstract

Provided is a personal information de-identification apparatus, comprising: a reception unit (100) for receiving a video image from a video acquisition device; a performance degradation region determination unit (200) for determining a performance degradation region in which an object among objects in the video image is partially concealed; and a de-identification processing unit (300) for de-identifying the object of the performance degradation region determined by the performance degradation region determination unit.

Description

Device and method for anonymizing personal information

[0001] The present invention relates to a device and method for de-identifying personal information, and more specifically, to a device and method for de-identifying personal information that can prevent degradation of de-identification performance caused by partial occlusion by predetermining an area where partial occlusion occurs and processing de-identification in that area.

[0002] In accordance with legal requirements for the protection of personal information, video data must be stored, managed, used, and transmitted after undergoing de-identification processing. Here, de-identification refers to a series of processes that transform some or all of personal information so that specific individuals cannot be identified. De-identification methods include data deletion, which removes some identifying elements, data masking, which makes important identifiers invisible, and blurring, which obscures them.

[0003] In particular, the anonymization of video data captured by autonomous vehicles, CCTVs, etc., is a very important process required by law. For anonymization processing, personal information areas are generally extracted frame by frame from the video data, the extracted personal information areas are processed by masking or blurring, and then encrypted or encoded for storage or transmission to an external device.

[0004] However, if part of the object to be de-identified is obscured, there is a problem where the de-identification device recognizes it as not being a de-identified object, and thus de-identification does not proceed.

[0005] Figure 1 is an example illustrating the problem caused by such partial occlusion in two cases.

[0006] Referring to FIG. 1, first, a case (1) in which an object to be de-identified, such as a person, is obscured at the outer boundary of the screen, and a case (2) in which an object to be de-identified is obscured by an obstacle object that is not a background, such as a utility pole. In this case, a problem occurs in which the object to be de-identified is exposed because the object to be de-identified is not properly recognized.

[0007] Therefore, there is a need to develop de-identification devices and methods to solve these problems.

[0008] Therefore, the problem that the present invention aims to solve is to provide a de-identification device and method capable of solving the problem caused by partial occlusion of an object to be de-identified.

[0009] To solve the above problem, the present invention provides a personal information de-identification device comprising: a receiving unit (100) that receives an image from an image acquisition device; a performance degradation area determining unit (200) that determines a performance degradation area that causes partial occlusion of an object among the objects of the image; and a de-identification processing unit (300) that de-identifies an object of the performance degradation area determined by the performance degradation area determining unit.

[0010] In one embodiment of the present invention, the performance degradation area is a preset area at the outer edge of the image or a preset area around an obstacle object that does not correspond to the background within the image.

[0011] In one embodiment of the present invention, if the performance degradation area is a preset area of ​​the outer edge of the image, the de-identification processing unit deletes the preset area from the image and enlarges the image with the preset area deleted to the size of the image.

[0012] In one embodiment of the present invention, if the performance degradation area is an obstacle object that does not correspond to the background within the image, the de-identification processing unit performs de-identification processing on a preset area around the obstacle object.

[0013] In one embodiment of the present invention, if the performance degradation area is an obstacle object that does not correspond to the background within the image, the de-identification processing unit lowers the object recognition threshold or raises the object recognition confidence in a preset area around the obstacle object.

[0014] In one embodiment of the present invention, when the performance degradation area is an obstacle object that does not correspond to the background within the image, the personal information de-identification device further includes the obstacle object identification unit.

[0015] In one embodiment of the present invention, the obstacle object identification unit identifies obstacle objects among the image images using an object segmentation model.

[0016] The present invention also provides a method for de-identifying personal information, performed by the personal information de-identification device described above, comprising: receiving an image from an image acquisition device; determining a performance degradation area that causes partial occlusion of an object among the objects of the image; and de-identifying an object of the performance degradation area determined by the performance degradation area determination unit.

[0017] In one embodiment of the present invention, the performance degradation area is a preset area at the outer edge of the image or a preset area around an obstacle object that does not correspond to the background within the image.

[0018] In one embodiment of the present invention, the range of the performance degradation area may be variable.

[0019] In order to solve the problem caused by partial occlusion of an image acquired from an image acquisition device such as a CCTV, the present invention can effectively prevent the problem of exposure of an unidentified object due to partial occlusion by predetermining the area where partial occlusion occurs in the image and processing de-identification in that area.

[0020] Figure 1 is an example illustrating the problem caused by such partial occlusion in two cases.

[0021] FIG. 2 is a block diagram of a personal information de-identification device according to one embodiment of the present invention.

[0022] FIGS. 3 and 4 are drawings illustrating a de-identification processing method in Example 1 of the present invention.

[0023] Figures 5 and 6 are drawings illustrating a de-identification processing method in Example 1 of the present invention.

[0024] FIG. 7 is a step diagram of a de-identification processing method that can be performed by a personal information de-identification device according to one embodiment of the present invention.

[0025] Figures 8 and 9 are the original image and the result image to which the de-identification processing techniques of Examples 1 and 2 have been applied, respectively.

[0026] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. Such embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be modified from one embodiment to another without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each embodiment may be modified without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not meant to be limiting, and the scope of the invention should be understood to encompass the scope claimed by the claims and all equivalents thereof.

[0027] Hereinafter, in order to enable a person skilled in the art to easily practice the present invention, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0028] Generally, video image data acquired from autonomous vehicle cameras, drone cameras, CCTVs, etc., undergoes de-identification processing before being transmitted to other devices, including external servers, or stored and managed in memory. Video data, having undergone personal information detection and de-identification processing frame by frame, is then encrypted and encoded for storage, management, utilization, and transmission. Video data processing, including detection and de-identification, can be performed by a processor embedded in the autonomous vehicle's camera, but is not limited to this; it may also be executed by a separate device mounted within the vehicle or by a server outside the vehicle.

[0029] In addition, the de-identification device according to the present invention is a variety of devices including (1) a communication device such as a communication modem for performing communication with various devices or wired / wireless networks, (2) a memory for storing various programs and data, and (3) a microprocessor for executing programs to perform calculations and control. According to at least one embodiment, the memory may be a computer-readable recording / storage medium such as Random Access Memory (RAM), Read Only Memory (ROM), flash memory, optical disk, magnetic disk, or Solid State Disk (SSD).

[0030] In addition, according to one embodiment of the present invention, a microprocessor may be programmed to selectively perform one or more operations and functions described in the specification. According to at least one embodiment, the microprocessor may be implemented wholly or partially as hardware or a combination of hardware such as an Application Specific Integrated Circuit (ASIC), CPU, GPU, digital signal processor (DSP), or field programmable gate array (FPGA) of a specific configuration.

[0031] In addition, the image acquisition device is a device for acquiring images (e.g., road driving images), and may be a camera of an autonomous vehicle, a black box camera of a general vehicle, a CCTV, etc., but may also be a device that collects images captured by another physically separated device via wired or wireless communication. The image acquisition device may include an image sensor module such as a CCD, memory, a processor, and a wired or wireless communication module.

[0032] In addition, objects subject to de-identification processing (hereinafter referred to as objects subject to de-identification) are personal information areas such as faces and license plates, and the detection of personal information areas can be determined by various algorithms, such as a face recognition engine or a number recognition engine, for the divided objects after object segmentation. Meanwhile, when personal information areas such as human faces or license plates are detected, de-identification processing is performed by a processor, which is carried out in various ways such as masking, blurring, replacement, and deletion, and objects that are not determined to be objects subject to de-identification after object segmentation are exposed as they are without undergoing de-identification processing.

[0033] However, if the object to be de-identified is not properly identified at an obstacle or boundary, the problem of personal information being exposed as is occurs, as explained in Fig. 1.

[0034] Accordingly, the present invention provides a de-identification device to solve the problem caused by partial occlusion of an image acquired from an image acquisition device such as a CCTV.

[0035] FIG. 2 is a block diagram of a personal information de-identification device according to one embodiment of the present invention.

[0036]

[0037] * Referring to FIG. 2, a personal information de-identification device according to one embodiment of the present invention includes: a receiving unit (100) that receives an image captured from an image acquisition device (not shown), such as a CCTV; a performance degradation area determining unit (200) that determines an area (hereinafter referred to as 'performance degradation area') in which partial obscuration of an object to be de-identified occurs among the objects of the image to de-identify may degrade the performance of the de-identification device; and a de-identification processing unit (300) that de-identifies an object within the performance degradation area determined by the performance degradation area determining unit (200).

[0038] In the present invention, the performance degradation area is an area where partial occlusion of the object to be de-identified is highly likely to occur, and Example 1) the edge area around the received image (original image); Example 2) the area around an object (hereinafter obstacle object) where partial occlusion of the object to be de-identified occurs and the object to be de-identified may be partially exposed.

[0039] In other words, the present invention can improve anonymization performance by resolving problems caused by errors in identifying objects to be anonymized without reducing real-time processing efficiency, by predetermining a performance degradation area and lowering the threshold or increasing the confidence of recognized objects within that area, or by performing pre-anonymization processing regardless of the presence or absence of objects.

[0040] Furthermore, de-identification processing includes methods such as de-identifying personal information through masking, blurring, or mosaic processing; expanding the scope of de-identification processing by lowering the threshold for determining de-identified objects within a performance degradation area from a previously set value; or increasing confidence within the aforementioned performance degradation area. Additionally, the degree of such threshold or confidence adjustment can be pre-set automatically or manually. Alternatively, it may be varied automatically or manually depending on the level of personal information protection desired by the user, which will be explained in more detail below.

[0041] The de-identification processing methods of Example 1 and Example 2 will be explained in more detail below.

[0042] FIGS. 3 and 4 are drawings illustrating a de-identification processing method in Example 1 of the present invention.

[0043] Referring to Fig. 3, a preset area at the edge border of the received original image is first determined as the 'performance degradation area'.

[0044] In Fig. 3, the black area becomes a performance degradation area pre-set from the edge of the original image toward the inside of the image.

[0045] Referring to Fig. 4, the edge portion of the black area in Fig. 3 is deleted, and then the image with the performance degradation area deleted is expanded to the size of the original image.

[0046] Figures 5 and 6 are drawings illustrating a de-identification processing method in Example 2 of the present invention.

[0047] Referring to FIG. 5, first, non-background objects (obstacle objects) in the image are identified. To this end, a de-identification processing device according to one embodiment of the present invention may further include an obstacle object identification device.

[0048] In FIG. 5, a utility pole becomes an obstacle object. As an obstacle object recognition method, a pre-trained segmentation model, a panoptic segmentation model, a depth prediction model, etc., may be used, and other known background-object separation recognition technologies may be used for obstacle object identification of the present invention.

[0049] For example, since obstacles have a different depth from the background, obstacle objects can be distinguished using depth thresholds within the image. Alternatively, obstacles can be identified using a deep learning-based object detector, which can be implemented as a multilayer artificial intelligence neural network constructed by methods such as MLP (multilayer perceptron), CNN (convolutional newral network), or RNN (recurrent newral network).

[0050] Referring to FIG. 6, the preset area (20) around the obstacle object becomes the 'performance degradation area' in the present invention. That is, the performance degradation area is an area where partial obscuration of an object to be de-identified may occur by the obstacle object (10), and for the object in that area, it is possible to lower the object recognition threshold, raise the object recognition confidence, or directly mask or blur the object within that area.

[0051] FIG. 7 is a step diagram of a de-identification processing method that can be performed by a personal information de-identification device according to one embodiment of the present invention.

[0052] Referring to FIG. 7, a de-identification processing method according to one embodiment of the present invention includes: a step of receiving an image from an image acquisition device; a step of determining a performance degradation area that causes partial occlusion of an object among the objects of the image; and a de-identification processing step of de-identifying an object in the performance degradation area determined by the performance degradation area determination unit.

[0053] The above-described de-identification processing step may be performed by pre-processing objects within the corresponding performance degradation area through blurring or masking, by lowering the threshold value for determining objects to be de-identified to expand the range of de-identified objects only within the performance degradation area, or by increasing object detection confidence, as described above. Furthermore, in the de-identification device and method according to the present invention, the range of the performance degradation area or the degree of de-identification may be varied according to the level of personal information protection. For example, in cases where personal information is extremely sensitive, the range of the performance degradation area may be set widely (expansion of the performance degradation area range), or methods such as further lowering the threshold value or increasing object detection confidence (enhanced de-identification processing) may be used. In addition, such variation can be performed automatically or manually.

[0054] Figures 8 and 9 are the original image and the result image to which the de-identification processing techniques of Examples 1 and 2 have been applied, respectively.

[0055] Referring to Figures 8 and 9, it can be seen that the area around the utility pole, which is an obstacle object rather than a background, and the edge area of ​​the original image become a performance degradation area, and the object (person) within that area is de-identified (mosaic-processed).

[0056] As described above, the de-identification device and method according to the present invention can effectively prevent performance degradation caused by partial occlusion of objects in image de-identification using an object segmentation model.

[0057] The present invention is an apparatus and method for anonymizing personal information, and its industrial applicability is recognized.

Claims

1. As a personal information anonymization device, A receiving unit (100) that receives an image from an image acquisition device; A performance degradation area determining unit (200) that determines a performance degradation area that causes partial occlusion of an object among the objects of the above image; and A personal information de-identification device comprising a de-identification processing unit (300) that de-identifies objects of a performance degradation area determined by the above performance degradation area determination unit.

2. In Paragraph 1, A personal information de-identification device characterized in that the above-mentioned performance degradation area is a preset area at the outer edge of the image or a preset area around an obstacle object that does not correspond to the background within the image.

3. In Paragraph 2, A personal information de-identification device characterized in that, when the performance degradation area is a preset area on the outer edge of the image, the de-identification processing unit deletes the preset area from the image and enlarges the image with the preset area deleted to the size of the image.

4. In Paragraph 2, A personal information de-identification device characterized in that, when the above-mentioned performance degradation area is an obstacle object that does not correspond to the background within the image, the de-identification processing unit performs de-identification processing on a preset area around the obstacle object.

5. In Paragraph 4, A personal information de-identification device characterized in that, when the above-mentioned performance degradation area is an obstacle object that does not correspond to the background within the image, the de-identification processing unit lowers the object recognition threshold or raises the object recognition confidence in a preset area around the obstacle object.

6. In Paragraph 2, If the above-mentioned performance degradation area is an obstacle object that does not correspond to the background within the image, the personal information de-identification device is characterized by further including the obstacle object identification unit.

7. In Paragraph 6, The above obstacle object identification unit is characterized by identifying obstacle objects among the above image using an object segmentation model.

8. A method of de-identifying personal information performed by a personal information de-identification device pursuant to Paragraphs 1 through 7, A step of receiving an image from an image acquisition device; A step of determining a performance degradation area that causes partial occlusion of an object among the objects of the above-mentioned image; and A method for de-identifying personal information, comprising the step of de-identifying an object of a performance degradation area determined by the performance degradation area determination unit.

9. In Paragraph 8, A method for de-identifying personal information, characterized in that the above-mentioned performance degradation area is a preset area at the outer edge of the image or a preset area around an obstacle object that does not correspond to the background within the image.

10. In Paragraph 9, A method for de-identifying personal information characterized in that the range of the above-mentioned performance degradation area can be varied.

Images