Apparatus and method for de-identifying personal information

Abstract

The present invention relates to an apparatus and method for processing personal information for de-identification. A method for de-identifying personal information according to one embodiment of the present invention may include a first detection step for detecting and tracking face and skeleton information of a person as a first object from an input image, a second detection step for detecting and tracking information of a vehicle and license plate of a vehicle as a second object from an input image, and a de-identification step for refining duplicate detection results from the results of the first detection and the second detection and blurring the face and license plate of the vehicle.

Description

Technology Field

[0001] The present invention relates to an apparatus and method for processing personal information for de-identification. Background Technology

[0003] Existing face and license plate recognition technologies faced difficulties in data utilization due to performance degradation in various environments and false positives in the models. In particular, issues such as performance degradation regarding small objects, contour intrusion caused by rectangular anonymization, and the sensitivity of license plates were identified. Additionally, problems regarding object omission in continuous images and limitations of standard bounding boxes existed.

[0004] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention. Prior art literature

[0006] Korean Registered Patent Publication No. 10-1861520 (May 21, 2018) The problem to be solved

[0007] One objective of the present invention is to provide a personal information anonymization device and method that overcome the limitations of existing technologies in anonymization operations for personal information protection and can perform anonymization while maintaining high performance stably in various road environments.

[0008] One objective of the present invention is to solve existing problems and realize effective anonymization by improving performance for small objects, preventing intrusion of object contours, improving the accuracy of license plate anonymization, and tracking objects in continuous images.

[0009] The problems that the present invention aims to solve are not limited to those mentioned above, and other problems and advantages of the present invention not mentioned can be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be understood that the problems and advantages that the present invention aims to solve can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0011] A personal information de-identification method according to one embodiment of the present invention is a personal information de-identification method performed by a processor of a personal information de-identification device, and may include a first detection step of detecting and tracking face and skeleton information of a person as a first object from an input image, a second detection step of detecting and tracking information of a vehicle and license plate of a vehicle as a second object from an input image, and a de-identification step of refining duplicate detection results from the results of the first detection and the second detection and blurring the face and license plate of the vehicle.

[0012] A personal information de-identification device according to one embodiment of the present invention comprises at least one processor and a memory operably connected to the processor and storing at least one code executed by the processor, wherein the at least one processor may be configured to perform a first detection of detecting and tracking face and skeleton information of a person as a first object from an input image, perform a second detection of detecting and tracking information of a vehicle and license plate of a vehicle as a second object from an input image, refine the results of duplicate detections from the results of the first detection and the results of the second detection, and perform de-identification by blurring the face and the license plate of the vehicle.

[0013] In addition to this, other methods for implementing the present invention, other systems, and computer-readable recording media storing a computer program for executing said methods may be further provided.

[0014] Other aspects, features, and advantages other than those described above will become clear from the following drawings, claims, and detailed description of the invention. Effects of the invention

[0016] According to the present invention, faces and license plates can be identified while maintaining stable high performance in various environments.

[0017] In addition, it provides face and license plate anonymization with improved accuracy compared to existing methods, thereby ensuring the safety of sensitive personal information.

[0018] In addition, data loss can be minimized and data utilization increased through a post-processing algorithm that refines duplicate detection results.

[0019] In addition, data can be handled safely while maintaining high performance and accuracy, in compliance with privacy and ethical considerations.

[0020] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0022] FIG. 1 is a block diagram illustrated to schematically explain the configuration of a personal information de-identification device according to the present embodiment. Figure 2 is an example diagram illustrating the operation of the personal information de-identification device shown in Figure 1. FIG. 3 is a block diagram illustrated to schematically explain the configuration of a personal information de-identification device according to another embodiment. FIG. 4 is a flowchart illustrating a method for de-identifying personal information according to the present embodiment. Specific details for implementing the invention

[0023] The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, detailed descriptions of related known technologies are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention.

[0024] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Terms such as “first,” “second,” etc., may be used to describe various components, but the components should not be limited by these terms. These terms are used solely for the purpose of distinguishing one component from another.

[0025] Additionally, in this application, "part" may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0026] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. In describing with reference to the attached drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0027] In the following embodiments, the terms first, second, etc. are used not in a restrictive sense, but for the purpose of distinguishing one component from another component.

[0028] In the following embodiments, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0029] In the following embodiments, terms such as "include" or "have" mean that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components may be added.

[0030] Where an embodiment can be implemented differently, a specific process sequence may be performed differently from the order described. For example, two processes described consecutively may be performed substantially simultaneously or proceed in the reverse order of the description.

[0032] FIG. 1 is a block diagram illustrating the configuration of a personal information de-identification device according to the present embodiment, and FIG. 2 is an example diagram illustrating the operation of the personal information de-identification device illustrated in FIG. 1. Referring to FIG. 1 and FIG. 2, the personal information de-identification device (100) may include a first detection unit (110), a second detection unit (120), and a de-identification processing unit (130).

[0033] The personal information de-identification device (100) detects and tracks a first object and a second object from an input image and can blur personal information included in the first object and the second object, such as a person's face or a vehicle's license plate. In the present embodiment, the first object may include a person, and the second object may include a vehicle. The personal information de-identification device (100) can generate an output image containing the result of blurring the personal information.

[0034] In this embodiment, the personal information de-identification device (100) may use an artificial intelligence algorithm to de-identify personal information. Here, artificial intelligence (AI) is a field of computer engineering and information technology that studies methods to enable computers to perform thinking, learning, self-development, etc., which are possible with human intelligence, and may mean enabling computers to imitate human intelligent behavior.

[0035] Furthermore, artificial intelligence does not exist in isolation but is closely related, directly and indirectly, to many other fields of computer science. Particularly in the modern era, there are very active attempts to introduce AI elements into various sectors of information technology and utilize them to solve problems within those fields.

[0036] Machine learning is a field of artificial intelligence that encompasses research areas that empower computers to learn without explicit programming. Specifically, machine learning can be defined as a technology that studies and builds systems and algorithms capable of learning, making predictions, and improving their own performance based on empirical data. Rather than executing strictly defined static program commands, machine learning algorithms may adopt an approach of constructing specific models to derive predictions or decisions based on input data.

[0037] Both unsupervised learning and supervised learning can be used as machine learning methods for such artificial neural networks. In addition, deep learning, a type of machine learning, can learn by descending to deep levels in multiple stages based on data. Deep learning can represent a set of machine learning algorithms that extract key data from multiple data as the stages are increased.

[0038] In this embodiment, the personal information de-identification device (100) may exist independently in the form of a server, or the personal information de-identification device (100) may implement the personal information de-identification function in the form of an application and be installed on a user terminal (not shown).

[0039] Here, the user terminal may include a communication terminal capable of performing the functions of a computing device (not shown), and in addition to a desktop computer, smartphone, and laptop operated by the user, it may be a tablet PC, smart TV, mobile phone, PDA (personal digital assistant), media player, micro server, GPS (global positioning system) device, e-book reader, digital broadcasting terminal, navigation, kiosk, MP3 player, digital camera, home appliance, and other mobile or non-mobile computing device, but is not limited thereto. Furthermore, the user terminal may be a wearable terminal such as a watch, glasses, hair band, and ring equipped with communication functions and data processing functions. Such user terminals are not limited to the above-described contents, and any terminal capable of web browsing may be used without restriction.

[0040] A first detection unit (110) included in a personal information de-identification device (100) can detect and track face and skeleton information of a person as a first object from an input image. In this embodiment, the first detection unit (110) may include a face detection unit (111), a pose prediction unit (112), and a first object tracking unit (113).

[0041] The face detection unit (111) can detect a face of a person as a first object from an input image using a face detection model to which deep learning is applied. The face detection unit (111) can detect the face of a first object from an input image using a first deep neural network model that detects a face from an image. Here, the first deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and labels a face detection result including a first bounding box that encloses the face.

[0042] The face detection unit (111) can train an initially set first deep neural network model using labeled training data in a supervised learning manner. Here, the initially set first deep neural network model is a first initial model designed to be configured as a model capable of detecting faces from images, and the parameter values ​​may be set to arbitrary initial values. As the first initial model is trained through the aforementioned training data, the parameter values ​​are optimized so that it can be completed as a first detection model capable of accurately detecting faces from images.

[0043] In this embodiment, the face detection model may include TinaFace. TinaFace is one of the face detection algorithms and can be used to accurately detect faces by combining multiple deep learning models. The operation of TinaFace is as follows. First, multi-scale features can be acquired from an input image through a feature extractor (not shown). Next, the receptive area can be enhanced through an inception block (not shown), and faces can be detected and positioned through a classification head (not shown) and a regression head (not shown). Finally, the most accurate face can be identified using an IoU Aware Head (not shown), and information about the corresponding face can be returned.

[0044] The pose prediction unit (112) can predict skeleton information of the first object from an input image using a pose estimation model to which deep learning is applied. In this embodiment, the skeleton represents a structure related to the bones of a human or animal, and the extracted data can be used in a behavior recognition model. Skeleton information of the human body is useful for motion pattern and pose analysis, and based on this, human-centered inference can be performed to enable accurate behavior recognition and explanation.

[0045] The pose prediction unit (112) can predict the skeleton information of the first object from the input image using a second deep neural network model that predicts skeleton information from the image. Here, the second deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and skeleton information as a label.

[0046] The pose prediction unit (112) can train an initially set second deep neural network model using labeled training data in a supervised learning manner. Here, the initially set second deep neural network model is a second initial model designed to be configured as a model capable of predicting skeleton information from an image, and the parameter values ​​may be set to arbitrary initial values. As the second initial model is trained through the aforementioned training data, the parameter values ​​are optimized so that it can be completed as a second prediction model capable of accurately predicting skeleton information from an image.

[0047] In this embodiment, the pose prediction model may include either HRNet (high-resolution network) or Cascade R-CNN. HRNet may be equipped with a network structure that effectively integrates information of various resolutions while maintaining high-resolution features. Cascade R-CNN may adopt a method of improving performance by connecting multiple stages of object detection networks.

[0048] The first object tracking unit (113) can generate first characteristic information by combining the face information and skeleton information of the first object by combining the result of detecting the face of the first object output from the face detection unit (111) and the result of predicting the skeleton information of the first object output from the pose prediction unit (112). In the present embodiment, the first characteristic information may include identification information, location, and visual features (e.g., appearance, color, texture, etc.) of the first object.

[0049] The first object tracking unit (113) can track the movement of a first object containing first feature information from a series of images. To track the movement of the first object, the first object tracking unit (113) may primarily use one of optimization-based tracking, convolution filter tracking, herristoric tracking, and deep learning-based tracking.

[0050] Optimization-based tracking treats the movement of a first object as an optimization problem, utilizing positional information between image frames to estimate and predict the object's movement through optimization algorithms. Convolutional filter tracking can track and predict the location of a first object using convolutional filters such as Kalman filters or particle filters. Heristoric tracking can track the first object by utilizing empirical rules and heuristics. Deep learning-based tracking can estimate the location and movement of an object from images using deep learning neural networks.

[0051] A second detection unit (120) included in the personal information de-identification device (100) can detect and track information regarding a vehicle and a license plate of a vehicle as a second object from an input image. In this embodiment, the second detection unit (120) may include an object detection unit (121), a license plate detection unit (122), and a second object tracking unit (123).

[0052] The object detection unit (121) can detect a vehicle as a second object from an input image using an object detection model to which deep learning is applied. The object detection unit (121) can detect a vehicle as a second object from an input image using a third deep neural network model that detects a vehicle from an input image. Here, the third deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and labels a vehicle detection result including a bounding box surrounding the vehicle.

[0053] The object detection unit (121) can train an initially configured third deep neural network model using labeled training data in a supervised learning manner. Here, the initially configured third deep neural network model is a third initial model designed to be configured as a model capable of detecting a vehicle from an image, and the parameter values ​​may be set to arbitrary initial values. As the third initial model is trained through the aforementioned training data, the parameter values ​​are optimized to be completed as a third detection model capable of accurately detecting a vehicle from an image.

[0054] In this embodiment, the object detection model may include Faster R-CNN. Faster R-CNN can be used to rapidly detect various objects within an image. The Faster R-CNN model can operate in conjunction with a region proposal network (RPN) to provide high speed and accuracy. The RPN may include a structure that generates candidate object regions within an image, identifies objects based on these candidate regions, and predicts bounding boxes. Faster R-CNN enables end-to-end learning and possesses robustness against objects of various sizes and types for effective object detection.

[0055] The license plate detection unit (122) can detect a license plate from an image in which a vehicle as a second object is detected using a license plate detection model to which deep learning is applied. The license plate detection unit (122) can detect a license plate included in a bounding box in which a vehicle as a second object is detected using a fourth deep neural network model that detects a license plate from an image containing an object. Here, the fourth deep neural network model may be a model trained in a supervised learning manner using training data that takes as input an image containing an object and the angle of the camera that took the image, and takes as label a text detection result including a bounding box that encloses the text contained in the object.

[0056] The license plate detection unit (122) can train an initially set fourth deep neural network model using labeled training data in a supervised learning manner. Here, the initially set fourth deep neural network model is a fourth initial model designed to be configured as a model capable of detecting a vehicle's license plate from an image containing a vehicle, and the parameter values ​​may be set to arbitrary initial values. As the third initial model is trained through the aforementioned training data, the parameter values ​​are optimized to complete a fourth detection model capable of accurately detecting a license plate included in a bounding box where a vehicle is detected as a second object.

[0057] In this embodiment, the license plate detection model detects a second object detected by the object detection unit (121) and can detect the license plate regardless of the object's orientation by using a rotated object detection model therein. The license plate detection model can detect the license plate regardless of the object's orientation through an algorithm that reflects the object's size and the camera angle. In this embodiment, the license plate detection model may include a large selective kernel network (LSKNet).

[0058] The second object tracking unit (123) can generate second characteristic information by combining the result of detecting a vehicle as a second object output from the object detection unit (121) and the result of detecting a license plate output from the license plate detection unit (122), thereby combining the vehicle and license plate information as a second object. In this embodiment, the second characteristic information may include identification information, location, and visual characteristics (e.g., appearance, color, texture, etc.) of the second object.

[0059] The second object tracking unit (123) can track the movement of a second object containing second feature information from a series of images. To track the movement of the second object, the second object tracking unit (123) may primarily use one of optimization-based tracking, convolution filter tracking, herristoric tracking, and deep learning-based tracking.

[0060] The de-identification processing unit (130) included in the personal information de-identification device (100) can refine duplicate detection results from the results of the first detection and the second detection, and can blur the face and the license plate of the vehicle. The de-identification processing unit (130) can generate an output image in which the face and the license plate of the vehicle are blurred. In this embodiment, the de-identification processing unit (130) may include a post-processing unit (131) and a blurring processing unit (132).

[0061] In this embodiment, post-processing can be used to improve or adjust the results of the deep neural network model used for the detection and tracking of the first and second objects, thereby optimizing the output of the deep neural network model or removing unnecessary information to obtain more accurate and reliable results.

[0062] The post-processing unit (131) can remove duplicate detection results from the first detection result output from the first detection unit (110) and the second detection result output from the second detection unit (120). The blurring processing unit (132) can blur the faces and vehicle license plates included in the first detection result and the second detection result from which duplicate detection results have been removed.

[0063] In one embodiment, the post-processing unit (131) can calculate a confidence score representing the reliability of the deep neural network model applied to generate the results of the first detection and the second detection. Here, the deep neural network model may include one or more of the first to fourth deep neural network models described above. Here, the post-processing unit (131) can obtain a probability value for each class by applying a Softmax function to the output layer of the deep neural network model and use the probability value of the corresponding class as the confidence score. The post-processing unit (131) can remove a confidence score below a threshold value by comparing the confidence score with a preset threshold value. The blurring processing unit (132) can blur the faces and license plates of vehicles included in the results of the first detection and the second detection, from which the confidence score below the threshold value has been removed.

[0064] In another embodiment, the post-processing unit (131) can calculate a confidence score representing the reliability of the deep neural network model applied to generate the results of the first detection and the second detection. Among the results of calculating the confidence scores, the post-processing unit (131) can remove confidence scores that exceed a preset reference distance by calculating the distance to the results of surrounding detections based on the highest confidence score. Here, the distance may include the Euclidean distance. The Euclidean distance is a method of measuring the straight-line distance between two points in space, and in this embodiment, the Euclidean distance may be used to evaluate the location or characteristics of the first and second objects or to measure similarity. The blurring processing unit (132) can blur the faces and license plates of vehicles included in the results of the first detection and the second detection from which confidence scores exceeding the preset reference distance have been removed.

[0065] The personal information de-identification device (100) disclosed in this embodiment can increase efficiency compared to conventional de-identification devices, effectively detect small objects in various environments, and drastically reduce the amount of data loss that occurs during de-identification processing.

[0066] The personal information de-identification device (100) disclosed in this embodiment can be designed to minimize infringement of object ethics and privacy protection, as well as data loss, while generating a dataset. Thus, it is possible to increase the usability of the dataset while complying with privacy protection and ethical considerations. Through this, organizations can safely handle sensitive information, rapidly collect and analyze data, and consequently achieve cost reduction. Furthermore, the personal information de-identification device (100) offers innovative possibilities in various fields that use datasets, and companies and research institutions can utilize data more efficiently.

[0068] FIG. 3 is a block diagram illustrating the configuration of a personal information de-identification device according to another embodiment. In the following description, parts that overlap with the description of FIG. 1 and FIG. 2 will be omitted. Referring to FIG. 3, a personal information de-identification device (100) according to another embodiment may include a processor (140) and a memory (150).

[0069] In this embodiment, the processor (140) can process the functions performed by the first detection unit (110), the second detection unit (120), and the de-identification processing unit (130) disclosed in FIGS. 1 and 2.

[0070] Such a processor (140) can control the operation of the entire personal information de-identification device (100). Here, 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program, for example. Examples of such data processing devices embedded in hardware may include processing devices such as microprocessors, central processing units, processor cores, multiprocessors, ASICs, and FPGAs, but the scope of the present invention is not limited thereto.

[0071] The memory (150) is operably connected to the processor (140) and can store at least one code associated with an operation performed by the processor (140).

[0072] Additionally, the memory (150) can perform the function of temporarily or permanently storing data processed by the processor (140). Here, the memory (150) may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto. Such memory (150) may include internal memory and / or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, non-volatile memory such as OTPROM, PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash memory, or NOR flash memory, flash drives such as SSD, CF card, SD card, Micro-SD card, Mini-SD card, xD card, or Memory Stick, or storage devices such as HDD.

[0074] FIG. 4 is a flowchart illustrating a method for de-identifying personal information according to the present embodiment. In the following description, parts that overlap with the descriptions of FIG. 1 to 3 will be omitted. The method for de-identifying personal information according to the present embodiment will be described under the assumption that the personal information de-identification device (100) performs the task in the processor (140) with the help of surrounding components.

[0075] Referring to FIG. 4, in step S410, the processor (140) can perform a first detection to detect and track face and skeleton information of a person as a first object from an input image. In this embodiment, when performing the first detection, the processor (140) can detect the face of the first object from the input image using a first deep neural network model that detects a face from an image. Here, the first deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and labels a face detection result including a first bounding box that encloses the face. The processor (140) can predict the skeleton information of the first object from the input image using a second deep neural network model that predicts skeleton information from an image. Here, the second deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and labels skeleton information. The processor (140) can generate first characteristic information by combining the result of detecting the face of the first object and the result of predicting the skeleton information of the first object, and can track the movement of the first object including the first characteristic information from a series of images.

[0076] In step S420, the processor (140) can perform a second detection to detect and track information regarding a vehicle and a license plate of a vehicle as a second object from an input image. In this embodiment, the processor (140) can detect a vehicle as a second object from an input image using a third deep neural network model that detects a vehicle from an image. Here, the third deep neural network model may be a model trained in a supervised learning manner using training data that takes an image as input and labels a vehicle detection result including a bounding box surrounding the vehicle. The processor (140) can detect a license plate included in the bounding box where the vehicle as a second object is detected using a fourth deep neural network model that detects a license plate from an image containing an object. Here, the fourth deep neural network model may be a model trained in a supervised learning manner using training data that takes an image containing an object and the angle of the camera that took the image as input, and labels a text detection result including a bounding box surrounding the text contained in the object. The processor (140) can generate second characteristic information by combining the result of detecting a vehicle as a second object and the result of detecting a license plate, and can track the movement of the second characteristic information from a continuous image.

[0077] In step S430, the processor (140) can refine the duplicate detection results from the results of the first detection and the second detection, and perform de-identification by blurring the face and the license plate of the vehicle. The processor (140) can generate the results of the blurring of the face and the license plate of the vehicle as an output image. In one embodiment, when performing de-identification, the processor (140) can calculate a confidence score representing the reliability of the deep neural network model applied to generate the results of the first detection and the second detection. The processor (140) can remove confidence scores below a threshold by comparing the confidence score with a preset threshold. The processor (140) can blur the face and the license plate of the vehicle included in the results of the first detection and the second detection from which confidence scores below the threshold have been removed. In another embodiment, the processor (140) may calculate a confidence score representing the reliability of the deep neural network model applied to generate the results of the first detection and the second detection when performing de-identification. Among the results of calculating the confidence scores, the processor (140) may remove confidence scores that exceed a preset reference distance by calculating the distance to the results of surrounding detections based on the highest confidence score. The processor (140) may blur the faces and vehicle license plates included in the results of the first detection and the second detection from which confidence scores exceeding the preset reference distance have been removed.

[0079] The embodiments according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, RAM, or flash memory.

[0080] Meanwhile, the above-mentioned computer program may be one specifically designed and configured for the present invention, or one known and available to those skilled in the art of computer software. Examples of computer programs may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0081] In the specification of the present invention (particularly in the claims), the use of the term "above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is to include an invention to which individual values ​​belonging to said range are applied (unless otherwise stated), and this is equivalent to describing each individual value constituting said range in the detailed description of the invention.

[0082] Unless explicitly stated or contrary to the order of the steps constituting the method according to the present invention, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by said examples or exemplary terms unless limited by the claims. Furthermore, those skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the claims or equivalents to which they are added.

[0083] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0085] 100: Personal information anonymization device 110: First detection unit 120: Second detection unit 130: De-identification processing unit

Claims

Claim 1 A method for de-identifying personal information performed by a processor of a personal information de-identification device, comprising: a first detection step for detecting and tracking face and skeleton information of a person as a first object from an input image; a second detection step for detecting and tracking information of a vehicle and license plate of a vehicle as a second object from an input image; and a de-identification step for refining duplicate detection results from the results of the first detection and the results of the second detection, and blurring the face and license plate of the vehicle; wherein the first detection step comprises: a step of detecting the face of a first object from an input image using a first deep neural network model for detecting a face from an image; and a step of predicting the skeleton information of a first object from an input image using a second deep neural network model for predicting skeleton information from an image. The method includes the step of generating first characteristic information by combining the result of detecting the face of the first object and the result of predicting the skeleton information of the first object, and tracking the movement of the first object including the first characteristic information from a series of images, wherein the second detection step comprises: the step of detecting a vehicle as a second object from an input image using a third deep neural network model that detects a vehicle from an image; and the step of detecting a license plate included in a bounding box in which the vehicle as the second object is detected using a fourth deep neural network model that detects a license plate from an image containing an object.The method includes the step of generating second characteristic information by combining the result of detecting a vehicle as a second object and the result of detecting a license plate, and tracking the movement of the second characteristic information from a series of images; the first deep neural network model is a model trained in a supervised learning manner by training data that takes an image as input and labels a face detection result including a first bounding box surrounding a face; the second deep neural network model is a model trained in a supervised learning manner by training data that takes an image as input and labels skeleton information; the third deep neural network model is a model trained in a supervised learning manner by training data that takes an image as input and labels a vehicle detection result including a bounding box surrounding a vehicle; the fourth deep neural network model is a model trained in a supervised learning manner by training data that takes an image containing an object and the angle of the camera that captured the image as input, and labels a text detection result including a bounding box surrounding text contained in the object; and the de-identification step calculates a confidence level representing the reliability of the deep neural network model applied to generate the result of the first detection and the result of the second detection. Step; a step of removing a confidence level exceeding a preset reference distance by calculating the distance from the result of surrounding detection based on the highest confidence level among the results of calculating the confidence levels above; and a step of blurring the face and vehicle license plate included in the result of the first detection and the result of the second detection from which the confidence level exceeding the preset reference distance has been removed;A method for de-identifying personal information, comprising: a step of calculating the confidence level, wherein the step of calculating the confidence level includes, for a detection result generated by one or more of the first to fourth deep neural network models, applying a Softmax function to the output layer of the deep neural network model to obtain a probability value for each class and using the probability value of the corresponding class as the confidence level; wherein the distance in the step of removing the confidence level includes a Euclidean distance, and the Euclidean distance is used to evaluate the location or characteristics of the first object and the second object or to measure similarity. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete

Images