Method and system for de-sensitizing pictures having personal sensitive information

Abstract

The application discloses a kind of methods and systems to the picture with personal sensitive information is desensitized, by splitting into multiple identical size image blocks to be handled picture, and each image block is extracted as a multidimensional vector set with SIFT algorithm to be handled picture vector set, respectively, only include name information, mobile phone information, identity card number information picture three comparison picture groups are made, and using SIFT algorithm, the picture in it is extracted as corresponding multidimensional vector and is merged into a comparison picture vector dataset;The difference between each multidimensional vector in the to-be-processed picture vector set and the comparison picture vector dataset is calculated, if there is at least one multidimensional vector corresponding to the image block is close to each multidimensional vector of a comparison picture group, then the image block is regarded as the desensitization position area containing the sensitive information type corresponding to the close comparison picture group;Finally, the desensitization position area is masked, and the final desensitized picture is obtained by using image generation model to complete the masking area.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for desensitizing images containing sensitive personal information. Background Technology

[0002] Image anonymization technology, as one of the core means of preventing the leakage of sensitive information, is crucial for preventing serious consequences such as identity theft, fraud, and harassment, and effectively safeguards personal privacy rights. In practical business scenarios, especially when image information is involved, sensitive personal information on documents such as ID cards, driver's licenses, and passports, as well as facial and license plate recognition features, all require enhanced protection through image anonymization processing. This typically includes methods such as blurring, occlusion, or replacement of sensitive areas to effectively prevent unauthorized personnel from identifying and using this information. Currently, the market primarily uses a combination of manual delineation and image completion techniques such as Generative Adversarial Networks (GANs) and Deep Diffusion Probabilistic Models (DDPM) for the identification and replacement of sensitive information in images. These techniques automatically identify and replace manually delineated sensitive areas using trained sensitive information location recognition models, such as CNN-based object detection models. However, this method has limitations in the automation of manual delineation, and its practicality is limited in scenarios without public network connectivity or lacking a large amount of scene-specific sensitive information location annotation data. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies by providing a method for desensitizing images containing sensitive personal information, including name information, mobile phone number information, and ID card number information, comprising the following steps:

[0004] S1, the image to be processed is split into multiple image blocks of the same size, with overlapping areas between adjacent image blocks. Each image block is extracted into a multi-dimensional vector using the SIFT algorithm, forming a vector set of the image to be processed containing the multi-dimensional vectors corresponding to each image block.

[0005] S2. Create three sets of comparison images. The first set contains only name information in each sample image, the second set contains only mobile phone number information in each sample image, and the third set contains only ID card number information in each sample image. Use the SIFT algorithm to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third sets. Merge all the multi-dimensional vectors from the three sets of comparison images into a single comparison image vector dataset.

[0006] S3, calculate the difference between each multidimensional vector in the image vector set to be processed and the comparison image vector dataset. If there is at least one image block whose multidimensional vector is close to each multidimensional vector in a comparison image group, then the image block is regarded as a desensitized location region containing the sensitive information type corresponding to the similar comparison image group.

[0007] S4. After masking the obtained desensitized areas and using an image generation model to complete the masked areas, the final desensitized image is obtained.

[0008] Preferably, step S2 includes:

[0009] S21, using the SIFT algorithm, each sample image is extracted into a corresponding 128-dimensional multi-vector. The 128-dimensional multi-vector y in the image vector dataset is then subjected to product quantization, dividing the multi-dimensional vector y into 16 sub-vectors, each with a length of 8, as follows:

[0010] ;

[0011] in Let y be one of the 128-dimensional vectors. Let y be a subvector of vector y.

[0012] S22 performs 16 k-means clustering operations on the 16 sub-vectors, with each cluster storing 50 centroids.

[0013] Preferably, step S3 includes:

[0014] S31, use the following approximate formula to calculate the approximate distance between the 128-dimensional multi-dimensional vector x of each image patch in the image vector set to be processed and the multi-dimensional vector y in the image vector dataset:

[0015]

[0016] Where x is a vector in the set of image vectors to be processed, and y is a vector in the image vector dataset. Let y be a subvector of vector y. for The mapping is to the centroid of its corresponding k-means algorithm. Let x be a subvector of vector x.

[0017] S32, calculate the approximate distance between all multidimensional vectors x in the image vector set to be processed and all multidimensional vectors y in the image vector dataset, obtain a set number of candidate vector groups with the highest approximate distance, the candidate vector group includes a multidimensional vector x and a multidimensional vector y, and determine whether the multidimensional vector x in the candidate vector group belongs to a certain type of sensitive information.

[0018] Preferably, step S32 includes: classifying the candidate vector group based on the different comparison image groups from which the multidimensional vector y in the candidate vector group comes, calculating the average approximate distance in each candidate vector group set, and if the average approximate distance exceeds a set threshold, then the image to be processed is considered to contain sensitive information types corresponding to candidate vector group sets that exceed the set threshold.

[0019] Preferably, step S32 further includes: if the average approximate distance exceeds a set threshold, then obtain the multidimensional vector x in each candidate vector group corresponding to the candidate vector group set that exceeds the set threshold, and take the image block region corresponding to the multidimensional vector x as the location region containing the sensitive information type corresponding to the existing classification candidate vector group.

[0020] Preferably, step S4 includes: first marking the obtained desensitized location regions as masked, and then using the DDPM model to complete the image: the specific DDPM completion formula is as follows:

[0021] ;

[0022] in This refers to the image region sampled from the unmasked portion of the image to be processed. These are the default parameters for the DDPM model. This represents the original, masked image area. The image region is sampled by the DDPM model. These are image samples from the previous round of sampling. These are image samples after this round of sampling; and The input to a well-trained neural network is The parameters are ; It is the identity matrix. Here, a special product represents a mask, starting from... Start by repeatedly applying the formula above until the set number of iterations is reached. This will allow you to sample the unmasked portion. The splicing is sampled from the model. To complete the image.

[0023] This invention also discloses a system for desensitizing images containing sensitive personal information, including name information, mobile phone number information, and ID card number information. The system includes an image segmentation module, a sample generation module, a difference comparison module, and a desensitization and completion module. The image segmentation module divides the image to be processed into multiple image blocks of the same size, with overlapping areas between adjacent blocks. Each image block is extracted into a multi-dimensional vector using the SIFT algorithm, forming a vector set of the image to be processed containing the multi-dimensional vectors corresponding to each image block. The sample generation module creates three comparison image groups. The first image group contains only name information in each sample image; the second image group contains only mobile phone number information in each sample image; and the third image group contains only name information in each sample image. Each sample image in the image set contains only ID card number information. The SIFT algorithm is used to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third image sets. All multi-dimensional vectors from the three comparison image sets are merged into a comparison image vector dataset. The difference comparison module is used to calculate the difference between each multi-dimensional vector in the image vector set to be processed and the comparison image vector dataset. If at least one image patch has a multi-dimensional vector that is close to each multi-dimensional vector in a comparison image set, then that image patch is regarded as a desensitized location region containing the sensitive information type corresponding to the close comparison image set. The desensitization completion module is used to mask the obtained desensitized location region and use an image generation model to complete the masked region to obtain the final desensitized image.

[0024] Preferably, the sample generation module includes a vector module and a clustering module. The vector module is used to extract each sample image into a corresponding 128-dimensional multi-dimensional vector using the SIFT algorithm, perform product quantization processing on each 128-dimensional multi-dimensional vector y in the image vector dataset, and divide the multi-dimensional vector y into 16 sub-vectors, each sub-vector having a length of 8, as follows:

[0025] ,in Let y be one of the 128-dimensional vectors. y is a subvector of vector y; the clustering module is used to perform 16 k-means clusterings on the 16 subvectors, with each cluster storing 50 centroids.

[0026] The present invention also discloses an apparatus for desensitizing images containing sensitive personal information, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0027] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0028] This invention discloses a method and system for desensitizing images containing sensitive personal information. It utilizes the traditional SIFT algorithm for image feature extraction combined with an image generation model to automatically identify and desensitize sensitive information in images. The method involves splitting the image to be processed containing sensitive information into image blocks of equal size with overlapping areas. Each image block is then extracted into a multi-dimensional vector using the SIFT algorithm to form a vector set of images to be processed. Three comparison image sets are then created, each containing only name information, mobile phone number information, and ID card number information. The SIFT algorithm is then used to extract the corresponding multi-dimensional vectors from these images and merge them into a comparison image vector dataset. Then, the differences between each multidimensional vector in the vector set of the images to be processed and the vector set of the comparison images are calculated. If there is at least one image block whose multidimensional vector is close to each multidimensional vector of a comparison image group, then the image block is regarded as the desensitized location region containing the sensitive information type corresponding to the similar comparison image group. Finally, the desensitized location region is masked and the masked region is completed using an image generation model to obtain the final desensitized image. This allows enterprises or organizations to obtain images that avoid sensitive information in scenarios where they are not connected to the public network or lack a large amount of scene sensitive information location annotation data. This improves the applicability and processing speed of image processing and reduces the risk of leakage of users' personal information.

[0029] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0031] Figure 1 This is a schematic diagram illustrating the steps of a method for desensitizing images containing sensitive personal information, as disclosed in an embodiment of the present invention.

[0032] Figure 2 This is an image containing sensitive information to be processed, as disclosed in an embodiment of the present invention.

[0033] Figure 3 This is an image that has been processed and does not contain sensitive information, as disclosed in an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram illustrating how an image to be processed is divided into different image blocks according to an embodiment of the present invention.

[0035] Figure 5 This is a schematic diagram illustrating the specific steps of step S2 disclosed in an embodiment of the present invention.

[0036] Figure 6 This is a schematic diagram illustrating the specific steps of step S3 disclosed in an embodiment of the present invention.

[0037] Figure 7 This is a schematic diagram illustrating the specific steps of step S32 disclosed in an embodiment of the present invention.

[0038] Figure 8 This is a schematic diagram of a system for desensitizing images containing sensitive personal information, as disclosed in another embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0040] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0041] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0042] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a limitation of quantity, but rather indicate the presence of at least one.

[0043] Image anonymization technology, as one of the core means of preventing the leakage of sensitive information, is crucial for preventing serious consequences such as identity theft, fraud, and harassment, and effectively safeguards personal privacy rights. In practical business scenarios, especially when image information is involved, sensitive personal information on documents such as ID cards, driver's licenses, and passports, as well as facial and license plate recognition features, all require enhanced protection through image anonymization processing. This typically includes methods such as blurring, occlusion, or replacement of sensitive areas to effectively prevent unauthorized personnel from identifying and using this information. Currently, the market primarily uses a combination of manual delineation and image completion techniques such as Generative Adversarial Networks (GANs) and Deep Diffusion Probabilistic Models (DDPM) for the identification and replacement of sensitive information in images. These techniques automatically identify and replace manually delineated sensitive areas using trained sensitive information location recognition models, such as CNN-based object detection models. However, this method has limitations in the automation of manual delineation, and its practicality is limited in scenarios without public network connectivity or lacking a large amount of scene-specific sensitive information location annotation data.

[0044] Therefore, this embodiment discloses a method for desensitizing images containing sensitive personal information, including name information, mobile phone number information, and ID card number information, as shown in the attached figure. Figure 1 As shown, the steps include the following.

[0045] Step S1: Divide the image to be processed into multiple image blocks of the same size, with overlapping areas between adjacent image blocks. Extract each image block into a multidimensional vector using the SIFT algorithm to form a vector set of the image to be processed containing the multidimensional vectors corresponding to each image block.

[0046] Specifically, attached Figure 2 Given an image P containing sensitive information (name, ID number, mobile phone number), our goal is to erase the sensitive information from image P without leaving any traces. This actually falls under the category of image completion tasks in computer vision (CV), aiming to complete the image P as shown in the attached image. Figure 3The image shown is restored to a state that is as natural as possible and free of sensitive information. Therefore, this embodiment uses the traditional SIFT method to address the dependence of current sensitive information replacement methods on open-source models or online services. The Scale Invariant Feature Transform (SIFT) method is a computer vision algorithm used to detect and describe local features in images. These features are invariant to image scaling, rotation, and even affine transformations, enabling the search and extraction of key points, i.e., feature points, in different scale spaces. Specifically, as shown in the attached... Figure 4 As shown, the image P to be processed is first divided into overlapping image blocks of the same size using bounding boxes of the same size. Then, the SIFT algorithm is used to extract features from each image block and generate a corresponding 128-dimensional vector. In this embodiment, Figure 2 The image to be processed containing sensitive information is divided into 100 blocks. This number is a hyperparameter, and the specific value may vary depending on different scenarios. In practice, the number of blocks will be determined based on the desired effect, ultimately generating a vector set of the image to be processed containing 100 128-dimensional vectors.

[0047] Step S2: Create three comparison image groups. The first image group contains only name information in each sample image. The second image group contains only mobile phone number information in each sample image. The third image group contains only ID card number information in each sample image. Use the SIFT algorithm to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third image groups. Merge all the multi-dimensional vectors of the three comparison image groups into a comparison image vector dataset.

[0048] Specifically, three different types of image data were designed with a preset number of images: a preset number of images containing only names, a preset number of images containing only mobile phone numbers, and a preset number of images containing only ID card numbers. Since generating this data is far easier and faster than generating bounding boxes in object detection, multiple image datasets can be quickly produced; in this embodiment, 1000 images of each type were designed. Subsequently, the SIFT algorithm was used to extract 128-dimensional vectors from each image, resulting in a total of 3000 128-dimensional vectors constituting the image vector dataset D.

[0049] Step S3: Calculate the difference between each multidimensional vector in the image vector set to be processed and the comparison image vector dataset. If there is at least one image block whose multidimensional vector is close to each multidimensional vector in a comparison image group, then the image block is regarded as a desensitized location region containing the sensitive information type corresponding to the similar comparison image group.

[0050] Specifically, after generating the image vector set corresponding to the image to be processed and a custom image vector dataset D, the differences between each vector in the image vector set and each vector in the image vector dataset D are calculated to classify each image patch in the image to be processed and identify sensitive information. For example, if the vectors corresponding to certain patches in the image to be processed are closest to the 1000 vectors containing names in the image vector dataset, then the locations of those patches are considered to correspond to sensitive information such as names, and the information needs to be de-identified. Similarly, the locations of sensitive information such as ID card numbers or mobile phone numbers in the image to be processed are also found in this way.

[0051] Since the image to be processed needs to be compared with the vectors in the image vector dataset 100*3000 times, involving 128-dimensional calculations, this embodiment adopts the well-known PQ product quantization technique to improve the comparison speed. This technique is an efficient vector quantization method, mainly used for the compression and acceleration of high-bit data. It divides the data vector into multiple word vectors and quantizes these sub-vectors separately to reduce memory usage and improve search speed. In this embodiment, step S2 is as shown in the attached figure. Figure 5 As shown, it also includes the following content.

[0052] Step S21: Use the SIFT algorithm to extract each sample image into a corresponding 128-dimensional multi-dimensional vector. Perform product quantization on each 128-dimensional multi-dimensional vector y in the image vector dataset, and divide the multi-dimensional vector y into 16 sub-vectors, each sub-vector having a length of 8, as follows:

[0053] ;

[0054] in Let y be one of the 128-dimensional vectors. Let y be a subvector of vector y.

[0055] Step S22: Perform 16 k-means clustering operations on the 16 sub-vectors, and store 50 centroids in each cluster.

[0056] Specifically, the 128-dimensional vectors y in the image vector dataset D are multiplied and quantized. Each vector y is then divided into 16 sub-vectors of length 8, and these 16 sub-vectors are subjected to 16 k-means clustering operations. The number of segments of vector y and the number of centroids stored in each cluster are determined based on the actual processing results. Since the number of centroids is smaller than the length of the original vector, the data storage requirements are significantly reduced.

[0057] After clustering, step S3 is detailed in the appendix. Figure 6 As shown, it also includes the following content.

[0058] Step S31: Calculate the approximate distance between the 128-dimensional multi-dimensional vector x of each image patch in the image vector set to be processed and the multi-dimensional vector y in the image vector dataset using the following approximation formula:

[0059] ;

[0060] Where x is a vector in the set of image vectors to be processed, and y is a vector in the image vector dataset. Let y be a subvector of vector y. for The mapping is to the centroid of its corresponding k-means algorithm. Let x be a subvector of vector x.

[0061] Step S32: Calculate the approximate distance between all multidimensional vectors x in the image vector set to be processed and all multidimensional vectors y in the image vector dataset, obtain a set number of candidate vector groups with the highest approximate distance, the candidate vector group containing a multidimensional vector x and a multidimensional vector y, and determine whether the multidimensional vector x in the candidate vector group belongs to a certain type of sensitive information.

[0062] Specifically, approximate formulas are used to calculate the approximate distance between each multidimensional vector x in the image vector set to be processed and all multidimensional vectors y in the image vector dataset. The top-ranking multidimensional vectors y with the largest approximate distances to each multidimensional vector x are then used to form candidate vector groups to determine whether the image patch corresponding to a multidimensional vector x contains a certain type of sensitive information. In this embodiment, the 100 candidate vector groups with the largest approximate distances are selected as the most approximate vectors for decision-making, as shown in the attached figure. Figure 7 The specific content shown includes the following.

[0063] Step S321: Based on the different comparison image groups from which the multidimensional vector y in the candidate vector group comes, classify the sets and calculate the approximate average distance in each candidate vector group set:

[0064] ;

[0065] Where x is a vector in the set of image vectors to be processed, and y is a vector in the image vector dataset. , , Let x represent the set of multidimensional vectors y that contain sensitive information of type name, ID number, and mobile phone number in the candidate vector group corresponding to x. , , Representing sets respectively , , The number of elements in , , They represent , , The average of the approximate distances among the candidate vector groups formed by the multidimensional vectors y and x.

[0066] For example, when a multidimensional vector x corresponds to a multidimensional vector x in 100 candidate vector groups, 60 of these candidate vector groups contain multidimensional vector y that corresponds to sensitive information such as a name, 30 of these candidate vector groups contain multidimensional vector y that corresponds to sensitive information such as an ID card number, and 10 of these candidate vector groups contain multidimensional vector y that corresponds to sensitive information such as a mobile phone number, then... , , Let each represent a set of 60 multidimensional vectors y corresponding to sensitive information such as names, 30 multidimensional vectors y corresponding to sensitive information such as ID card numbers, and 10 multidimensional vectors y corresponding to sensitive information such as mobile phone numbers.

[0067] Step S322: Determine whether the average approximate distance exceeds a set threshold. If the average approximate distance exceeds the set threshold, it is considered that the image to be processed contains sensitive information types corresponding to candidate vector groups exceeding the set threshold. Obtain the multidimensional vector x in each candidate vector group corresponding to the candidate vector group exceeding the set threshold, and take the image block region corresponding to the multidimensional vector x as the location region containing the sensitive information type corresponding to the existing classification candidate vector group.

[0068] Specifically, for example, when the set threshold is 0.4, if If the value is greater than 0.4, then it is considered that... The corresponding image patch of the multidimensional vector x is a location region containing sensitive name information, which needs to be desensitized.

[0069] Step S4: After masking the obtained desensitized area and using an image generation model to complete the masked area, the final desensitized image is obtained.

[0070] Specifically, after locating the specific desensitization area, the found desensitization area is masked, and the DDPM model is used to complete the image to obtain the final desensitized image. This includes:

[0071] First, mark the locations of the desensitized areas as masked, and then use the DDPM model to complete the image:

[0072] The specific DDPM completion formula is as follows:

[0073] ;

[0074] in This refers to the image region sampled from the unmasked portion of the image to be processed. These are the default parameters for the DDPM model. This represents the original, masked image area. The image region is sampled by the DDPM model. These are image samples from the previous round of sampling. These are image samples after this round of sampling; and The input to a well-trained neural network is The parameters are ; It is the identity matrix. Here, a special product represents a mask, starting from... Start by repeatedly applying the formula above until the set number of iterations is reached. This will allow you to sample the unmasked portion. The splicing is sampled from the model. To complete the image.

[0075] The method and system for desensitizing images containing sensitive personal information disclosed in this embodiment utilizes the traditional SIFT algorithm for image feature extraction combined with an image generation model to automatically identify and desensitize sensitive information in images. This is achieved by splitting the image to be processed containing sensitive information into image blocks of the same size with overlapping areas, and then using the SIFT algorithm to extract a multi-dimensional vector from each image block to form a vector set of images to be processed. Three comparison image groups are then created, each containing only name information, mobile phone number information, and ID card number information. The SIFT algorithm is then used to extract the corresponding multi-dimensional vectors from these images and merge them into a comparison image vector dataset. Then, the differences between each multidimensional vector in the vector set of the images to be processed and the vector set of the comparison images are calculated. If there is at least one image block whose multidimensional vector is close to each multidimensional vector of a comparison image group, then the image block is regarded as the desensitized location region containing the sensitive information type corresponding to the similar comparison image group. Finally, the desensitized location region is masked and the masked region is completed using an image generation model to obtain the final desensitized image. This allows enterprises or organizations to obtain images that avoid sensitive information in scenarios where they are not connected to the public network or lack a large amount of scene sensitive information location annotation data. This improves the applicability and processing speed of image processing and reduces the risk of leakage of users' personal information.

[0076] In another embodiment, as shown in the appendix Figure 8As shown, a system for desensitizing images containing sensitive personal information, including name, mobile phone number, and ID card number, is also disclosed. The system includes an image segmentation module 1, a sample generation module 2, a difference comparison module 3, and a desensitization and completion module 4. The image segmentation module 1 divides the image to be processed into multiple image blocks of the same size, with overlapping areas between adjacent blocks. Each image block is extracted into a multi-dimensional vector using the SIFT algorithm, forming a vector set of the image to be processed containing the corresponding multi-dimensional vectors of each image block. The sample generation module 2 creates three comparison image groups. The first image group contains only name information, the second image group contains only mobile phone number information, and the third image group contains only ID card number information. The SIFT algorithm is used to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third image groups. All multi-dimensional vectors from the three comparison image groups are merged into a comparison image vector dataset. The difference comparison module 3 is used to calculate the difference between each multi-dimensional vector in the vector set of images to be processed and the vector set of comparison images. If at least one image patch has a multi-dimensional vector that is close to each multi-dimensional vector in a comparison image group, then that image patch is taken as a desensitized location region containing the sensitive information type corresponding to the close comparison image group. The desensitization completion module 4 is used to mask the obtained desensitized location region and use an image generation model to complete the masked region to obtain the final desensitized image.

[0077] In this embodiment, the sample generation module includes a vector module and a clustering module. The vector module is used to extract each sample image into a corresponding 128-dimensional multi-dimensional vector using the SIFT algorithm. It performs product quantization processing on each 128-dimensional multi-dimensional vector y in the image vector dataset, dividing the multi-dimensional vector y into 16 sub-vectors, each sub-vector having a length of 8, as detailed below:

[0078] ,in Let y be one of the 128-dimensional vectors. y is a subvector of vector y; the clustering module is used to perform 16 k-means clusterings on the 16 subvectors, with each cluster storing 50 centroids.

[0079] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar parts between embodiments can be referred to interchangeably. Regarding the system for desensitizing images containing sensitive personal information disclosed in the embodiments, since it corresponds to the method for desensitizing images containing sensitive personal information disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to the foregoing method section.

[0080] In other embodiments, an apparatus for desensitizing images containing sensitive personal information is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the methods for desensitizing images containing sensitive personal information as described in the above embodiments. The server may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagrams are merely examples of a server and do not constitute a limitation on the server; it may include more or fewer components than illustrated, or combine certain components, or use different components.

[0081] If the device for desensitizing images containing sensitive personal information is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above embodiments of the method for desensitizing images containing sensitive personal information. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory, random access memory, electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0082] In summary, the above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be covered by the present invention.

Claims

1. A method for desensitizing images containing sensitive personal information, wherein the sensitive personal information includes name information, mobile phone number information, and ID card number information, characterized in that, Includes the following steps: S1, the image to be processed is split into multiple image blocks of the same size, with overlapping areas between adjacent image blocks. Each image block is extracted into a multi-dimensional vector using the SIFT algorithm, forming a vector set of the image to be processed containing the multi-dimensional vectors corresponding to each image block. S2. Create three sets of comparison images. The first set contains only name information in each sample image, the second set contains only mobile phone number information in each sample image, and the third set contains only ID card number information in each sample image. Use the SIFT algorithm to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third sets. Merge all the multi-dimensional vectors from the three sets of comparison images into a single comparison image vector dataset. S3, calculate the difference between each multidimensional vector in the image vector set to be processed and the comparison image vector dataset. If there is at least one image block whose multidimensional vector is close to each multidimensional vector in a comparison image group, then the image block is regarded as a desensitized location region containing the sensitive information type corresponding to the similar comparison image group. After generating the image vector set corresponding to the image to be processed and the custom image vector dataset D, the differences between each vector in the image vector set and each vector in the image vector dataset D are calculated to classify each image patch in the image to be processed and identify the sensitive information therein. The well-known PQ product quantization technique is used to divide the data vector into multiple sub-vectors, and each sub-vector is quantized separately; step S3 includes: S31, use the following approximate formula to calculate the approximate distance between the 128-dimensional multi-dimensional vector x of each image patch in the image vector set to be processed and the multi-dimensional vector y in the image vector dataset: Where x is a vector in the set of image vectors to be processed, y is a vector in the image vector dataset, and u j (y) is a subvector of vector y, q j (u j (y)) is u j (y) is mapped to the centroid of its corresponding k-means algorithm, u j (x) is a subvector of vector x; S32, calculate the approximate distance between all multidimensional vectors x in the image vector set to be processed and all multidimensional vectors y in the image vector dataset, obtain a set number of candidate vector groups with the highest approximate distance, the candidate vector group includes a multidimensional vector x and a multidimensional vector y, and determine whether the multidimensional vector x in the candidate vector group belongs to a certain type of sensitive information. Step S321: Based on the different comparison image groups from which the multidimensional vector y in the candidate vector group comes, classify the sets and calculate the approximate average distance in each candidate vector group set: Where x is a vector in the set of image vectors to be processed, y is a vector in the image vector dataset, and D n D i D p Let each of the following represent a set of candidate vectors for x containing multidimensional vectors y of types corresponding to sensitive information: name, ID number, and mobile phone number. n |、|D i |、|D p | represent sets D respectively n D i D p The number of elements in D n D i D p The average of the approximate distances among the candidate vector groups formed by the multidimensional vectors y and x; Step S322: Determine whether the average approximate distance is less than a set threshold. If the average approximate distance is less than the set threshold, it is considered that the image to be processed contains sensitive information types corresponding to candidate vector groups with a value less than the set threshold. Obtain the multidimensional vector x in each candidate vector group corresponding to the candidate vector group with a value less than the set threshold, and take the image block region corresponding to the multidimensional vector x as the location region containing the sensitive information type corresponding to the candidate vector group of the current classification. S4. After masking the obtained desensitized areas and using an image generation model to complete the masked areas, the final desensitized image is obtained.

2. The method for desensitizing images containing sensitive personal information according to claim 1, wherein step S2 includes: S21, using the SIFT algorithm, each sample image is extracted into a corresponding 128-dimensional multi-vector. The 128-dimensional multi-vector y in the image vector dataset is then subjected to product quantization, dividing the multi-dimensional vector y into 16 sub-vectors, each with a length of 8, as follows: y = [y1, y2, …, y 128 = [y1, …, y8|y9, …, y 16 |……|y 121 ,…y 128 is denoted as [u1(y),..., u 16 (y)] Where y i Let u be one of the 128-dimensional vectors y. i (y) is a subvector of vector y; S22 performs 16 k-means clustering operations on the 16 sub-vectors, with each cluster storing 50 centroids.

3. The method for desensitizing images containing sensitive personal information according to claim 2, wherein step S4 includes: First, mark the locations of the desensitized areas as masked, and then use the DDPM model to complete the image: The specific DDPM completion formula is as follows: in This refers to the image region sampled from the unmasked portion of the image to be processed. These are the default parameters for the DDPM model, where x0 represents the original masked image region. x represents the image region sampled by the DDPM model. t For the image samples after the previous round of sampling, x t-1 These are image samples after this round of sampling; μ θ and ∑ θ The input to a well-trained neural network is x. t t, parameter θ; I is the identity matrix, ⊙ is the special product, m is the mask; first from Start by repeatedly applying the formula above until the set number of iterations is reached; this will allow sampling of the unmasked portion to be used. The splicing is sampled from the model Complete the image.

4. A system for desensitizing images containing sensitive personal information, including name information, mobile phone number information, and ID card number information, characterized in that... include: The image segmentation module is used to segment the image to be processed into multiple image blocks of the same size, where there are overlapping areas between adjacent image blocks. Each image block is extracted into a multi-dimensional vector using the SIFT algorithm, forming a vector set of the image to be processed containing the multi-dimensional vectors corresponding to each image block. The sample generation module is used to create three sets of comparison images. The first set of images contains only name information, the second set contains only mobile phone number information, and the third set contains only ID card number information. The SIFT algorithm is used to extract the corresponding multi-dimensional vectors from each sample image in the first, second, and third sets of images. All multi-dimensional vectors from the three sets of comparison images are then merged into a single comparison image vector dataset. The difference comparison module is used to calculate the difference between each multidimensional vector in the image vector set to be processed and the comparison image vector dataset. If there is at least one image block whose multidimensional vector is close to each multidimensional vector in a comparison image group, then the image block is regarded as a desensitized location region containing the sensitive information type corresponding to the close comparison image group. After generating the image vector set corresponding to the image to be processed and the custom image vector dataset D, the differences between each vector in the image vector set and each vector in the image vector dataset D are calculated to classify each image patch in the image to be processed and identify the sensitive information therein; the well-known PQ product quantization technique is adopted to divide the data vector into multiple sub-vectors and quantize these sub-vectors separately. The approximate distance between the 128-dimensional multi-dimensional vector x of each image patch in the image vector set to be processed and the multi-dimensional vector y in the image vector dataset is calculated using the following approximate formula: Where x is a vector in the set of image vectors to be processed, y is a vector in the image vector dataset, and u j (y) is a subvector of vector y, q j (u j (y)) is u j (y) is mapped to the centroid of its corresponding k-means algorithm, u j (x) is a subvector of vector x; calculate the approximate distance between all multidimensional vectors x in the image vector set to be processed and all multidimensional vectors y in the image vector dataset, obtain a set number of candidate vector groups with the highest approximate distance, each candidate vector group containing a multidimensional vector x and a multidimensional vector y, determine whether the multidimensional vector x in the candidate vector group belongs to a certain type of sensitive information; classify the candidate vector groups according to the different comparison image groups from which the multidimensional vector y in the candidate vector group comes, and calculate the average approximate distance in each classification candidate vector group set: Where x is a vector in the set of image vectors to be processed, y is a vector in the image vector dataset, and D n D i D p Let each of the following represent a set of candidate vectors for x containing multidimensional vectors y of types corresponding to sensitive information: name, ID number, and mobile phone number. n |、|D i |、|D p | represent sets D respectively n D i D p The number of elements in D n D i D p The average of the approximate distances in the candidate vector groups composed of multidimensional vectors y and x is calculated. It is then determined whether the average approximate distance is less than a set threshold. If the average approximate distance is less than the set threshold, it is considered that the image to be processed contains the sensitive information type corresponding to the candidate vector group set that is less than the set threshold. The multidimensional vector x in each candidate vector group corresponding to the candidate vector group set that is less than the set threshold is obtained, and the image block region corresponding to the multidimensional vector x is taken as the location region containing the sensitive information type corresponding to the candidate vector group of the current classification. The desensitization and completion module is used to mask the obtained desensitization area and then use an image generation model to complete the masked area to obtain the final desensitized image.

5. The system for desensitizing images containing sensitive personal information according to claim 4, wherein the sample generation module comprises: The vector module is used to extract each sample image into a corresponding 128-dimensional multi-dimensional vector using the SIFT algorithm. It performs product quantization on each 128-dimensional multi-dimensional vector y in the image vector dataset, dividing the multi-dimensional vector y into 16 sub-vectors, each with a length of 8, as detailed below: y = [y1, y2, …, y 128 = [y1, …, y8 | y9, …, y 16 | …… | y 121 , … y 128 is denoted as [u1(y), …, u 16 (y)] Where y i Let u be one of the 128-dimensional vectors y. i (y) is a subvector of vector y; The clustering module is used to perform 16 k-means clustering operations on the 16 sub-vectors, with each cluster storing 50 centroids.

6. An apparatus for desensitizing images containing sensitive personal information, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-3.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-3.

Images