Image processing methods and apparatuses, electronic devices and storage media

Abstract

A privacy-preserving image processing method, apparatus, electronic device, and storage medium are disclosed. The method includes: acquiring a preprocessed image and performing feature extraction on the preprocessed image to obtain a first feature vector (S10); generating a noisy image based on the preprocessed image and performing feature extraction on the noisy image to obtain a second feature vector (S20); establishing a loss function based on the structural similarity between the preprocessed image and the noisy image, using the first feature vector and the second feature vector (S30); processing the noisy image using the loss function to obtain a blurred image (S40); and replacing the preprocessed image with the blurred image (S50).

Description

Technical Field

[0001] This application relates to the field of imaging technology, and in particular to a privacy-preserving image processing method, image processing apparatus, electronic device, and storage medium. Background Technology

[0002] With the advent of the artificial intelligence era, facial information is increasingly being used and collected in real life, such as in facial recognition and video surveillance. However, in these applications, failure to effectively protect facial information can lead to privacy leaks and infringements on privacy and portrait rights. Current technologies employ image processing methods such as scrambling, blurring, and obscuring to protect personal identity information, but these methods are detrimental to subsequent facial recognition authentication and other business applications. Other methods use traditional encryption or encrypt private content in the transform domain, but these methods, due to the existence of encryption and decryption keys, pose a risk of unauthorized decryption. Summary of the Invention

[0003] This application provides a privacy-protecting image processing method, image processing apparatus, electronic device, and storage medium.

[0004] An image processing method for privacy protection according to an embodiment of this application includes: acquiring a preprocessed image and performing feature extraction on the preprocessed image to obtain a first feature vector; generating a noisy image based on the preprocessed image and extracting features from the noisy image to obtain a second feature vector; establishing a loss function based on the structural similarity between the preprocessed image and the noisy image, using the first feature vector and the second feature vector; processing the noisy image using the loss function to obtain a blurred image; and replacing the preprocessed image with the blurred image.

[0005] An image processing apparatus for privacy protection according to an embodiment of this application includes: an acquisition module for acquiring a preprocessed image and extracting features from the preprocessed image to obtain a first feature vector; a generation module for generating a noisy image and extracting features from the noisy image to obtain a second feature vector, wherein the noisy image corresponds to the preprocessed image; a processing module for establishing a loss function based on the structural similarity between the preprocessed image and the noisy image, the first feature vector, and the second feature vector; a blurring module for processing the noisy image using the loss function to obtain a blurred image; and a replacement module for replacing the preprocessed image with the blurred image.

[0006] An electronic device according to an embodiment of this application includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the image processing method described above.

[0007] This application provides a non-volatile computer-readable storage medium for a computer program, characterized in that, when the computer program is executed by one or more processors, it implements the image processing method described above.

[0008] Additional aspects and advantages of the embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0009] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0010] Figure 1 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0011] Figure 2 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0012] Figure 3 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0013] Figure 4 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0014] Figure 5 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0015] Figure 6 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0016] Figure 7 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0017] Figure 8 This is a schematic flowchart of the image processing method according to an embodiment of this application;

[0018] Figure 9 This is a block diagram of an image processing apparatus according to an embodiment of this application;

[0019] Figure 10 This is a block diagram of an image processing apparatus according to an embodiment of this application;

[0020] Figure 11 This is a block diagram of an image processing apparatus according to an embodiment of this application;

[0021] Figure 12 This is a block diagram of an image processing apparatus according to an embodiment of this application. Detailed Implementation

[0022] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0023] Please see Figure 1 This application provides a privacy-preserving image processing method, characterized by comprising:

[0024] S10: Obtain the preprocessed image and perform feature extraction on the preprocessed image to obtain the first feature vector;

[0025] S20: Generate a noisy image based on the preprocessed image to extract features from the noisy image and obtain a second feature vector;

[0026] S30: Based on the structural similarity between the preprocessed image and the noisy image, a loss function is established using the first feature vector and the second feature vector;

[0027] S40: Use a loss function to process noisy images to obtain blurred images;

[0028] S50: Replace the preprocessed image with a blurred image.

[0029] This application also provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and the processor is used to acquire a preprocessed image, extract features from the preprocessed image to obtain a first feature vector, generate a noisy image based on the preprocessed image, extract features from the noisy image to obtain a second feature vector, establish a loss function based on the structural similarity between the preprocessed image and the noisy image, using the first feature vector and the second feature vector, process the noisy image using the loss function to obtain a blurred image, and replace the preprocessed image with the blurred image.

[0030] In step S10, a preprocessed image is acquired. This image is the one for which privacy protection is desired; it can be an image present in a photograph, on a television screen, or on a computer screen, or an object in a video that requires privacy protection. The object can include a face or other areas requiring privacy protection, and the protection scope can be the entire image or a region of interest (ROI).

[0031] Specifically, the acquired preprocessed image is the image after which the image to be protected for privacy has undergone preprocessing. Preprocessing includes image cropping, feature extraction, and face detection if the image contains faces.

[0032] In some implementations, the image to be privacy protected is a face, and the face region after face detection is obtained as a preprocessed image.

[0033] In some implementations, if a fixed area in the image requires privacy protection, such as the watermark in the lower right corner, then a certain size of the lower right corner area of ​​the image is cropped as the preprocessed image.

[0034] Further, feature extraction is performed on the preprocessed image to obtain a first feature vector. Here, different images have different feature vectors depending on the specific context; the feature vector has a common meaning in the field of image processing. For example, the feature vector of a face image is a vector of facial key points, specifically including coordinate points on the face with clear semantic meaning, such as the tip of the nose, corners of the mouth, and corners of the eyes. This first feature vector is used to distinguish subsequent feature vectors.

[0035] In step S20, a noisy image is generated based on the preprocessed image, and features are extracted from the noisy image to obtain a second feature vector. The noisy image can be a noisy image that satisfies a preset mean and a preset variance, or a randomly generated noisy image. The feature extraction method for the noisy image can be the same as the feature extraction method for the preprocessed image, and the feature vector can also be the same.

[0036] Furthermore, in steps S30 and S40, a loss function is established based on the structural similarity between the preprocessed image and the noisy image, using the first feature vector and the second feature vector, and the noisy image is processed using the loss function to obtain a blurred image.

[0037] It is understandable that the loss function can be established based on different parameters. In this embodiment, the loss function is established by extracting the structural similarity and feature vectors between the preprocessed image and the noisy image. Structural similarity characterizes the similarity between the preprocessed image and the output image. By calculating the structural similarity between the input and output images, the structural similarity can be minimized, thereby maximizing the difference between the generated blurred image and the preprocessed image. In some implementations, structural dissimilarity can also be utilized.

[0038] The loss functions include classification loss functions such as cross-entropy loss to determine the closeness between the actual output and the expected output, and / or quality loss functions to determine the quality difference between the output image and the expected output, and / or distance loss functions to determine the distance between the feature vectors. Simultaneously, one or more loss functions can be used for multi-objective optimization. Furthermore, the optimized loss function aims to minimize the loss function so that the output image maximizes its difference from the preprocessed image while simultaneously making the feature vectors as close as possible.

[0039] Specifically, a loss function, such as Euclidean distance, can be established based on the distance between the first and second feature vectors. Simultaneously, the distance between the feature vectors can be minimized to make the feature vectors of the generated blurred image and the preprocessed image as close as possible. The noisy image is input into the loss function, and the desired blurred image is output by optimizing the loss function, such as by minimizing the established loss function to optimize intermediate parameters.

[0040] In some implementations, a cross-entropy loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image.

[0041] In some implementations, a cross-entropy loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image, while simultaneously ensuring that the feature vectors of the output blurred image and the preprocessed image are as close as possible.

[0042] In some implementations, a cross-entropy loss function and a distance loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image, while ensuring that the feature vectors of the output blurred image and the preprocessed image are as close as possible.

[0043] In step S50, the blurred image is used to replace the preprocessed image. Once the blurred image is obtained, it replaces the preprocessed image, or in other words, the blurred image is restored to the preprocessed image. For example, in a real-time monitoring screen, a loss function is used to blur the face, and then the blurred image is restored to the real-time monitoring screen, so that only the blurred face is displayed instead of a clear face.

[0044] In some implementations, the blurred image can be resized or transformed before being restored to the real-time monitoring screen.

[0045] In some implementations, the image is facial information, which can then be further processed for subsequent applications such as facial recognition.

[0046] Thus, the aforementioned privacy-preserving image processing method involves acquiring a preprocessed image, extracting features from it to obtain a first feature vector, generating a noisy image based on the preprocessed image, extracting features from the noisy image to obtain a second feature vector, establishing a loss function based on the structural similarity between the preprocessed image and the noisy image, using the loss function to process the noisy image to obtain a blurred image, and replacing the preprocessed image with the blurred image. This allows for blurring images requiring privacy protection and replacing the original image with the blurred image, enabling real-time application scenarios to output blurred images of privacy-preserving areas, thereby minimizing information collection and effectively reducing privacy leaks. Simultaneously, replacing the original image with the blurred image for storage and transmission mitigates the risk of leakage or copying to some extent. Furthermore, establishing and optimizing the loss function based on structural similarity and feature vectors to generate the blurred image ensures that the feature vectors are as close as possible to the preprocessed image while blurring the image, thus improving the quality of the blurred image to a certain extent. Furthermore, subsequent facial recognition models can more accurately reconstruct images from blurred images, effectively improving the accuracy of facial recognition and authentication. Moreover, facial recognition models can be used for subsequent business applications such as facial recognition authentication, effectively protecting image privacy while enabling recognition and authentication services.

[0047] Please see Figure 2 In some implementations, obtaining the preprocessed image includes:

[0048] S11: Perform face detection on the scene image including facial features to obtain the face region, and determine the face region as the preprocessed image.

[0049] In some implementations, the processor is used to perform face detection on a scene image including facial features to obtain a face region, and to determine the face region as a preprocessed image.

[0050] Specifically, face regions can be obtained by detecting scene images using face detection models such as Multi-task Convolutional Neural Network (MTCNN), and then the face regions can be used as preprocessed face images for subsequent blurring processing, with the specific process being the same as the implementation method described above.

[0051] In this way, facial regions can be extracted through face detection models, enabling privacy protection by blurring facial information in face-related business application scenarios.

[0052] Please see Figure 3In some embodiments, the image processing method further includes:

[0053] S60: Perform face recognition on the preprocessed face image to authenticate the face.

[0054] In some implementations, the processor is used to perform face recognition on the preprocessed face image to authenticate the face.

[0055] Specifically, face recognition is performed on preprocessed face images to authenticate faces. Various face recognition models can be used for face recognition; for example, Euclidean distance or cosine distance can be used in feature extraction.

[0056] Euclidean distance measures the absolute distance between points in two images. It is used to extract facial features, and then the Euclidean distance is compared with a threshold to identify the face. The specific formula for calculating Euclidean distance is omitted here; this application utilizes Euclidean distance for face recognition of preprocessed facial images.

[0057] In some implementations, cosine distance can be used for face recognition in scene images. Cosine distance uses the cosine of the angle between two vectors in vector space as a measure of the difference between two images. Face features are extracted using cosine distance, and then face recognition is performed by comparing the cosine distance with a threshold. The specific formula for calculating cosine distance is omitted here. This application can use cosine distance to perform face recognition on preprocessed face images.

[0058] It should be noted that there is no specific order between face recognition, blurring, and image replacement; the program can be designed according to the actual business requirements.

[0059] Thus, by simultaneously recognizing faces, facial authentication can be performed while protecting privacy through blurring. Alternatively, by adding blurring processing before facial recognition for authentication, facial authentication services can effectively protect facial information privacy and improve the security of facial authentication.

[0060] Please see Figure 4 In some embodiments, step S11 further includes:

[0061] S111: Transform the face region to change the original first size to a second size to obtain a preprocessed image.

[0062] Meanwhile, step S50 includes:

[0063] S51: Transform the blurred image to the first size and replace the preprocessed image.

[0064] In some implementations, the processor is used to resize the face region to transform the original first size to a second size to obtain a preprocessed image, and to resize the blurred image to the first size to replace the preprocessed image.

[0065] Specifically, the extracted face region undergoes a size transformation, such as linear interpolation, to change the original first size to a second size, resulting in a preprocessed image. After blurring the preprocessed image to obtain a blurred image, the blurred image needs to be restored to the first size to replace the preprocessed image.

[0066] In this way, the size of each extracted face region can be unified by size transformation, which also effectively improves the efficiency of subsequent blurring processing.

[0067] In some implementations, the noise image is a random noise image, and the size of the noise image is a second size.

[0068] Please see Figure 5 In some implementations, step S30 includes:

[0069] S31: Establish the first loss function To determine the loss function, the first loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, E2 is the second feature vector, E0 is the first feature vector, and λ is the penalty term.

[0070] In some implementations, the processor is used to establish a first loss function to obtain a loss function, and to minimize the first loss function to obtain a blurred image.

[0071] Specifically, the first loss function is established as follows:

[0072]

[0073] The input to the first loss function is the noisy image T1 and the preprocessed image T0, and the output is the optimal result T, i.e., the blurred image, generated by iteratively processing the intermediate result T2.

[0074] in, It is the L2 norm. The structural similarity is between T2 and T0. The formula for calculating structural similarity is omitted here. The larger the value of , the more similar T2 is to T0. The goal is to minimize the function to maximize the difference between the generated blurred image and the preprocessed image. At the same time, the feature vectors of the two images should be as close as possible.

[0075] Additionally, λ is a penalty term in the loss function, used to control the approximation degree of the feature vectors during optimization. A larger λ value indicates a higher requirement for the approximation degree of the feature vectors; it should be set according to the actual situation and adjusted accordingly during calculation.

[0076] Thus, by determining the loss function through the first loss function described above and minimizing the first loss function, a blurred image can be obtained.

[0077] Please see Figure 6 In some embodiments, the image processing method further includes:

[0078] S70: Establish the second loss function ,in, For function labels, The probability to be predicted is n, the number of samples is n, and the label is 1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold.

[0079] S80: Minimize the first comprehensive loss function to obtain the blurred image. The first comprehensive loss function includes the second loss function and the first loss function.

[0080] In some implementations, the processor is used to establish a second loss function and minimize a first comprehensive loss function to obtain a blurred image, the first comprehensive loss function including the second loss function and the first loss function.

[0081] Specifically, the second loss function is established as follows:

[0082]

[0083] The blurred image is restored to the preprocessed image to obtain the protected image. A face detection model is then used to detect faces on the protected image. If a face is detected, and the Euclidean distance between the face's location information and the location information in the preprocessed image is less than a given threshold, then the label is assigned as label=1; otherwise, the label is assigned as label=0. For example, if the Euclidean distance dist(A,B) between the top-left (x3,y3) and bottom-right (x,y4) coordinates in protected image A and the top-left (x1,y1) and bottom-right (x2,y2) coordinates in preprocessed image B is less than a given threshold, then the label is assigned as label=1; otherwise, the label is assigned as label=0.

[0084] Furthermore, the preserved image A and the preprocessed image B are input into the second loss function, and the results are processed accordingly. By minimizing the solution, we can determine whether the input image is a human face.

[0085] Furthermore, the first comprehensive loss function includes both the second loss function and the first loss function. Thus, the first comprehensive loss function can be defined. :

[0086]

[0087] By minimizing the first comprehensive loss function The resulting blurred image is a mathematical solution process that will not be elaborated here.

[0088] Thus, by adding a loss function for face detection in the protected image, the quality of the blurred image can be further improved, allowing the replaced blurred image to more accurately replace the preprocessed image.

[0089] Please see Figure 7 In some embodiments, step S30 further includes:

[0090] S32: Establish the third loss function A loss function is determined, and a third loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, and T0 is the preprocessed image. The distance is the cosine distance.

[0091] In some implementations, the processor is used to establish a third loss function to obtain a loss function, and to minimize the third loss function to obtain a blurred image.

[0092] Specifically, the third loss function is established as follows:

[0093]

[0094] The input to the third loss function is the noisy image T1 and the preprocessed image T0, and the output is the optimal result T, i.e., the blurred image, generated by iteratively processing the intermediate result T2.

[0095] in, Cosine distance SSIM represents the structural similarity between T2 and T0. The formula for calculating structural dissimilarity is omitted here. A higher SSIM value indicates greater similarity between T2 and T0. The goal is to minimize the difference between the generated blurred image and the preprocessed image by using a minimization function. Simultaneously, the feature vectors of the two images should be as close as possible.

[0096] Compared to Loss1 in the above embodiment In this embodiment, the calculation of Euclidean distance is replaced by cosine distance. That is, calculating the cosine of the angle between two vectors:

[0097]

[0098] Additionally, λ is a penalty term, a penalty factor used to control the approximation degree of the eigenvectors during the optimization process. The larger the value of λ, the higher the requirement for the approximation degree of the eigenvectors. It should be set according to the actual situation and adjusted accordingly during the calculation process.

[0099] Thus, by determining the loss function through the aforementioned third loss function and minimizing the third loss function, a blurred image can be obtained.

[0100] Please see Figure 8 In some embodiments, the image processing method further includes:

[0101] S90: Establish the second loss function ,in, For function labels, The probability to be predicted is n, the number of samples is n, and the label is 1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold.

[0102] S100: Minimize the second comprehensive loss function to obtain the blurred image. The second comprehensive loss function includes the second loss function and the third loss function.

[0103] In some implementations, the processor is used to minimize a second comprehensive loss function to obtain a blurred image, the second comprehensive loss function including a second loss function and a third loss function.

[0104] Specifically, the second comprehensive loss function includes a second loss function and a third loss function. Thus, the second comprehensive loss function can be defined. :

[0105]

[0106] Minimize the first comprehensive loss function The process of minimizing the blurred image is a mathematical solution and will not be elaborated here.

[0107] Thus, by adding a loss function for face detection in the protected image, the quality of the blurred image can be further improved, allowing the replaced blurred image to more accurately replace the preprocessed image.

[0108] Please see Figure 9This application also provides a privacy-preserving image processing apparatus 10. The image processing apparatus 10 includes an acquisition module 11, a generation module 12, a processing module 13, a blurring module 14, and a replacement module 15. The acquisition module 11 acquires a preprocessed image and performs feature extraction on the preprocessed image to obtain a first feature vector. The generation module 12 generates a noisy image and performs feature extraction on the noisy image to obtain a second feature vector; the noisy image corresponds to the preprocessed image. The processing module 13 establishes a loss function based on the structural similarity between the preprocessed image and the noisy image, the first feature vector, and the second feature vector. The blurring module 14 processes the noisy image using the loss function to obtain a blurred image. The replacement module 15 replaces the preprocessed image with the blurred image.

[0109] The acquisition module 11 acquires the preprocessed image. The image is the one for which privacy protection is desired; it can be an image in a photograph, on a television screen, or on a computer screen, or an object in a video that requires privacy protection. The object can include a face or other areas requiring privacy protection, and the protection scope can be the entire image or a region of interest (ROI).

[0110] Specifically, the acquired preprocessed image is the image after which the image to be protected for privacy has undergone preprocessing. Preprocessing includes image cropping, feature extraction, and face detection if the image contains faces.

[0111] In some implementations, the image to be privacy protected is a face, and the face region after face detection is obtained as a preprocessed image.

[0112] In some implementations, if a fixed area in the image requires privacy protection, such as the watermark in the lower right corner, then a certain size of the lower right corner area of ​​the image is cropped as the preprocessed image.

[0113] Further, feature extraction is performed on the preprocessed image to obtain a first feature vector. Here, different images have different feature vectors depending on the specific context; the feature vector has a common meaning in the field of image processing. For example, the feature vector of a face image is a vector of facial key points, specifically including coordinate points on the face with clear semantic meaning, such as the tip of the nose, corners of the mouth, and corners of the eyes. This first feature vector is used to distinguish subsequent feature vectors.

[0114] The generation module 12 generates a noisy image based on the preprocessed image and then extracts features from the noisy image to obtain a second feature vector. The noisy image can be a noisy image that meets a preset mean and a preset variance, or a randomly generated noisy image. The feature extraction method for the noisy image can be the same as the feature extraction method for the preprocessed image, and the feature vector can also be the same.

[0115] Furthermore, processing module 13 establishes a loss function based on the structural similarity between the preprocessed image and the noisy image, the first feature vector, and the second feature vector. Blurring module 14 then uses the loss function to process the noisy image to obtain a blurred image.

[0116] It is understandable that the loss function can be established based on different parameters. In this embodiment, the loss function is established by extracting the structural similarity and feature vectors between the preprocessed image and the noisy image. Structural similarity characterizes the similarity between the preprocessed image and the output image. By calculating the structural similarity between the input and output images, the structural similarity can be minimized, thereby maximizing the difference between the generated blurred image and the preprocessed image. In some implementations, structural dissimilarity can also be utilized.

[0117] The loss functions include classification loss functions such as cross-entropy loss to determine the closeness between the actual output and the expected output, and / or quality loss functions to determine the quality difference between the output image and the expected output, and / or distance loss functions to determine the distance between the feature vectors. Simultaneously, one or more loss functions can be used for multi-objective optimization. Furthermore, the optimized loss function aims to minimize the loss function so that the output image maximizes its difference from the preprocessed image while simultaneously making the feature vectors as close as possible.

[0118] Specifically, a loss function, such as Euclidean distance, can be established based on the distance between the first and second feature vectors. Simultaneously, the distance between the feature vectors can be minimized to make the feature vectors of the generated blurred image and the preprocessed image as close as possible. The noisy image is input into the loss function, and the desired blurred image is output by optimizing the loss function, such as by minimizing the established loss function to optimize intermediate parameters.

[0119] In some implementations, a cross-entropy loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image.

[0120] In some implementations, a cross-entropy loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image, while simultaneously ensuring that the feature vectors of the output blurred image and the preprocessed image are as close as possible.

[0121] In some implementations, a cross-entropy loss function and a distance loss function can be established to maximize the blurring degree between the output blurred image and the preprocessed image, while ensuring that the feature vectors of the output blurred image and the preprocessed image are as close as possible.

[0122] The replacement module 15 replaces the preprocessed image with a blurred image. Once a blurred image is obtained, it replaces the preprocessed image, or in other words, restores the blurred image to the preprocessed image. For example, in a real-time monitoring screen, a loss function is used to blur faces, and then the blurred image is restored to the real-time monitoring screen, so that only blurred faces are displayed instead of clear faces.

[0123] In some implementations, the blurred image can be resized or transformed before being restored to the real-time monitoring screen.

[0124] In some implementations, the image is facial information, which can then be further processed for subsequent applications such as facial recognition.

[0125] Thus, the aforementioned privacy-preserving image processing device, through the acquisition module 11 acquiring a preprocessed image and extracting features from it to obtain a first feature vector, the generation module 12 generating a noisy image based on the preprocessed image and extracting features from it to obtain a second feature vector, the processing module 13 establishing a loss function based on the structural similarity between the preprocessed image and the noisy image, the first feature vector, and the second feature vector, the blurring module 14 processing the noisy image using the loss function to obtain a blurred image, and the replacement module 15 replacing the preprocessed image with the blurred image, enables the blurring of images requiring privacy protection and the replacement of the original image with the blurred image. This allows real-time application scenarios to output blurred images of privacy-preserving areas, thereby minimizing information collection and effectively reducing privacy leaks. Simultaneously, replacing the original image with the blurred image for storage and transmission avoids the risk of leakage or copying to some extent. Furthermore, establishing and optimizing the loss function based on structural similarity and feature vectors to generate the blurred image ensures that the feature vectors are as close as possible to the preprocessed image while blurring the image, thus improving the quality of the blurred image to a certain extent. Furthermore, subsequent facial recognition models can more accurately reconstruct images from blurred images, effectively improving the accuracy of facial recognition and authentication. Moreover, facial recognition models can be used for subsequent business applications such as facial recognition authentication, effectively protecting image privacy while enabling recognition and authentication services.

[0126] Please see Figure 10 In some embodiments, the acquisition module 11 includes an extraction unit 111. The extraction unit 111 is used to perform face detection on a scene image including facial features to obtain a face region, and to determine the face region as a preprocessed image.

[0127] In this way, facial regions can be extracted through face detection models, enabling privacy protection by blurring facial information in face-related business application scenarios.

[0128] Please refer to it again. Figure 9 The image processing apparatus 10 in this embodiment further includes a recognition module 16. The recognition module 16 is used to perform face recognition on the preprocessed face image to authenticate the face.

[0129] Thus, by simultaneously recognizing faces, facial authentication can be performed while protecting privacy through blurring. Alternatively, by adding blurring processing before facial recognition for authentication, facial authentication services can effectively protect facial information privacy and improve the security of facial authentication.

[0130] Please refer to it again. Figure 10 In some embodiments, the extraction unit 111 further includes a size transformation subunit 1111, and the replacement module 15 includes a size replacement unit 151. The size transformation subunit 1112 is used to transform the size of the face region to transform the original first size to a second size to obtain a preprocessed image. The size replacement unit 151 is used to transform the size of the blurred image to the first size to replace the preprocessed image.

[0131] In this way, the size of each extracted face region can be unified by size transformation, which effectively improves the efficiency of subsequent blurring processing.

[0132] In some implementations, the noise image is a random noise image, and the size of the noise image is a second size.

[0133] Please refer to it again. Figure 10 In some embodiments, the processing module 13 includes a first loss function unit 131. The first loss function unit 131 is used to establish an optimized first loss function. To determine the loss function and minimize it to minimize the first loss function to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, E2 is the second feature vector, E0 is the first feature vector, and λ is the penalty term.

[0134] Thus, by determining the loss function through the first loss function described above and minimizing the first loss function, a blurred image can be obtained.

[0135] Please refer to it again. Figure 9 In some embodiments, the image processing apparatus 10 further includes a first comprehensive loss function module 17. The first comprehensive loss function module 17 is used to establish a second loss function. ,in, For function labels, The probability of prediction is given, n is the number of samples, and the label is 1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold. The first comprehensive loss function is minimized to obtain the blurred image. The first comprehensive loss function includes the second loss function and the first loss function.

[0136] Thus, by adding a loss function for face detection in the protected image, the quality of the blurred image can be further improved, allowing the replaced blurred image to more accurately replace the preprocessed image.

[0137] Please see Figure 11 In some embodiments, the processing module 13 further includes a third loss function unit 132, which is used to establish a third loss function. To determine the loss function, a third loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, and cos... The distance is the cosine distance.

[0138] Thus, by determining the loss function through the aforementioned third loss function and minimizing the third loss function, a blurred image can be obtained.

[0139] Please refer to it again. Figure 12 In some embodiments, the image processing apparatus 10 further includes a second comprehensive loss function module 18. The second comprehensive loss function module 18 is used to establish a second loss function. ,in, For function labels, The probability of prediction is given by n, the number of samples is given by n, and the label is given by label=1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold. The second comprehensive loss function is used to minimize the second comprehensive loss function to obtain the blurred image. The second comprehensive loss function includes the second loss function and the third loss function.

[0140] Thus, by adding a loss function for face detection in the protected image, the quality of the blurred image can be further improved, allowing the replaced blurred image to more accurately replace the preprocessed image.

[0141] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media storing a computer program, which, when executed by one or more processors, implements the image processing method of any of the above embodiments.

[0142] In summary, the privacy-preserving image processing method, image processing apparatus, electronic device, and storage medium of this application acquire a preprocessed image, extract features from the preprocessed image to obtain a first feature vector, generate a noisy image based on the preprocessed image, extract features from the noisy image to obtain a second feature vector, establish a loss function based on the structural similarity between the preprocessed image and the noisy image, use the loss function to process the noisy image to obtain a blurred image, and replace the preprocessed image with the blurred image. This method has at least the following beneficial effects:

[0143] First, it allows for blurring of images that require privacy protection.

[0144] Second, by replacing the original image with a blurred image, real-time application scenarios can output a blurred image of the privacy-protected area, thereby minimizing information collection and effectively reducing privacy leaks.

[0145] Third, replacing the original image with a blurred image for storage and transmission avoids the risk of leakage or copying to a certain extent.

[0146] Fourth, furthermore, facial recognition models can be used for subsequent business applications such as facial recognition authentication, enabling effective privacy protection of images while achieving recognition and authentication services.

[0147] Fifth, in the field of facial information, simultaneous facial recognition and blurring processing allows for facial authentication while protecting privacy through blurring. In other words, by adding blurring processing before facial recognition for authentication, facial authentication-related services can effectively protect the privacy of facial information and improve the security of facial authentication.

[0148] VI. Establishing and optimizing a loss function based on structural dissimilarity or similarity and feature vectors to generate blurred images allows for blurring of the image while ensuring that its feature vectors are as close as possible to the preprocessed image, avoiding excessive differences or distortions, such as blurring into a non-face image. This improves the quality of blurred images to a certain extent. Furthermore, subsequent face recognition models can perform more accurate reconstruction based on the blurred image, effectively improving the accuracy of face recognition and authentication.

[0149] 7. By using multiple loss functions for multi-objective optimization, the quality of blurred images can be further improved.

[0150] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0151] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the stated features. In the description of this application, "multiple" means at least two, such as two or three, unless otherwise explicitly specified.

[0152] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0153] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A privacy-preserving image processing method, characterized in that, include: A preprocessed image is acquired, and features are extracted from the preprocessed image to obtain a first feature vector; A noisy image is generated based on the preprocessed image, and a second feature vector is obtained by feature extraction from the noisy image. Based on the structural dissimilarity or structural similarity between the preprocessed image and the noisy image, a loss function is established using the first feature vector and the second feature vector; The noisy image is processed using the loss function to obtain a blurred image; The blurred image is used to replace the preprocessed image.

2. The image processing method according to claim 1, characterized in that, The acquisition of the preprocessed image includes: Face detection is performed on a scene image including facial features to obtain a face region, and the face region is determined as the preprocessed image.

3. The image processing method according to claim 2, characterized in that, The image processing method further includes: Face recognition is performed on the preprocessed image to authenticate the face.

4. The image processing method according to claim 2, characterized in that, The step of performing face detection on the scene image to obtain a face region, and determining the face region as the preprocessed image, further includes: The face region is resized to transform the original first size into a second size to obtain the preprocessed image; The step of replacing the preprocessed image with the blurred image includes: The size of the blurred image is transformed to the first size to replace the face region corresponding to the preprocessed image.

5. The image processing method according to claim 4, characterized in that, The noise image is a random noise image, and the size of the noise image is the second size.

6. The image processing method according to claim 1, characterized in that, The step of establishing the loss function based on the structural dissimilarity or structural similarity between the preprocessed image and the noisy image, using the first feature vector and the second feature vector, includes: Establish the first loss function The loss function is determined, and the first loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, E2 is the second feature vector, and E0 is the first feature vector. This is a penalty item.

7. The image processing method according to claim 6, characterized in that, The image processing method further includes: Establish a second loss function ,in, For function labels, The probability of prediction is given by n, the number of samples is given by n, and the label is given by n = 1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold. The blurred image is obtained by minimizing a first comprehensive loss function, which includes the second loss function and the first loss function.

8. The image processing method according to claim 1, characterized in that, The step of processing the noisy image using the loss function to obtain the blurred image further includes: Establish a third loss function The loss function is determined, and the third loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, and cos Let λ be the cosine distance and λ be the penalty term.

9. The image processing method according to claim 8, characterized in that, The image processing method further includes: Establish a second loss function ,in, For function labels, The probability of prediction is given by n, the number of samples is given by n, and the label is given by n = 1 if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold. The blurred image is obtained by minimizing the second comprehensive loss function, which includes the second loss function and the third loss function.

10. A privacy-preserving image processing apparatus, characterized in that, include: An acquisition module is used to acquire a preprocessed image and perform feature extraction on the preprocessed image to obtain a first feature vector; A generation module is used to generate a noisy image to extract features from the noisy image to obtain a second feature vector, wherein the noisy image corresponds to the preprocessed image; The processing module is used to establish a loss function based on the structural similarity between the preprocessed image and the noisy image, the first feature vector, and the second feature vector; A blurring module is used to process the noisy image using the loss function to obtain a blurred image; A replacement module is used to replace the preprocessed image with the blurred image.

11. The image processing apparatus according to claim 10, characterized in that, The acquisition module includes: The extraction unit is used to perform face detection on a scene image including facial features to obtain a face region, and to determine the face region as the preprocessed image.

12. The image processing apparatus according to claim 11, characterized in that, The image processing device further includes: The recognition module is used to perform face recognition on the preprocessed image to authenticate the face.

13. The image processing apparatus according to claim 11, characterized in that, The extraction unit further includes: A size transformation subunit is used to transform the size of the face region to transform the original first size into a second size to obtain the preprocessed image; The replacement module includes: A size replacement unit is used to transform the size of the blurred image to the first size to replace the face region corresponding to the preprocessed image.

14. The image processing apparatus according to claim 13, characterized in that, The noise image is a random noise image, and the size of the noise image is the second size.

15. The image processing apparatus according to claim 10, characterized in that, The processing module includes: The first loss function unit is used to establish and optimize the first loss function. The loss function is determined and minimized to minimize the first loss function to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, E2 is the second feature vector, E0 is the first feature vector, and λ is the penalty term.

16. The image processing apparatus according to claim 15, characterized in that, The image processing device further includes: The first comprehensive loss function module is used to establish the second loss function. ,in, For function labels, The probability to be predicted is given by n, where n is the number of samples. The label is defined as follows: if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold, then label=1; and The blurred image is obtained by minimizing a first comprehensive loss function, which includes the second loss function and the first loss function.

17. The image processing apparatus according to claim 10, characterized in that, The processing module further includes: The third loss function unit is used to establish the third loss function. The loss function is determined, and the third loss function is minimized to obtain the blurred image, where SSIM is structural similarity, T2 is the intermediate iteration image, T0 is the preprocessed image, and cos Let λ be the cosine distance and λ be the penalty term.

18. The image processing apparatus according to claim 17, characterized in that, The image processing device further includes: The second comprehensive loss function module is used to establish the second loss function. ,in, For function labels, The probability to be predicted is given by n, where n is the number of samples. The label is defined as follows: if the Euclidean distance between the protected image and the scene image is less than a predetermined threshold, then label=1; and The blurred image is obtained by minimizing the second comprehensive loss function, which includes the second loss function and the third loss function.

19. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the image processing method according to any one of claims 1-9.

20. A non-volatile computer-readable storage medium containing a computer program, characterized in that, When the computer program is executed by one or more processors, it implements the image processing method according to any one of claims 1-9.

Images