Image depth compressed sensing method and system oriented to multi-level privacy protection

Abstract

The invention discloses an image depth compressed sensing method and system for multilevel privacy protection, and relates to the technical field of image processing and information security, and the method comprises the steps: partitioning an image and a privacy mask to obtain a plurality of image blocks and a plurality of corresponding mask blocks; calculating the privacy weight of the image block based on the mask block, and dynamically allocating the sampling rate of the image block based on the privacy weight; judging a privacy block from the plurality of image blocks according to the privacy weight, and applying reversible encryption disturbance to the privacy block based on the key to obtain a disturbed privacy block; performing compressed sampling on the non-privacy block and the disturbed privacy block according to the sampling rate allocated for each image block to obtain an observation value; and inputting the observation value into a pre-trained double-branch reconstruction network, performing reconstruction through a first branch based on the observation value to obtain a privacy protection image, performing reconstruction through a second branch based on the observation value in combination with a key of a completely authorized user to obtain a high-fidelity image, and realizing image privacy protection and high-quality reconstruction under a deep compressed sensing framework.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and information security technology, and in particular to an image deep compressive sensing method and system for multi-level privacy protection. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the rapid development of information technology and the popularity of mobile Internet, image data has become an important part of various service platforms and application programs. The growth of image data not only brings great pressure on storage and transmission, but also poses a serious problem of privacy protection. In particular in the fields of medical, financial, public security, etc., images often contain a large amount of sensitive personal information (such as face, geographic location, identity card, etc.). Therefore, how to efficiently compress and transmit images without revealing user privacy has become a difficult problem to be solved in the current image processing field.

[0004] Since the theory of compressive sensing (CS) was proposed, it has been widely used in the field of image processing due to its significant advantages. The traditional image sampling method is based on the Nyquist sampling theorem, which requires a sampling rate greater than or equal to twice the signal bandwidth. However, the theory of compressive sensing points out that, in the case of a signal having sparsity or approximate sparsity, high-quality signal recovery can still be achieved through sampling at a rate much lower than the Nyquist rate. Therefore, compressive sensing technology provides an efficient solution for image compression, especially in environments with limited storage and bandwidth. Although compressive sensing has made significant progress in image compression, existing privacy protection technologies based on compressive sensing still face many challenges. In particular, in image privacy protection technology, the traditional compressive sensing framework usually relies on optimization algorithms (such as OMP, GPSR, etc.), which have high computational complexity and poor real-time performance in the image reconstruction process. In particular, at low sampling rates, the reconstruction quality is significantly reduced, and distortion phenomena such as blurring and artifacts are likely to occur. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides an image deep compressive sensing method and system for multi-level privacy protection, aiming to realize image privacy protection and high-quality reconstruction under the deep compressive sensing framework.

[0006] To achieve the above purpose, one or more embodiments of the present application provide the following technical solutions: In a first aspect, the present application provides an image deep compressive sensing method for multi-level privacy protection, comprising: Obtain the original image to be processed and its corresponding privacy mask, and preprocess the original image to obtain the preprocessed image; The preprocessed image and privacy mask are divided into blocks to obtain multiple image blocks and corresponding multiple mask blocks; the privacy weight of each image block is calculated based on the corresponding mask block, and the sampling rate of each image block is dynamically allocated based on the privacy weight; The privacy block is determined from multiple image blocks based on the privacy weight, and a reversible cryptographic perturbation is applied to the privacy block based on the key to obtain the perturbed privacy block; Compressed sampling is performed on non-privacy blocks and perturbed privacy blocks according to the sampling rate assigned to each image block to obtain observations; The observations are input into a pre-trained two-branch reconstruction network. The first branch reconstructs a privacy-preserving image based on the observations, and the second branch reconstructs a high-fidelity image based on the observations and the key of a fully authorized user.

[0007] A further technical solution measures privacy strength by the average pixel value within the mask block and uses piecewise linear normalization to obtain privacy weights.

[0008] A further technical solution is that the sampling rate is expressed as:

[0009]

[0010] in, For the first The sampling rate corresponding to each image patch and These are the base sampling rate and the maximum sampling rate, respectively. For the first Privacy weights for each image patch For the first Number of samples per image patch The number of pixels in a single image patch.

[0011] A further technical solution involves identifying image blocks with a privacy weight greater than a threshold as privacy blocks.

[0012] A further technical solution, based on applying a reversible cryptographic perturbation to the privacy block using a key, specifically involves superimposing two layers of cryptographic perturbations, both controlled by the same key, onto the privacy block: one is pixel offset, and the other is Gaussian noise.

[0013] A further technical solution, applying a reversible cryptographic perturbation, is represented as:

[0014] in, This is the perturbed pixel vector. For the first Each original image patch is zero-centered and converted into a quantized pixel vector. For the first Privacy weights for each image patch For indicator functions, For the threshold, For pixel offset, It is Gaussian noise.

[0015] A further technical solution is that the dual-branch reconstruction network adopts a deep compressed sensing network, and its reconstruction process is as follows: Before iterative reconstruction begins, a pseudo-inverse feedback operation is performed on the observations of each image patch to obtain an initial image estimate for each patch; Perform multi-stage iterations, each stage including data fidelity updates and deep prior denoising; After all stages of iterative processing, the first branch and the second branch output their final centered reconstructed images, respectively. The outputs are then subjected to inverse centered processing and numerical truncation to finally obtain privacy-preserving images for semi-authorized users and high-fidelity images for fully authorized users.

[0016] Secondly, the present invention provides an image depth compression sensing system for multi-level privacy protection, comprising: The image and mask acquisition module is configured to: acquire the original image to be processed and the corresponding privacy mask, and preprocess the original image to obtain the preprocessed image; An adaptive sampling rate allocation module is configured to: divide the preprocessed image and privacy mask into blocks to obtain multiple image blocks and corresponding multiple mask blocks; calculate the privacy weight of each image block based on the corresponding mask block; and dynamically allocate the sampling rate of each image block based on the privacy weight. The key-driven perturbation module is configured to: determine the privacy block from multiple image blocks according to the privacy weight, apply a reversible cryptographic perturbation to the privacy block based on the key, and obtain the perturbed privacy block; The compressed sensing sampling module is configured to compress and sample non-privacy blocks and perturbed privacy blocks according to the sampling rate allocated to each image block to obtain observations; The dual-branch reconstruction module is configured to: input observations into a pre-trained dual-branch reconstruction network; reconstruct a privacy-preserving image based on the observations through the first branch; and reconstruct a high-fidelity image based on the observations through the second branch, combined with the key of a fully authorized user.

[0017] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image depth compression sensing method for multi-level privacy protection as described in the first aspect.

[0018] Fourthly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the image depth compression sensing method for multi-level privacy protection as described in the first aspect.

[0019] The above one or more technical solutions have the following beneficial effects: To address the bottleneck in reconstruction performance of existing compressed sensing image privacy protection technologies, this invention proposes an image compressed sensing method and system for multi-level privacy protection. By introducing privacy-weighted adaptive sampling, a key-driven perturbation mechanism, and a bi-branch asymmetric reconstruction network, differentiated outputs for users with different permissions are achieved within a unified end-to-end framework. This ensures privacy protection for semi-authorized users while providing high-quality reconstruction results for fully authorized users, effectively resolving the technical contradiction between protection strength and reconstruction quality in traditional methods.

[0020] This invention optimizes overall sampling efficiency by prioritizing limited sampling resources to privacy-protected critical regions through a privacy-weighted adaptive sampling rate allocation strategy. A key-controlled two-layer perturbation mechanism applies visually irreversible strong noise obfuscation to privacy regions while ensuring the perturbation is reversible for fully authorized users. A parallel architecture combining a lightweight semi-authorized branch and a high-performance fully authorized branch strictly distinguishes access permissions for different users within a single model, reducing system complexity while ensuring the reliability of hierarchical access control. The end-to-end deep network constructed in this invention integrates the physical constraints of compressed sensing with the security requirements of privacy protection, demonstrating significant application value in practical scenarios such as medical image sharing and social media publishing. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of an image compression sensing method for multi-level privacy protection according to an embodiment of the present invention; Figure 2 An overall framework diagram of the image compression sensing method for multi-level privacy protection according to an embodiment of the present invention; Figure 3This is a schematic diagram of the multi-stage iterative reconstruction process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the second branch residual reconstruction network in an embodiment of the present invention; Figure 5 This is a comparison diagram showing the effect of the method described in the embodiments of the present invention with other existing methods in terms of privacy protection strength and reconstruction quality. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] With the rise of deep learning technology, Deep Compressed Sensing (DCS) technology offers a new solution for image compression and privacy protection. DCS introduces neural networks to replace traditional optimization algorithms, making image reconstruction more efficient and accurate. It applies the powerful nonlinear fitting capabilities of deep learning to the compressed sensing framework, forming a deep compressed sensing network. Through end-to-end learning from large amounts of data, DCS can better handle low sampling rates during reconstruction and improve image quality, especially in dealing with noise and artifacts. This technological breakthrough provides an effective solution to the reconstruction performance bottleneck in traditional compressed sensing frameworks for image privacy protection. Particularly at low sampling rates, DCS can significantly improve reconstruction quality, thus better balancing the needs of image compression and privacy protection. This invention, based on the need for multi-level privacy protection, innovatively designs deep compressed sensing. By combining deep compressed sensing with multi-level privacy protection strategies, it effectively achieves efficient image compression, privacy protection, and high-quality reconstruction. It addresses the shortcomings of existing image compressed sensing privacy protection technologies in terms of image reconstruction quality, especially under low sampling rate conditions, where existing technologies suffer from reconstruction performance bottlenecks and are prone to distortions such as blurring and artifacts.

[0027] Example 1 like Figure 1 , Figure 2 As shown, this embodiment discloses an image depth compression sensing method for multi-level privacy protection, which includes the following steps: To address the issues of degraded reconstruction quality at low sampling rates and the difficulty in balancing usability and privacy strength for users with different permissions in existing compressed sensing image privacy protection methods, this embodiment executes the process within an end-to-end deep compressed sensing framework, following a workflow of block-based privacy weight evaluation, weighted adaptive sampling, key-driven encryption perturbation, and dual-branch reconstruction. Fully authorized users holding the key can recover the high-fidelity full image, while semi-authorized users without the key only obtain an image with the privacy region obfuscated.

[0028] S1: Obtain the original image to be processed and the corresponding privacy mask, and preprocess the original image to obtain the preprocessed image; In this embodiment, the privacy mask is a binary mask image of the same size as the original image, generated based on a user-defined privacy region.

[0029] The original image is obtained, and its pixel values ​​are typically in the integer range [0, 255]. Preprocessing includes normalization and zero-centering. The original image is normalized by dividing its pixel values ​​by 255.0, resulting in an input image with pixel values ​​in the floating-point range [0, 1]. To avoid information loss due to data cropping, zero-centering is then performed, shifting the pixel values ​​from [0, 1] to [0, 1]. [0.5, 0.5], to obtain the preprocessed image (hereinafter simply referred to as the image), and then compare the preprocessed image with a binary privacy mask of the same size. Send them to the network together:

[0030] in, This represents the preprocessed image. This represents the input image after normalization to [0,1]; the constant 0.5 is the zero-centering shift, which shifts the pixel value range from [0,1] to [...]. [0.5, 0.5]; After training or inference ends, perform a reverse translation and clip back to [0, 1].

[0031] S2: Divide the preprocessed image and privacy mask into blocks to obtain multiple image blocks and corresponding multiple mask blocks; calculate the privacy weight of each image block based on the corresponding mask block, and dynamically allocate the sampling rate of each image block based on the privacy weight; In this embodiment, the encoder includes a privacy weight analysis module and a perturbation application module, which outputs observations. The image and corresponding privacy mask are synchronized according to... Non-overlapping blocks are used to obtain multiple image blocks and corresponding mask blocks. The number of pixels in a single block (image block or mask block) is recorded. Total number of blocks By calculating the mean of each mask block and normalizing it, the privacy weight of the corresponding image block is obtained. This directly reflects the sensitivity of the content.

[0032] In the privacy weight analysis module, the privacy strength is measured by the average pixel value within the mask block, and the weights are obtained using piecewise linear normalization. :

[0033]

[0034] in, The average value of the mask within the block; For the first Binary masks for image patches; For the first The first image patch The mask value of one pixel; To normalize the privacy weight, represents the th after image segmentation. Privacy weights of blocks; , These are the lower and upper privacy thresholds, respectively. Privacy weights below the lower threshold are all 0, and privacy weights above the upper threshold are all 1. Privacy weights within the range of the upper and lower thresholds are mapped to [0,1]. Indicates will Mapped to [0,1].

[0035] The privacy weight reflects the level of sensitivity and ensures that privacy information in the image is effectively protected. Based on this privacy weight, the compressed sensing sampling rate is dynamically allocated to each image block.

[0036] After obtaining the privacy weight for each image patch, the resources for compressed sampling, i.e., the sampling rate, are dynamically and non-uniformly allocated to each image patch according to this weight. :

[0037]

[0038] in, For the first The sampling rate corresponding to each image patch and These represent the base sampling rate and the maximum sampling rate, respectively. For the first Privacy weights for each image patch For the first Number of samples per image patch (i.e., the number of rows retained in the sampling matrix). Base sampling rate. With the highest sampling rate It can be a fixed value, or it can be dynamically adjusted according to the overall privacy region ratio of the image to further optimize the allocation of sampling resources.

[0039] To meet the global sampling rate ,right Perform iteration sample number correction:

[0040]

[0041] in, For the first In the nth iteration, the 1st The number of intermediate samples in the corrected but not rounded-down stage of each image patch; For the first In the nth iteration, the 1st The actual number of samples per image block; For the first In the iteration, any of the first-order components participating in the global sampling rate calculation... The number of samples per image patch (used to calculate the total number of samples from all current patches); For the first In the nth iteration, the 1st The actual number of samples per image block; To round up Mapped to the interval [0, N]; For the specific image block that is currently being operated on / corrected; This is the index used in the summation formula to traverse all blocks in the entire image.

[0042] Finally, a total budget correction strategy is used to ensure precise control of the overall compression rate; that is, if there is still an integer difference, random fine-tuning is performed until the final compression rate is reached. At the same time, the sampling rate of all image patches will be used. The constructed block sampling rate map is then upsampled to nearest neighbors, that is, the sampling rate of each image block is... Value copied to its corresponding A sampling rate guide map of the same size as the image is obtained from the pixel region. For use during reconstruction.

[0043] Privacy area ( ) Allocate a higher sampling rate, non-privacy region ( A lower sampling rate is then allocated to optimize image compression efficiency while ensuring privacy protection.

[0044] S3: Determine the privacy block from multiple image blocks based on the privacy weight, and apply a reversible cryptographic perturbation to the privacy block based on the key to obtain the perturbed privacy block; In this embodiment, the privacy weight is greater than the threshold. (This embodiment is set) The image block is identified as a privacy block, and two layers are superimposed on the privacy block using the same key. Controlled, reversible cryptographic perturbations. Key. It is a numerical vector with a predetermined dimension (e.g., 128 dimensions), whose element values ​​are generated by a reproducible pseudo-random sampling process (e.g., sampling from a standard normal distribution controlled by a security seed).

[0045] The single key The following two different mechanisms are used to generate two layers of cryptographic perturbation respectively: (1) By key Pixel offset generated as input to a sub-network (small neural network) ; The subnetwork is a lightweight multilayer perceptron (MLP) used to generate pixel offsets. Its specific structure consists of two fully connected layers: the first layer stores the key... The 128-dimensional feature is mapped to a 64-dimensional hidden layer and activated by ReLU. The second layer maps the hidden features to the image channel dimension (e.g., 3 channels), and finally passes them through a Sigmoid activation function, outputting pixel offset values ​​in the range [0, 1]. .

[0046] (2) By key A random seed is derived, and zero-mean Gaussian noise is generated from this seed. .

[0047] If it is RGB input ( To enhance the randomness and security of the disturbance, the total noise... Depend on A statistically independent two-dimensional Gaussian noise block Stacked along the channel dimension:

[0048] in, For the number of channels, It is represented as an independent Gaussian noise block of channel 1.

[0049] If a grayscale input is detected, the single-channel noise perturbation is copied to the three channels to maintain a consistent grayscale appearance.

[0050] in, To copy single-channel noise aisle.

[0051] A key-driven encryption perturbation mechanism is adopted to generate two types of encryption perturbations: one is a tiny pixel offset, and the other is high-intensity, zero-mean Gaussian noise. In the perturbation application module, encryption perturbation is only performed on image blocks that are determined to be privacy blocks according to privacy weights, ensuring that privacy information is not leaked.

[0052] The process of applying a disturbance can be represented as:

[0053] in, The perturbed pixel vector will be used as the final input for compressed sampling; For the first Each original image patch is zero-centered and quantized into a pixel vector (original image patch (zero-centered)). For indicator functions, The value is 1 when the condition is met and 0 when the condition is not met. A value of 0 indicates that the image patch is a non-privacy patch. Do not apply any disturbance.

[0054] S4: Compress and sample the non-privacy blocks and the perturbed privacy blocks according to the sampling rate assigned to each image block to obtain the observations; In this embodiment, compressed sampling is performed on all image blocks (including non-privacy blocks and perturbed privacy blocks). The compressed sampling is based on a learnable fundamental sampling matrix that is initialized with SVD and optimized end-to-end. (Measurement matrix) to perform dynamic sampling based on the sampling rate of each block.

[0055] in accordance with From learnable fundamental matrices Select rows to form a block-specific sampling matrix , Only in China Non-zero rows. Use a binary row selection matrix. Indicates the choice:

[0056]

[0057] in, According to the first Number of samples per block The generated binary row selection matrix, ,forward The first diagonal element has a value of 1, then The diagonal elements are all 0, used to extract values ​​from the base matrix. The required number of row vectors are precisely selected to form the sampling matrix specific to this block. The observed values ​​were obtained through compressed sampling. ; The observation vector after zero padding (only the first part) The component corresponding to each selected row is non-zero.

[0058] S5: Input the observations into a pre-trained two-branch reconstruction network. The first branch reconstructs a privacy-preserving image based on the observations, while the second branch, based on the observations and combined with the key of a fully authorized user, reconstructs a high-fidelity image. In other words, the first branch reconstructs an image for semi-authorized users, showing only the non-privacy areas, while the second branch reconstructs the complete image for fully authorized users.

[0059] The dual-branch reconstruction network (decoder) consists of a first branch and a second branch, both employing a deep compressed sensing network. This deep compressed sensing network comprises data fidelity and deep prior inpainting. The difference between the two branches is: (1) Different complexity: The first branch (path A) adopts a lightweight network structure, which is a network structure with fewer network layers, fewer channels, or fewer parameters. Because it does not need to reconstruct the entire image, the privacy region is noisy. It only needs to ensure high-quality reconstruction of the non-privacy region. This branch network cannot remove noise from the privacy region. The second branch (path B) adopts a high-performance (high-complexity) reconstruction network structure, which needs to reconstruct the entire image with high quality and ensure the removal of noise from the privacy region. Therefore, it has a deeper number of layers and a stronger nonlinear fitting ability to cope with complex denoising and decryption tasks.

[0060] like Figure 4 As shown, the residual reconstruction network of the second branch includes multiple upsampling base blocks, a bottleneck module, and multiple downsampling base blocks connected in sequence. Each upsampling base block includes a convolutional layer, a residual block, and a downsampling module connected in sequence. Each downsampling base block includes a convolutional layer, a residual block, and an upsampling module connected in sequence. The bottleneck module includes a 1×1 convolutional layer, a 3×3 convolutional layer, a 1×1 convolutional layer, and a residual block connected in sequence. Multi-channel feature maps are input into the residual reconstruction network of the second branch for processing, and residual feature maps are output.

[0061] (2) Different input conditions: The first branch has no key input and only receives the intermediate image of the iteration and the sampling rate guide map; the second branch has key input and additionally receives the embedding features and perturbation map features generated by the key as key conditions to guide the network to decrypt noise.

[0062] (3) Different functional objectives: The first branch lacks key information and cannot eliminate encryption perturbations in the privacy area. It can only output the image after the privacy area is blurred. The second branch uses key features to identify and reverse encryption perturbations, thereby recovering a high-fidelity complete image.

[0063] A dual-branch reconstruction network (deep compressed sensing network) is used to progressively reconstruct the compressed image, ensuring that fully authorized users with the key can recover the complete image, while partially authorized users without the key can only recover the non-privacy parts. This provides different image access permissions for users with different privileges. The process is as follows: A multi-stage reconstruction network architecture based on the concept of deep unfolding is adopted. The network consists of N cascaded reconstruction phases and includes two parallel asymmetric reconstruction paths designed specifically for users with different authorization levels: path A (first branch, semi-authorized user path) and path B (second branch, fully authorized user path). Path A, the semi-authorized user path, aims to reconstruct an image where non-privacy regions are clear and privacy regions are effectively obfuscated. This path consists of a series of lightweight reconstruction network modules. In each reconstruction stage, the module only receives publicly available compressed observations. And its related information as input, without a key Under these conditions, the image is iteratively optimized. Due to the lack of key information required for decryption, this path can only reconstruct non-privacy regions, thus protecting privacy information. Path B, the fully authorized user path, aims to recover the complete original image without loss or with high fidelity. This path consists of a high-performance, more expressive reconstruction network. In each reconstruction stage, in addition to receiving publicly available observation information, a key is also provided. As a crucial additional conditional input, the network uses this key information to learn to identify and precisely reverse (counteract) perturbations applied during the encryption phase.

[0064] Compressed observations for each image patch before iterative reconstruction begins. Perform a pseudo-reverse feedback operation to obtain an initial image estimate for each block. :

[0065] in, For the first Initial reconstruction of each block, For the first The transpose of the sampling matrix used by each block. Initial image estimation is obtained by overlaying the entire image using Fold. That is, stacking the initial reconstructions of all blocks together to form the initial image estimate of the entire map domain. As the starting point for subsequent iterations.

[0066] like Figure 3 As shown, the multi-step iterative reconstruction process: Build Each phase, from 1 to... Iterate continuously until the 1st In each reconstruction stage, the reconstructed image from the previous stage... Update to obtain the output of the current stage. Each stage of the update includes two sub-steps: data fidelity update and depth prior denoising. Specifically, each stage first performs a gradient descent step in the block domain to ensure data fidelity, and then performs depth prior denoising in the image domain. That is, after completing the gradient descent update, in order to extract spatial features using the deep neural network, a folding operation is needed to fold all the updated vector blocks. The image domain data is reconstructed by rearranging the data according to their spatial location in the original image. It is then fed into a subsequent convolutional network for deep prior repair.

[0067] (1) Data fidelity: In the block domain, for Each block Perform one step of gradient descent to minimize the observation domain error and obtain the intermediate block domain. , where each block The update formula is:

[0068]

[0069] in, for The reconstructed image output by the stage. For the first The first stage after data fidelity update An intermediate image patch, For the stage Learnable step size for t Image after phased gradient descent. Then each block... A preliminary estimate for the current stage is formed by combining Fold operations. Then, the data was verified. Send in deep prior repair.

[0070] (2) Depth prior repair (residual reconstruction): The input path is concatenated with the conditional graph (including the sampling rate guide graph, embedded feature map, and perturbation map features). The subnet is used to output the residual and perform residual connections:

[0071]

[0072] in, For residual feature maps, For U-Net type subnets, the parameters vary depending on the path; This is a sampling rate guide graph. To embed feature maps, This is a feature of the perturbation map. for The image after phase reconstruction, This is a residual connection.

[0073] The deep prior repair step utilizes the spatial information processing capabilities of convolutional networks in the image domain to extract residual features. This is followed by receiving data with fidelity. Then, it is input into a depth prior module. In the middle, residual learning is used to... Repair and denoise removal are performed to obtain residual feature map Finally, the residual feature map After data fidelity Perform residual connections to obtain the output of the current stage. Deep Prior Module The two parallel reconstruction paths have different network structures and conditional inputs, consistent with the previous description, namely, path A uses a lightweight network. Furthermore, without key input, path B uses a high-performance reconstruction network. Furthermore, the presence or absence of a key input determines whether there is the ability to decrypt encrypted perturbations in the privacy area.

[0074] Path A (semi-authorized user) only uses With sampling rate guide graph .path (Fully authorized user) in With sampling rate guide graph In addition to the above, use a key key An embedding feature map is generated using an independent embedding network. Embedded feature maps The perturbation map features are generated by processing the channel mean of Gaussian noise m through a dedicated feature embedding network (a small convolutional network). Perturbation map features Used to identify and reverse privacy disturbances.

[0075] The entire reconstruction process is a continuous cycle of iteration, going through all After iterative processing through each reconstruction stage, path A and path B each output their final centered reconstructed images. These two outputs are then subjected to inverse centered processing and numerical truncation to ultimately obtain a privacy-preserving image for semi-authorized users. and high-fidelity images for fully licensed users Specifically, complete... After each stage, the zero center region results for the two paths are obtained respectively. and Perform decentralization and prune to [0,1]:

[0076]

[0077] in, For privacy protection of the image (reconstructed image of path A). It is a high-fidelity image (the reconstructed image of path B). For cropping operations, this is used to strictly limit pixel values ​​to the range [0, 1].

[0078] Path A (e.g., using network A) is directed to a semi-authorized user (user A), and can only reconstruct the graph where privacy areas are obfuscated. Path B (e.g., using network B) is directed to fully authorized users (user B) and can restore the entire map in high fidelity. Furthermore, when it is necessary to protect the privacy of images... and high-fidelity images (All values ​​are in the range [0, 1]) When used for saving or displaying, the pixel value is multiplied by 255.0 and converted to an 8-bit integer, then inversely normalized to the original pixel display range of [0, 255].

[0079] To obtain a high-quality reconstructed image, the reconstruction loss function consists of the following three weighted terms: Fully authorized reconstruction loss (Path B) is:

[0080] in, This is the original image.

[0081] Fidelity loss in semi-authorized non-privacy areas (Path A) is:

[0082] in, For the non-privacy region mask, a binary tensor, where the values ​​of the public region are 1 and the rest are 0; This is element-wise multiplication.

[0083] Loss due to privacy obfuscation of semi-authorized user areas (Path A) is:

[0084] in, For the privacy region mask, a binary tensor, where the privacy region values ​​are 1 and the rest are 0; Gaussian noise block The noisy target image is obtained by combining (Fold) and then performing inverse centering (adding 0.5 and cropping to the [0,1] range).

[0085] These three sub-losses are weighted by adjustable coefficients. We perform weighted summation to construct the final total loss function, which is used for overall model optimization.

[0086] in, This represents the total loss.

[0087] In summary, this invention combines deep compressed sensing for image privacy protection, thereby achieving higher quality reconstruction than traditional compressed sensing-based image privacy protection methods. It balances privacy protection with controllable and authoritative decryption, and significantly reduces computational and storage overhead.

[0088] In this embodiment, to verify the effectiveness of the proposed method, it is experimentally compared with existing mainstream compressed sensing image privacy protection methods such as Multi-level reversible data anonymization via compressive sensing and data hiding (Yamac et al.) and Multitiered reversible data privacy protection scheme for IoT based on compression sensing and digital watermarking (Suo et al.). The experimental results are as follows: Figure 5 As shown. Performance data comparisons use Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), commonly used in image reconstruction, as the main evaluation metrics: As shown in Table 1, the given size is The original image and reconstructed images Its mean square error (MSE) is:

[0089] The formula for calculating PSNR (unit: dB) is:

[0090] in, This represents the maximum possible value of an image pixel.

[0091] Table 1. Comparison of PSNR between the present invention and other existing methods.

[0092] SSIM is a metric for measuring the structural similarity between two images, comparing them in terms of brightness, contrast, and structure. As shown in Table 2, for the original image... and reconstructed images :

[0093] in, yes The average value, yes The average value, yes variance yes variance yes and The covariance. , Here, a constant is used to maintain stability. The dynamic range of pixel values. , .

[0094] Table 2. Comparison of the present invention with other existing methods in SSIM.

[0095] For fully authorized users (User B), a higher PSNR value and a SSIM value closer to 1 indicate higher fidelity and less distortion in the reconstructed image. For semi-authorized users (User A), higher PSNR and SSIM values ​​are better in non-privacy areas of the image, while lower PSNR and SSIM values ​​in privacy areas indicate greater deviation between the reconstructed result and the original privacy content, less information leakage, and better privacy protection.

[0096] Example 2 This embodiment discloses an image depth compression sensing system for multi-level privacy protection, including: The image and mask acquisition module is configured to: acquire the original image to be processed and the corresponding privacy mask, and preprocess the original image to obtain the preprocessed image; An adaptive sampling rate allocation module is configured to: divide the preprocessed image and privacy mask into blocks to obtain multiple image blocks and corresponding multiple mask blocks; calculate the privacy weight of each image block based on the corresponding mask block; and dynamically allocate the sampling rate of each image block based on the privacy weight. The key-driven perturbation module is configured to: determine the privacy block from multiple image blocks according to the privacy weight, apply a reversible cryptographic perturbation to the privacy block based on the key, and obtain the perturbed privacy block; The compressed sensing sampling module is configured to compress and sample non-privacy blocks and perturbed privacy blocks according to the sampling rate allocated to each image block to obtain observations; The dual-branch reconstruction module is configured to: input observations into a pre-trained dual-branch reconstruction network; reconstruct a privacy-preserving image based on the observations through the first branch; and reconstruct a high-fidelity image based on the observations through the second branch, combined with the key of a fully authorized user.

[0097] Example 3 The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0098] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 1.

[0099] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0100] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0102] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A multi-level privacy oriented image depth compressive sensing method, characterized in that, The method comprises the following steps: obtain an original image to be processed and a corresponding privacy mask, and pre-process the original image to obtain a pre-processed image; block the pre-processed image and the privacy mask to obtain a plurality of image blocks and a plurality of corresponding mask blocks; calculate the privacy weight of each image block based on the corresponding mask block, and dynamically assign the sampling rate of each image block based on the privacy weight; determine the privacy block from the plurality of image blocks according to the privacy weight, and apply reversible encryption disturbance to the privacy block based on the key to obtain a disturbed privacy block; compress and sample the non-privacy block and the disturbed privacy block according to the sampling rate assigned to each image block to obtain an observation value; input the observation value into a pre-trained double-branch reconstruction network, reconstruct a privacy-protected image based on the observation value through a first branch, and reconstruct a high-fidelity image based on the observation value through a second branch in combination with the key of the fully authorized user.

2. The image depth compressive sensing method for multi-level privacy protection according to claim 1, wherein, The privacy strength is measured by the mean value of the pixels in the mask block, and the privacy weight is obtained by piecewise linear normalization.

3. The image depth compressive sensing method for multi-level privacy protection of claim 1, wherein, The sampling rate is represented as: wherein, is the sampling rate corresponding to the th image block, and are the base sampling rate and the highest sampling rate, respectively, is the privacy weight of the th image block, is the number of samples of the th image block, is the number of pixels of a single image block.

4. The image depth compressive sensing method for multi-level privacy protection of claim 1, wherein, An image block with a privacy weight greater than a threshold value is determined as a privacy block.

5. The image depth compressive sensing method for multi-level privacy protection of claim 1, wherein, Applying reversible encryption disturbance to the privacy block based on the key specifically means superimposing two layers of encryption disturbance controlled by the same key on the privacy block, one being pixel offset and the other being Gaussian noise.

6. The image depth compressive sensing method oriented to multi-level privacy protection according to claim 5, characterized in that, Applying reversible encryption disturbance is represented as: wherein, is the perturbed pixel vector, is the zero-centered and vectorized pixel vector of the th original image block, is the privacy weight of the th image block, is the indicator function, is the threshold value, is the pixel offset, is the Gaussian noise.

7. The image depth compressive sensing method for multi-level privacy protection of claim 1, wherein, The double-branch reconstruction network uses a deep compressed sensing network, and the reconstruction process is as follows: Before the start of iterative reconstruction, perform a pseudo-inverse feedback operation on the observation value of each image block to obtain an initial image estimate of each block; perform multi-stage iteration, each stage including data fidelity update and deep prior denoising; After all-stage iteration, the first branch and the second branch respectively output their final centralized reconstruction images, perform inverse centralization processing on the outputs and perform numerical truncation to finally obtain a privacy-protected image for semi-authorized users and a high-fidelity image for fully authorized users.

8. A multi-level privacy oriented image depth compressive sensing system, characterized in that, The method comprises the following steps: an image and mask acquisition module configured to obtain an original image to be processed and a corresponding privacy mask, and pre-process the original image to obtain a pre-processed image; an adaptive sampling rate allocation module configured to block the pre-processed image and the privacy mask to obtain a plurality of image blocks and a plurality of corresponding mask blocks, calculate the privacy weight of each image block based on the corresponding mask block, and dynamically assign the sampling rate of each image block based on the privacy weight; a key-driven disturbance module configured to determine the privacy block from the plurality of image blocks according to the privacy weight, and apply reversible encryption disturbance to the privacy block based on the key to obtain a disturbed privacy block; a compressed sensing sampling module configured to compress and sample the non-privacy block and the disturbed privacy block according to the sampling rate assigned to each image block to obtain an observation value; a double-branch reconstruction module configured to input the observation value into a pre-trained double-branch reconstruction network, reconstruct a privacy-protected image based on the observation value through a first branch, and reconstruct a high-fidelity image based on the observation value through a second branch in combination with the key of the fully authorized user.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of a method for multi-level privacy preserving image depth compressive sensing according to any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of a method for multi-level privacy preserving image depth compressive sensing according to any one of claims 1-7 when executing the program.

Images