A privacy protection image retrieval method and system based on ciphertext vision

Abstract

The application discloses a privacy protection image retrieval method and system based on a ciphertext vision, belongs to the technical field of image retrieval, and aims to solve the technical problem that the existing privacy protection image retrieval method cannot simultaneously consider effective protection of image privacy content and high accuracy of image retrieval, and cannot meet current retrieval requirements. The method comprises the following steps: performing encryption processing on a to-be-queried image through a pre-defined reliable encryption algorithm to obtain an encrypted JPEG bit stream of the to-be-queried image; extracting DCT coefficients in the encrypted JPEG bit stream, and extracting corresponding image ciphertext features based on the DCT coefficients; wherein the image ciphertext features at least include a local length sequence feature and a global Huffman coding frequency feature; an unsupervised learning retrieval model based on a visual Transformer is constructed, and the unsupervised learning retrieval model is trained through a self-defined comprehensive loss function.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to a privacy-preserving image retrieval method and system based on dense-state vision. Background Technology

[0002] The goal of image retrieval technology is to search for similar images in a vast image database given a user's query image and then provide the search results to the user. With the rapid development of the mobile internet, the demand for image retrieval technology in business scenarios such as social media and e-commerce is surging. To meet users' needs for storing and retrieving massive amounts of image data, traditional local storage models have shifted to cloud storage, allowing users to access data anytime, anywhere in the cloud. Traditional local computing strategies have also moved to cloud computing, utilizing the abundant computing resources on cloud server clusters to provide image retrieval services.

[0003] However, although cloud computing systems help solve the challenges of limited storage space and insufficient computing resources on the user end, users' private image data is at risk of leakage because cloud servers cannot be fully trusted and are vulnerable to hacker attacks. To achieve image retrieval with privacy protection, the current common strategy is to encrypt the image before uploading it to the cloud server and then use the features of the encrypted image for image retrieval. However, this strategy has the following drawbacks: (1) The encrypted image obtained by traditional image encryption algorithms lacks effective features and cannot be directly used for image data retrieval operations; (2) Existing privacy-preserving image retrieval technologies rely only on a single weak feature, resulting in low retrieval accuracy; (3) The feature retrieval model used is still a simple convolutional neural network, which has poor feature learning capabilities and faces performance bottlenecks.

[0004] It is evident that while existing privacy-preserving image retrieval methods can provide some protection for image content, they also reduce the accuracy of image retrieval to some extent because they hide some key features in the image during the encryption process. Furthermore, the retrieval capabilities of existing retrieval models are relatively outdated, leading to a further decrease in the accuracy of image retrieval and failing to meet current retrieval needs. Summary of the Invention

[0005] This invention provides a privacy-preserving image retrieval method and system based on dense-state vision to solve the following technical problem: existing privacy-preserving image retrieval methods cannot simultaneously ensure effective protection of image privacy content and high accuracy of image retrieval, thus failing to meet current retrieval needs.

[0006] The embodiments of the present invention adopt the following technical solutions:

[0007] On one hand, embodiments of the present invention provide a privacy-preserving image retrieval method based on dense-state vision, the method comprising: encrypting the image to be queried using a predefined reliable encryption algorithm to obtain an encrypted JPEG bitstream of the image to be queried;

[0008] DCT coefficients are extracted from the encrypted JPEG bitstream, and corresponding image ciphertext features are extracted based on the DCT coefficients; wherein the image ciphertext features include at least local length sequence features and global Huffman coding frequency features.

[0009] An unsupervised learning retrieval model based on visual Transformer is constructed, and the unsupervised learning retrieval model is trained using a custom comprehensive loss function;

[0010] The encrypted features of the image are input into the trained unsupervised learning retrieval model for image retrieval, and the retrieval results are returned.

[0011] In one feasible implementation, the image to be queried is encrypted using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried, specifically including:

[0012] The image to be queried is converted from the RGB color gamut to the YUV color gamut to obtain the target image;

[0013] The image data of the target image in the Y, U, and V color channels are encrypted sequentially, including:

[0014] The image data in the current color channel is divided into several image blocks of a preset size, and the image blocks are scrambled using a first preset key to obtain a scrambled image with the image blocks in a randomized order.

[0015] DCT transformation is performed on each image block in the scrambled image to obtain the DCT coefficients of each image block; wherein, the DCT coefficients include DC coefficient DCV and AC coefficient ACV;

[0016] The DCT coefficients of each image block are quantized, and the quantized DCT coefficients are VLI encoded based on the DC Huffman table and the AC Huffman table to obtain the encrypted JPEG bitstream.

[0017] In one feasible implementation, DCT transformation is performed on each image block in the scrambled image to obtain the DCT coefficients of each image block, specifically including:

[0018] If the DC coefficient DCV of the current image block corresponds to a bit 1 in the second preset key, then the DC coefficient DCV of the current image block is inverted to obtain the initial encrypted DC coefficient.

[0019] Traverse the 63 AC coefficients ACV of the current image block. If the current AC coefficient corresponds to a bit of 1 in the third preset key, invert the current AC coefficient ACV to obtain the initial encrypted AC coefficient. Combine the 63 initial encrypted AC coefficients of the current image block into an initial encrypted AC coefficient set.

[0020] Obtain the first bit in the fourth preset key corresponding to the current image block, and the second bit in the fifth preset key corresponding to the current image block, respectively;

[0021] The initial encrypted DC coefficient is XORed with the first bit to obtain the final encrypted DC coefficient;

[0022] The initial encrypted AC coefficient set is XORed with the second bit to obtain the final encrypted AC coefficient set; the final encrypted DC coefficient and the final encrypted AC coefficient set form a matrix to obtain the DCT coefficients of the current image block.

[0023] In one feasible implementation, DCT coefficients are extracted from the encrypted JPEG bitstream, and corresponding image ciphertext features are extracted based on the DCT coefficients, specifically including:

[0024] The encrypted JPEG bitstream is subjected to entropy decoding and inverse quantization to extract the DCT coefficients of each image block;

[0025] Obtain the VLI coding length value corresponding to the DCT coefficients of each image block;

[0026] By traversing all image blocks using the Zig-Zag scanning method, the VLI coding length values ​​corresponding to the DCT coefficients of each image block are combined into a length sequence.

[0027] The length sequences obtained from the Y, U, and V color channels are combined to form the complete local length sequence feature.

[0028] In one feasible implementation, DCT coefficients are extracted from the encrypted JPEG bitstream, and corresponding image ciphertext features are extracted based on the DCT coefficients, specifically including:

[0029] The number of times each coding scheme in the DC Huffman table and the AC Huffman table is used in VLI encoding of the DCT coefficients of each image block, and the corresponding number of times of use is combined into a Huffman coding frequency feature sequence according to the order of each coding scheme in the Huffman table.

[0030] The Huffman-coded frequency feature sequences obtained from the Y, U, and V color channels are combined to form the global Huffman-coded frequency features.

[0031] In one feasible implementation, an unsupervised learning retrieval model based on visual Transformer is constructed, specifically including:

[0032] The unsupervised learning retrieval model is constructed using a visual Transformer network structure as the backbone; the local length sequence features and the global Huffman coding frequency features are used as inputs to the unsupervised learning retrieval model.

[0033] The local length sequence features are mapped to a first input embedding through a linear layer, and the global Huffman encoded frequency features are converted to a second input embedding through four layers of operations.

[0034] In one feasible implementation, the unsupervised learning retrieval model is trained using a custom comprehensive loss function, specifically including:

[0035] During model training, a dynamic dictionary is constructed to store training data in batches; wherein, the dynamic dictionary is a first-in-first-out queue, and when the current batch of training data is enqueued, the oldest batch of training data is dequeued; in the current batch of training data, the sample image x i With x j These are positive samples; the remaining images are negative samples.

[0036] Based on the feature representations extracted from the unsupervised learning retrieval model using the training data, an InfoNCE loss function is constructed.

[0037] The unsupervised learning retrieval model is used as an autoencoder, and a Transformer decoder with the same structure is constructed. The autoencoder and the Transformer decoder are jointly trained using the training data.

[0038] During training, a preset proportion of input features are removed using a preset mask matrix, and the training objective is set to decode the removed input features; based on the mean square error during the decoding process, a mean square error loss function (MAE) is constructed.

[0039] Based on the scaling parameter, the InfoNCE loss function and the MAE loss function are fused into the comprehensive loss function; the model is then optimized based on the comprehensive loss function until the model converges.

[0040] In one feasible implementation, the InfoNCE loss function is constructed based on the feature representations extracted from the unsupervised learning retrieval model using the training data, specifically including:

[0041] according to Constructing the InfoNCE loss function l INCE ;

[0042] Where τ is a hyperparameter; h i with h j The positive sample images x are respectively i With x j The feature representation obtained by the unsupervised learning retrieval model, h k - indicates the feature representation obtained by extracting other negative sample images through the unsupervised learning retrieval model.

[0043] In one feasible implementation, a mean square error (MAE) loss function is constructed based on the mean square error during the decoding process, specifically including:

[0044] according to Construct the MAE loss function l MAE ;

[0045] Wherein, Mask is the preset mask matrix, ~Mask indicates that the input data is operated on using the opposite Mask matrix, and Emb indicates that the input image Img is mapped into the form of feature embedding.

[0046] On the other hand, embodiments of the present invention also provide a privacy-preserving image retrieval system based on dense-state vision, the system comprising:

[0047] The image encryption module is used to encrypt the image to be queried using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried.

[0048] The feature extraction module is used to extract DCT coefficients from the encrypted JPEG bitstream and extract corresponding image ciphertext features based on the DCT coefficients; wherein the image ciphertext features include at least local length sequence features and global Huffman coding frequency features.

[0049] The image retrieval module is used to construct an unsupervised learning retrieval model based on visual Transformer, and to train the unsupervised learning retrieval model through a custom comprehensive loss function; the encrypted features of the image are input into the trained unsupervised learning retrieval model to perform image retrieval, and the retrieval results are returned.

[0050] Compared with the prior art, the privacy-preserving image retrieval method and system based on dense state vision provided by the embodiments of the present invention have the following beneficial effects:

[0051] This invention addresses the shortcomings of existing solutions by proposing a privacy-preserving image retrieval framework based on a dense-state visual Transformer. It innovatively integrates privacy-preserving image retrieval technology with a visual Transformer model, meeting the need for retrieval using multi-layered features. Based on a meticulously designed image encryption process, feature extraction strategy, and unsupervised learning of the visual Transformer, this invention can fully implement multi-layered feature extraction while maintaining the reliability of the encryption algorithm. Furthermore, it designs and trains a feature retrieval model based on the visual Transformer architecture, ultimately improving the privacy protection capability and overall retrieval performance of image retrieval. This provides a novel solution for privacy-preserving image retrieval frameworks in cloud computing scenarios. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0053] Figure 1 A flowchart of a privacy-preserving image retrieval method based on dense-state vision provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the steps of a predefined reliable encryption algorithm provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the structure of a privacy-preserving image retrieval system based on dense state vision, provided in an embodiment of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0057] This invention provides a privacy-preserving image retrieval method based on dense-state vision, such as... Figure 1 As shown, the privacy-preserving image retrieval method based on dense-state vision specifically includes steps S101-S104:

[0058] S101. Encrypt the image to be queried using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried.

[0059] Specifically, firstly, this invention defines a more reliable image encryption algorithm, which obtains the target image by converting the image to be queried from the RGB color gamut to the YUV color gamut. Then, the image data of the target image in the Y, U, and V color channels are encrypted sequentially, including:

[0060] The image data in the current color channel is divided into several image blocks of a preset size. These blocks are then scrambled using a first preset key to obtain a scrambled image with the block order disordered. A Direct Transform (DCT) is performed on each image block in the scrambled image to obtain DCT coefficients for each block. These DCT coefficients include DC coefficients (DCV) and AC coefficients (ACV). Finally, the DCT coefficients for each image block are quantized, and VLI encoding is performed on the quantized DCT coefficients based on the DC Huffman table and the AC Huffman table to obtain an encrypted JPEG bitstream.

[0061] Specifically, DCT transformation is performed on each image block in the scrambled image to obtain the DCT coefficients of each image block, including:

[0062] If the DC coefficient (DCV) of the current image block corresponds to a bit 1 in the second preset key, then the DC coefficient (DCV) of the current image block is inverted to obtain the initial encrypted DC coefficient. The 63 AC coefficients (ACV) of the current image block are traversed. If the current AC coefficient corresponds to a bit 1 in the third preset key, then the current AC coefficient (ACV) is inverted to obtain the initial encrypted AC coefficient. The 63 processed initial encrypted AC coefficients of the current image block are combined into an initial encrypted AC coefficient set.

[0063] Further, the first bit of the current image block corresponding to the fourth preset key and the second bit of the current image block corresponding to the fifth preset key are obtained respectively. The initial encrypted DC coefficient is XORed with the first bit to obtain the final encrypted DC coefficient. The initial encrypted AC coefficient set is XORed with the second bit to obtain the final encrypted AC coefficient set; the final encrypted DC coefficient and the final encrypted AC coefficient set form a matrix to obtain the DCT coefficient of the current image block.

[0064] As a feasible implementation method, Figure 2 This is a schematic diagram of a predefined reliable encryption algorithm step provided in an embodiment of the present invention, such as... Figure 2 As shown, the input to the reliable encryption algorithm is the original image I, and a series of keys are generated using the blake2b hash algorithm, including: a first preset key. Second preset key Third preset key Fourth preset key and the fifth preset key in,

[0065] *∈{Y,U,V}, corresponding to the relevant keys for the three color channels Y, U, and V, respectively. First preset key. The block scrambling key is used to shuffle the order of the 8×8 image block sequence; the second preset key is... and the third preset key This represents the symbolic encryption of the DC and AC coefficients obtained after DCT transformation of an 8×8 image block. The encryption method is as follows: the key has the same number of bits as DCV / ACV; if the current bit of the key is 1, the corresponding bit of DCV / ACV is inverted; if the current bit of the key is 0, the corresponding bit of DCV / ACV remains unchanged. Fourth preset key. and the fifth preset key These represent the XOR stream cipher encryption keys for DCV and ACV, respectively. The encryption method is as follows: the key has the same number of bits as DCV / ACV, and the DCV / ACV sequences are XORed bitwise. The output of this algorithm is an encrypted JPEG bitstream. The ultimate goal is to obtain encrypted JPEG format data that can be used for subsequent processing and transmission.

[0066] Next Figure 2 The algorithm steps are explained step by step:

[0067] Step 1: Color Gamut Conversion: Convert the original image I from the RGB color gamut to the YUV color gamut. JPEG compression is more efficient and reasonable when processed in the YUV color gamut (especially the Y luminance component and the U and V chrominance components). After conversion, it is easier to perform subsequent encryption operations that conform to the JPEG compression process. At the same time, the characteristics of different color gamuts are also conducive to encryption interference from the perspective of color information.

[0068] Step 2: Record width and height: Record the width and length of the original image I as W and H, respectively. These parameters are fundamental for subsequent calculations of the number of blocks and are used to determine the scale of the data to be processed and the block configuration.

[0069] Step 3-22: Perform cyclic encryption processing sequentially for the color channels Y, U, and V. This loop is to encrypt the data of the three color channels Y, U, and V separately, because different channels have differences in image visual presentation and data characteristics, and processing them separately can more subtly interfere with image information.

[0070] Step 4: 8×8 block division: Divide the image I processed for the current color channel into 8×8 blocks. i The image is divided into non-overlapping 8×8 blocks (also called pixel blocks). JPEG compression itself uses 8×8 pixel blocks as the basic processing unit for DCT and other operations. Here, we first divide the image into blocks, and the subsequent encryption operations are carried out around the 8×8 blocks. This not only conforms to the JPEG process, but also destroys the original image structure through block scrambling and other operations.

[0071] Step 5: Block Scrambling: Set the first preset key As a seed, a random sequence is obtained using Python's built-in random sequence generator (the elements in the sequence are integers from 0 to the number of image patches). Then, the order of the image patches is shuffled according to this random sequence, and the I... i Perform block scrambling. The scrambled image blocks are denoted as... And calculate the total number of blocks. By scrambling the positions of the blocks, the spatial structure information of the original image is destroyed, making the originally continuous and regular distribution of image blocks disordered. This provides initial encryption at the macroscopic structural level, making it difficult for attackers to directly reconstruct the original image content from the block arrangement.

[0072] Step 6-21: Process images in a loop: For each block Performing more detailed encryption operations and processing each image block individually can ensure the specificity and integrity of the encryption for each block.

[0073] Steps 7-10: DC coefficient (DCV) symbol encryption related operations:

[0074] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full Record as a block Corresponding to the second preset key The bits in this context. This is the basis for determining the key used to encrypt the DC coefficient symbol for the current block, preparing for subsequent conditional checks.

[0075] Steps 8-10: If but In other words, the sign of the DC coefficient is flipped according to the key bit position. The DC coefficient reflects key information such as the average brightness of pixels within the block. By changing the numerical characteristics of the DC coefficient, the statistical regularity of the original data can be destroyed, and the image data can be interfered with from the perspective of the DC coefficient.

[0076] Steps 11-17: Operations related to ACV symbol encryption:

[0077] Steps 11-16: Iterate through the 63 AC coefficients within the block (i.e., the AC coefficients shown in the diagram). Step 12: Determine the key bits for encrypting the t-th AC coefficient symbol of the current image block. If the bit in steps 13-15 is 1, then That is, the sign of the AC coefficients is flipped based on the key pair. The AC coefficients reflect detailed information such as the difference between pixels and average brightness within a block. The sign flipping further destroys the statistical characteristics of pixel distribution, making the image data more difficult to reconstruct. Step 17 integrates the processed AC coefficients to prepare for subsequent operations.

[0078] Steps 18-20: Stream XOR Encryption:

[0079] Step 18: Define blocks separately Corresponding to the fourth preset key and the fifth preset key Bits in and Determine the key basis for the XOR operation.

[0080] Steps 19-20: By using the XOR operation, the DC and AC coefficient data are further obfuscated using the key, enhancing the randomness of the data and making the encrypted data more difficult to crack. Even if the previous scrambling and symbol encryption are partially broken, the XOR encryption can still increase the difficulty of recovery.

[0081] Step 22: End channel loop. After all three channels (Y, U, V) have been processed according to the above process, the encryption operation of the entire image is completed. Finally, the encrypted JPEG bitstream is obtained by quantizing and encoding the DC coefficient and AC coefficient.

[0082] S102. Extract DCT coefficients from the encrypted JPEG bitstream, and extract the corresponding image ciphertext features based on the DCT coefficients; wherein the image ciphertext features include at least local length sequence features and global Huffman coding frequency features.

[0083] Specifically, the encrypted JPEG bitstream is subjected to entropy decoding and inverse quantization to extract the DCT coefficients of each image block. The VLI encoding length value corresponding to the DCT coefficients of each image block is then obtained.

[0084] Furthermore, by traversing all image blocks using the Zig-Zag scanning method, the VLI encoded length values ​​corresponding to the DCT coefficients of each image block are combined into a length sequence. The length sequences obtained from the Y, U, and V color channels are then combined to form a complete local length sequence feature.

[0085] In one embodiment, after dividing the three components of the original image in the YUV color gamut into 8×8 image blocks, the DCT coefficients (DCV and ACV) of each image block can be obtained through encryption. The length of the VLI code corresponding to each DCT coefficient is calculated during the encryption process. A Zig-Zag scan is used to traverse all image blocks, and the lengths of the VLI codes corresponding to each DCT coefficient are concatenated into a length sequence. For example, if the VLI code length of the DCT coefficient of a certain image block is 20, and the corresponding code lengths of other image blocks are 25, 15, 40, and 50 respectively, then the length sequence is [20, 25, 15, 40, 50]. Finally, the length sequences obtained in the Y, U, and V color gamuts are synthesized into a complete local length sequence feature. At the preset image resolution of this invention, the final local length sequence feature corresponding to each image is 128×8. A two-dimensional vector.

[0086] Furthermore, during VLI encoding, the number of times each coding scheme in the DC Huffman table and AC Huffman table is used for the DCT coefficients of each image block is obtained. Based on the order of each coding scheme in the Huffman table, the corresponding usage counts are combined into a Huffman coding frequency feature sequence. The Huffman coding frequency feature sequences obtained from the Y, U, and V color channels are then combined to form a global Huffman coding frequency feature.

[0087] In one embodiment, during VLI encoding, DC coefficients and AC coefficients are encoded using DC Huffman tables and AC Huffman tables, respectively. The global Huffman coding frequency feature represents the number of times each coding scheme is used in the image encryption process within the two Huffman tables. For example, if the second row of the DC Huffman table is used 10 times during the encoding phase, the corresponding Huffman coding frequency feature is recorded as 10. The DC Huffman table has 12 rows, and the AC Huffman table has 162 rows. Therefore, the dimension of the Huffman coding frequency feature is (12 + 162) × 3 = 522, where 3 represents the three color channels: Y, U, and V.

[0088] S103. Construct an unsupervised learning retrieval model based on visual Transformer, and train the unsupervised learning retrieval model using a custom comprehensive loss function.

[0089] Specifically, an unsupervised learning retrieval model is constructed using a visual Transformer network structure as the backbone; local length sequence features and global Huffman coding frequency features are used as inputs to the unsupervised learning retrieval model.

[0090] In this process, local length sequence features are mapped to the first input embedding through a linear layer, and global Huffman encoded frequency features are converted to the second input embedding through four layers of operations.

[0091] As a feasible implementation, local sequence features LL are mapped to input embeddings LLe through a linear layer, while global Huffman-coded frequency features Hf need to be converted into input embeddings He through four layers of operations to better represent the specific information of different images. Where FC(.) represents a fully connected layer, σ(.) represents the ReLU activation function, and LN(.) represents the layer normalization operation.

[0092] Furthermore, in order to enable the retrieval model to learn the information contained in the features more fully, this invention designs a training method that combines InfoNCE loss and MAE loss.

[0093] First, during model training, a dynamic dictionary is constructed to store training data in batches. This dynamic dictionary is a first-in, first-out (FIFO) queue; when the current batch of training data is enqueued, the oldest batch is dequeued. In the current batch of training data, the sample image x... i With x j The first image is a positive sample, and the remaining images are negative samples.

[0094] Then, based on the feature representations extracted from the unsupervised learning retrieval model using the training data, the InfoNCE loss function is constructed: Where τ is a hyperparameter; h i with h j The positive sample images x are respectively i With x j The feature representation obtained by the unsupervised learning retrieval model, h k - This represents the feature representations extracted from other negative sample images through an unsupervised learning retrieval model. InfoNCE loss addresses the problem of large datasets by constructing a dynamic dictionary with queue and momentum updates.

[0095] Furthermore, the unsupervised learning retrieval model is used as an autoencoder, and a Transformer decoder with the same structure is constructed. The autoencoder and Transformer decoder are jointly trained using training data. During training, a predetermined proportion of input features are removed using a preset mask matrix, and the training objective is set to decode the removed input features to the maximum extent possible.

[0096] Furthermore, based on the mean squared error during the decoding process, the MAE loss function is constructed: Here, Mask is a preset mask matrix, ~Mask indicates that the input data is operated on using the opposite Mask matrix, and Emb indicates that the input image Img is mapped to the form of feature embedding.

[0097] Finally, based on the scaling parameter, the InfoNCE loss function and the MAE loss function are merged into a comprehensive loss function l. total =l INCE +λl MAE λ is a hyperparameter controlling the proportions of the two losses. The model is then optimized based on this combined loss function until it converges.

[0098] S104. Input the encrypted features of the image into the trained unsupervised learning retrieval model to perform image retrieval and return the retrieval results.

[0099] Specifically, the local length sequence features and global Huffman coding frequency features extracted in S102 are input into the trained unsupervised learning retrieval model, enabling it to perform accurate retrieval in the target image database based on the ciphertext features, retrieve images with similarity higher than a preset value, and return them to the user for display.

[0100] To more clearly describe the technical effects and application scenarios of the present invention, the following detailed description is provided through two embodiments in different scenarios:

[0101] Example 1

[0102] Social media platforms store massive amounts of user-uploaded images, which may contain private information such as personal faces and home environments. These platforms need to provide image retrieval functions (e.g., categorized by scene or object) while preventing the misuse or leakage of user privacy, especially during cloud storage and processing. The privacy-preserving image retrieval framework provided by this invention can hide the original pixel information of images through encryption technology, preventing direct access to private content even if the cloud server is untrusted. Simultaneously, by utilizing a cryptographic retrieval model to extract multi-level features (e.g., object contours, scene structure) from encrypted images, it can support content-based semantic retrieval (e.g., "retrieve all photos containing beaches") without decrypting the image. This maintains the platform's image management and retrieval functions while protecting user privacy, improving user experience, and avoiding risks caused by privacy leaks.

[0103] Example 2

[0104] Enterprises need to store confidential images (such as product design drawings, patent images, and commercially sensitive materials) in the cloud and allow authorized employees to search through the image content, but must prevent confidential information from being accessed by the cloud server or unauthorized parties. The encryption algorithm designed in this invention ensures that encrypted images cannot be reverse-engineered, preventing the leakage of design details or trade secrets. Simultaneously, authorized employees can perform similarity searches using encrypted query images. The cloud server only processes the encryption features and does not access the original content, achieving secure collaboration where "data is usable but not visible." This prevents the cloud server from using confidential enterprise images for commercial misuse or leakage, while simultaneously meeting the enterprise's need for efficient management of image resources.

[0105] In addition, embodiments of the present invention also provide a privacy-preserving image retrieval system based on dense-state vision, such as... Figure 3 As shown, the privacy-preserving image retrieval system 300 based on dense-state vision specifically includes:

[0106] Image encryption module 310 is used to encrypt the image to be queried using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried.

[0107] The feature extraction module 320 is used to extract DCT coefficients from the encrypted JPEG bitstream and extract corresponding image ciphertext features based on the DCT coefficients; wherein the image ciphertext features include at least local length sequence features and global Huffman coding frequency features.

[0108] The image retrieval module 330 is used to construct an unsupervised learning retrieval model based on visual Transformer, and to train the unsupervised learning retrieval model through a custom comprehensive loss function; to input the encrypted features of the image into the trained unsupervised learning retrieval model to perform image retrieval, and to return the retrieval results.

[0109] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0110] The foregoing has described specific embodiments of the present invention. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0111] The above description is merely an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present invention should be included within the protection scope of the present invention.

Claims

1. A privacy-preserving image retrieval method based on dense-state vision, characterized in that, The method includes: The image to be queried is encrypted using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried. DCT coefficients are extracted from the encrypted JPEG bitstream, and corresponding image ciphertext features are extracted based on the DCT coefficients; wherein, the image ciphertext features include at least local length sequence features and global Huffman coding frequency features; specifically including: The encrypted JPEG bitstream is subjected to entropy decoding and inverse quantization to extract the DCT coefficients of each image block; the VLI encoding length value corresponding to the DCT coefficients of each image block is obtained. The Zig-Zag scanning method is used to traverse all image blocks, and the VLI encoded length values ​​corresponding to the DCT coefficients of each image block are combined into a length sequence; the length sequences obtained under the Y, U, and V color channels are combined into the complete local length sequence feature. An unsupervised learning retrieval model based on visual Transformer is constructed, and the unsupervised learning retrieval model is trained using a custom comprehensive loss function; wherein, the comprehensive loss function is obtained by fusing the InfoNCE loss function and the MAE loss function; The encrypted features of the image are input into the trained unsupervised learning retrieval model for image retrieval, and the retrieval results are returned.

2. The privacy-preserving image retrieval method based on dense-state vision according to claim 1, characterized in that, The image to be queried is encrypted using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried, specifically including: The image to be queried is converted from the RGB color gamut to the YUV color gamut to obtain the target image; The image data of the target image in the Y, U, and V color channels are encrypted sequentially, including: The image data in the current color channel is divided into several image blocks of a preset size, and the image blocks are scrambled using a first preset key to obtain a scrambled image with the image blocks in a randomized order. DCT transformation is performed on each image block in the scrambled image to obtain the DCT coefficients of each image block; wherein, the DCT coefficients include DC coefficient DCV and AC coefficient ACV; The DCT coefficients of each image block are quantized, and the quantized DCT coefficients are VLI encoded based on the DC Huffman table and the AC Huffman table to obtain the encrypted JPEG bitstream.

3. The privacy-preserving image retrieval method based on dense-state vision according to claim 2, characterized in that, Perform DCT transformation on each image block in the scrambled image to obtain the DCT coefficients of each image block, specifically including: If the DC coefficient DCV of the current image block corresponds to a bit 1 in the second preset key, then the DC coefficient DCV of the current image block is inverted to obtain the initial encrypted DC coefficient. Traverse the 63 AC coefficients ACV of the current image block. If the current AC coefficient corresponds to a bit of 1 in the third preset key, invert the current AC coefficient ACV to obtain the initial encrypted AC coefficient. Combine the 63 initial encrypted AC coefficients of the current image block into an initial encrypted AC coefficient set. Obtain the first bit in the fourth preset key corresponding to the current image block, and the second bit in the fifth preset key corresponding to the current image block, respectively; The initial encrypted DC coefficient is XORed with the first bit to obtain the final encrypted DC coefficient; The initial encrypted AC coefficient set is XORed with the second bit to obtain the final encrypted AC coefficient set; the final encrypted DC coefficient and the final encrypted AC coefficient set form a matrix to obtain the DCT coefficients of the current image block.

4. The privacy-preserving image retrieval method based on dense-state vision according to claim 2, characterized in that, Extracting DCT coefficients from the encrypted JPEG bitstream, and extracting corresponding image ciphertext features based on the DCT coefficients, specifically includes: The number of times each coding scheme in the DC Huffman table and the AC Huffman table is used in VLI encoding of the DCT coefficients of each image block, and the corresponding number of times of use is combined into a Huffman coding frequency feature sequence according to the order of each coding scheme in the Huffman table. The Huffman-coded frequency feature sequences obtained from the Y, U, and V color channels are combined to form the global Huffman-coded frequency features.

5. The privacy-preserving image retrieval method based on dense-state vision according to claim 1, characterized in that, Constructing an unsupervised learning retrieval model based on visual Transformer, specifically including: The unsupervised learning retrieval model is constructed using a visual Transformer network structure as the backbone; the local length sequence features and the global Huffman coding frequency features are used as inputs to the unsupervised learning retrieval model. The local length sequence features are mapped to a first input embedding through a linear layer, and the global Huffman encoded frequency features are converted to a second input embedding through four layers of operations.

6. The privacy-preserving image retrieval method based on dense-state vision according to claim 1, characterized in that, The unsupervised learning retrieval model is trained using a custom comprehensive loss function, specifically including: During model training, a dynamic dictionary is constructed to store training data in batches; wherein, the dynamic dictionary is a first-in-first-out queue, and when the current batch of training data is enqueued, the oldest batch of training data is dequeued; in the current batch of training data, sample images and These are positive samples; the remaining images are negative samples. Based on the feature representations extracted from the unsupervised learning retrieval model using the training data, an InfoNCE loss function is constructed. The unsupervised learning retrieval model is used as an autoencoder, and a Transformer decoder with the same structure is constructed. The autoencoder and the Transformer decoder are jointly trained using the training data. During training, a preset proportion of input features are removed using a preset mask matrix, and the training objective is set to decode the removed input features; based on the mean square error during the decoding process, a mean square error loss function (MAE) is constructed. Based on the scaling parameter, the InfoNCE loss function and the MAE loss function are fused into the comprehensive loss function; the model is then optimized based on the comprehensive loss function until the model converges.

7. The privacy-preserving image retrieval method based on dense-state vision according to claim 6, characterized in that, Based on the feature representations extracted from the unsupervised learning retrieval model using the training data, an InfoNCE loss function is constructed, specifically including: according to Construct the InfoNCE loss function ; in, For hyperparameters; and Positive sample images and The feature representations extracted by the unsupervised learning retrieval model are as follows. This represents the feature representations extracted from other negative sample images through the unsupervised learning retrieval model.

8. The privacy-preserving image retrieval method based on dense-state vision according to claim 6, characterized in that, Based on the mean squared error during the decoding process, a mean squared error (MAE) loss function is constructed, which specifically includes: according to Construct the MAE loss function ; Wherein, Mask is the preset mask matrix. This indicates that the input data is processed using the opposite Mask matrix. Indicates that the input image will be displayed. The mapping is in the form of feature embedding.

9. A privacy-preserving image retrieval system based on dense-state vision, characterized in that, The system includes: The image encryption module is used to encrypt the image to be queried using a predefined reliable encryption algorithm to obtain the encrypted JPEG bitstream of the image to be queried. The feature extraction module is used to extract DCT coefficients from the encrypted JPEG bitstream and extract corresponding image ciphertext features based on the DCT coefficients. The image ciphertext features include at least local length sequence features and global Huffman coding frequency features. Specifically, it includes: performing entropy decoding and dequantization on the encrypted JPEG bitstream to extract the DCT coefficients of each image block; obtaining the VLI coding length value corresponding to the DCT coefficients of each image block; traversing all image blocks using a Zig-Zag scanning method and combining the VLI coding length values ​​corresponding to the DCT coefficients of each image block into a length sequence; and combining the length sequences obtained under the Y, U, and V color channels into the complete local length sequence feature. The image retrieval module is used to construct an unsupervised learning retrieval model based on visual Transformer, and to train the unsupervised learning retrieval model through a custom comprehensive loss function; wherein, the comprehensive loss function is obtained by fusing the InfoNCE loss function and the MAE loss function; the encrypted features of the image are input into the trained unsupervised learning retrieval model to perform image retrieval, and the retrieval results are returned.

Images