Image query method and device, equipment and storage medium

Abstract

The invention discloses an image query method, device and equipment and a storage medium, and relates to the technical field of image processing.The method comprises the steps that trap door query information sent by a query user is received, and the trap door query information is determined by the query user based on a query image and a user token; determining the similarity between the trap door query information and the encrypted index information of each image, and determining target encrypted index information according to each similarity; and determining an encrypted query result according to the target encrypted index information, and sending the encrypted query result to the query user, so that the query user determines a query result by decrypting the encrypted query result. According to the technical scheme, the user behavior association risk caused by trap door repetition in the prior art is solved, the non-linkability of trap doors in a multi-user scene is ensured, and the problem of cross privacy leakage in the multi-user scene is avoided.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an image query method, apparatus, device and storage medium. Background Technology

[0002] With the widespread adoption of cloud computing and artificial intelligence technologies, image retrieval technology is increasingly used in fields requiring high-precision image retrieval and data privacy protection, such as security monitoring, e-commerce, and medical imaging. To protect privacy, images are typically encrypted before being uploaded to servers, but this encryption process makes image retrieval even more difficult. Therefore, how to effectively retrieve images while protecting their privacy has become a pressing technical challenge.

[0003] In existing technologies, after the image owner extracts the image features, they upload the image features to the server. The server calculates the Euclidean distance between the query image features and the image features based on the k-Nearest Neighbors (KNN) algorithm, and determines the query result based on the Euclidean distance.

[0004] However, traditional encrypted image retrieval schemes typically only support single-user environments, making it difficult to adapt to the needs of multi-user collaborative retrieval. Furthermore, the server may return tampered or incorrect retrieval results, leading to the leakage of user privacy or unreliable retrieval results. Summary of the Invention

[0005] This invention provides an image query method, apparatus, device, and storage medium to enable encrypted image querying in multi-user scenarios.

[0006] In a first aspect, embodiments of the present invention provide an image query method, including:

[0007] The system receives trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token; determines the similarity between the trapdoor query information and the encrypted index information of each image, and determines target encrypted index information based on the similarity; determines an encrypted query result based on the target encrypted index information, and sends the encrypted query result to the querying user so that the querying user can determine the query result by decrypting the encrypted query result.

[0008] The technical solution of this invention provides an image query method, comprising: receiving trapdoor query information sent by a query user, wherein the trapdoor query information is determined by the query user based on a query image and a user token; determining the similarity between the trapdoor query information and the encrypted index information of each image, and determining target encrypted index information based on the similarity; determining an encrypted query result based on the target encrypted index information, and sending the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result. In the above technical solution, after the server receives the trapdoor query information sent by the query user, it initiates an image query, determines the target encrypted index information based on the similarity between the trapdoor query information and the encrypted index information of each image, and determines the encrypted image corresponding to the target encrypted index information as the encrypted query result, thereby realizing image query based on encrypted index information, and thus realizing encrypted image query. After sending the encrypted query result to the query user, the query user can decrypt the encrypted query result to determine the query result, thus realizing the determination of the query result. By assigning a unique user token to each querying user and constructing trapdoor query information based on the user token, it ensures that different querying users generate different trapdoor query information for the same query image. This solves the risk of user behavior association caused by duplicate trapdoors in traditional solutions, guarantees the unlinkability of trapdoors in multi-user scenarios, and avoids cross-privacy leakage issues in multi-user scenarios.

[0009] Furthermore, the process by which the querying user determines the trapdoor query information based on the query image includes: determining the image features of the query image; performing feature expansion on the image features based on the user token to obtain extended features; performing feature splitting on the extended features to obtain a first split feature; and processing the first split feature using an invertible matrix to obtain the trapdoor query information.

[0010] Further, determining the similarity between the trapdoor query information and the encrypted index information of each image includes: for each encrypted index information, performing a secure inner product operation on the trapdoor query information and the encrypted index information, and determining the operation result as the similarity corresponding to the encrypted index information.

[0011] Further, determining the target encrypted index information based on each of the similarities includes: sorting each of the similarities in descending order; and determining the encrypted index information corresponding to the top N similarities as the target encrypted index information, where N is a natural number.

[0012] Further, determining the encrypted query result based on the target encrypted index information includes: determining the encrypted image corresponding to the target encrypted index information as the encrypted query result.

[0013] Furthermore, the process by which the querying user decrypts the encrypted query result to determine the query result includes: determining the querying user's content key based on the querying user's private key; and decrypting the encrypted query result based on the content key to obtain the query result.

[0014] Furthermore, after determining the content key of the querying user based on the querying user's private key, the method further includes: for each encrypted query result, verifying the content key of the querying user based on the encrypted query result, and determining whether the verification result is successful or unsuccessful.

[0015] Secondly, embodiments of the present invention also provide an image query device, comprising:

[0016] A receiving module is used to receive trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token;

[0017] The determining module is used to determine the similarity between the trapdoor query information and the encrypted index information of each image, and to determine the target encrypted index information based on the similarity.

[0018] The query module is used to determine the encrypted query result based on the target encrypted index information, and send the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

[0019] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform an image query method as described in any of the first aspects.

[0020] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, characterized in that the computer-executable instructions, when executed by a computer processor, are used to perform the image query method as described in any of the first aspects.

[0021] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the image query method as provided in the first aspect.

[0022] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the image query device, or it may be packaged separately from the processor of the image query device; this application does not impose any limitations on this.

[0023] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0024] In this application, the name of the aforementioned image query device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0025] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of an image query method provided in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of an image query system provided in an embodiment of the present invention;

[0029] Figure 3 A flowchart of another image query method provided in an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of the feature extraction device provided in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the structure of an image query device provided in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0034] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0035] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0036] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0037] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0038] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0040] Figure 1This is a flowchart illustrating an image query method provided in an embodiment of the present invention. This embodiment is applicable to situations requiring encrypted image queries in multi-user scenarios. The method can be executed by an image query device, such as... Figure 1 As shown, the specific steps include the following:

[0041] Step 110: Receive the trapdoor query information sent by the querying user.

[0042] The trapdoor query information is determined by the querying user based on the query image and the user token.

[0043] Figure 2 This is a schematic diagram of an image query system provided in an embodiment of the present invention, such as... Figure 2 As shown, the image query system includes a server, an image owner, a query user, a proxy server, and a key generation center. The image owner is used to determine the encrypted index of the image and encrypt the image. The proxy server is used to re-encrypt the image. The query user is used to generate a query trapdoor based on the query image. The server is used to store the encrypted index and the encrypted image, and is also used to determine the query result based on the query trapdoor. The key generation center is used to generate public parameters and secret parameters, and to generate keys for the image owner and the query user.

[0044] Specifically, when a user needs to perform an image search, the first step is to determine the trapdoor query information based on the query image. This involves first extracting features from the query image to identify its corresponding image features. Then, feature expansion and splitting are performed on these image features to determine the first splitting feature. For example, the extended features can be obtained by expanding the image features based on the user's token. The first splitting feature is then obtained by splitting the extended features. Finally, the trapdoor query information is determined by processing the first splitting feature using an invertible matrix, generating the trapdoor query information for the user to perform the image search. The user can then send the trapdoor query information to the server for image retrieval based on the trapdoor query information.

[0045] Of course, when multiple users perform image queries, by generating trapdoor query information for each user to perform image queries, the trapdoor query information generated by different users for the same image is different, thus avoiding the problem of cross-privacy leakage in multi-user scenarios.

[0046] In this embodiment of the invention, the server determines to enable image query by receiving the trapdoor query information sent by the querying user.

[0047] Step 120: Determine the similarity between the trapdoor query information and the encrypted index information of each image, and determine the target encrypted index information based on the similarity.

[0048] The server stores encrypted index information and encrypted information for each image. The encrypted index information is obtained by the image owner through feature extraction, feature expansion and splitting to determine the second split feature, and then processing this second split feature using an invertible matrix to determine the encrypted index information. The encrypted information is obtained by the image owner encrypting the image using a key, sending the encrypted image to the proxy server, and then the proxy server re-encrypting the encrypted image using a conversion key.

[0049] Specifically, the server stores encrypted index information and encrypted information corresponding to images owned by the image owner. After receiving a trapdoor query from a user, the server needs to perform an image query based on the trapdoor query information to determine the query result. The encrypted information corresponding to the image is obtained by encrypting the image twice, making it difficult to query directly. The encrypted index information corresponding to the image is constructed based on image features and can be used for image querying. Therefore, the encrypted index corresponding to the image can be queried through the trapdoor query information. Specifically, the similarity between the trapdoor query information and the encrypted index information corresponding to each image can be determined, and all similarity scores are sorted in descending order. The encrypted index information corresponding to the top N similarity scores is determined as the target encrypted index information.

[0050] In this embodiment of the invention, the similarity between the trapdoor query information and the encrypted index information of each image is determined, and the target encrypted index information is determined based on the similarity, thereby realizing image query based on the encrypted index information of the image.

[0051] Step 130: Determine the encrypted query result based on the target encrypted index information, and send the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

[0052] Specifically, the encrypted image within the encrypted information corresponding to the target encrypted index information stored on the server can be identified as the encrypted query result, and this encrypted query result can be sent to the querying user. After receiving the encrypted query result, the querying user can decrypt it to determine the query result.

[0053] In this embodiment of the invention, the encrypted image in the encrypted information corresponding to the target encrypted index information is determined as the encrypted query result, thereby realizing the determination of the encrypted query result. After the encrypted query result is sent to the query user, the query user can decrypt the encrypted query result to determine the query result, thus realizing the determination of the query result.

[0054] The image query method provided in this invention includes: receiving trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token; determining the similarity between the trapdoor query information and the encrypted index information of each image, and determining target encrypted index information based on the similarity; determining an encrypted query result based on the target encrypted index information, and sending the encrypted query result to the querying user so that the querying user can determine the query result by decrypting the encrypted query result. In the above technical solution, after the server receives the trapdoor query information sent by the querying user, it initiates an image query, determines the target encrypted index information based on the similarity between the trapdoor query information and the encrypted index information of each image, and determines the encrypted image in the encrypted information corresponding to the target encrypted index information as the encrypted query result, thereby realizing image query based on encrypted index information, and thus realizing encrypted image query. After sending the encrypted query result to the querying user, the querying user can determine the query result by decrypting the encrypted query result, thus realizing the determination of the query result. By assigning a unique user token to each querying user and constructing trapdoor query information based on the user token, it ensures that different querying users generate different trapdoor query information for the same query image. This solves the risk of user behavior association caused by duplicate trapdoors in traditional solutions, guarantees the unlinkability of trapdoors in multi-user scenarios, and avoids cross-privacy leakage issues in multi-user scenarios.

[0055] Figure 3 This is a flowchart of another image query method provided by an embodiment of the present invention. This embodiment is a specific modification based on the above embodiments. Figure 3 As shown, in this embodiment, the method may further include:

[0056] Step 310: Receive the trapdoor query information sent by the querying user.

[0057] The trapdoor query information is determined by the querying user based on the query image and the user token.

[0058] In one implementation, the process by which the querying user determines the trapdoor query information based on the query image includes:

[0059] The image features of the query image are determined; the image features are extended based on the user token to obtain extended features; the extended features are split to obtain a first split feature; the first split feature is processed by an invertible matrix to obtain the trapdoor query information.

[0060] In this application, users can fully and comprehensively mine the image features of the query image based on the feature extraction device, thereby improving the accuracy of image query.

[0061] Figure 4 This is a schematic diagram of the feature extraction device provided in an embodiment of the present invention, such as... Figure 4 As shown, the feature extraction device includes an object detector, a Contrastive Language-Image Pre-Training (CLIP) model, and a dual attention module.

[0062] The CLIP model uses a Vision Transformer (ViT) encoder to slice the query image into patches of equal size, each patch representing a token. For example, if the query image is 224x224 and the patch size is 16x16, it can generate 224x224 / 16x16 = 196 patches. This means the input sequence length is 196, and each patch has a dimension of 16x16x3 = 768, resulting in 196 tokens, each with a dimension of 768. Adding the trainable parameter cls, the resulting content feature vector IF... C The dimensions are 197x768.

[0063] The CLIP model is a multimodal pre-trained model that achieves cross-modal association between images and natural language text through contrastive learning. It compares the semantic connections between images and text by calculating the scaled pairwise cosine similarity of the feature vectors of text and images, overcoming the limitations of traditional unimodal models. It can accurately capture objects and relationships in images, learn multimodal general semantic representations more flexibly, and perform well in unseen scenes and objects. It can accurately identify the relationship between the text and image of "a dog running on the grass." Furthermore, it has zero-shot transfer learning capabilities; specifically, it does not require object detection and relationship prediction models, but directly establishes a semantic mapping between images and text through contrastive learning. It can complete classification / retrieval tasks directly through natural language prompts without manual labeling. This significantly reduces deployment costs in scenarios with high labeling costs, such as medical imaging. It features a large-scale pre-training paradigm. Compared to traditional supervised learning that relies on closed supervised data with manual labeling, the CLIP model's contrastive learning employs self-supervised learning capabilities, using publicly available image-text pairs from the internet for training, covering open-domain semantic relationships.

[0064] The object detector uses a pre-trained Fast R-CNN model to extract an N×2048 dimensional feature vector. Simultaneously, it uses the content feature vector (IF) generated by the CLIP model. C Perform linear projection to generate a 1×2048 dimensional feature vector. Replace the features corresponding to the maximum bounding box areas of the first k indices in the N×2048 dimensional feature vector with the 1×2048 dimensional feature vector to obtain the fused feature X.

[0065] The encoder of the dual attention module achieves the fusion of dual attention mechanisms by introducing channel attention mechanism and self-attention mechanism, and pays attention to the channel correlation and spatial correlation of image features at the same time, so that the feature extraction device can more comprehensively understand the content of the query image.

[0066] Attention mechanisms originate from human processing capabilities for external information. They are dynamic weighting techniques that assign higher weights to key information by calculating the correlations between features. In deep learning, their core function is to enhance the model's ability to perceive important features. The main types include self-attention and channel attention. Self-attention mechanisms can notice the correlations between different parts of the input. The main idea is to dynamically allocate attention weights to different positions based on their correlations in the sequence. This is typically achieved through the following steps: For a fused feature X, its corresponding query vector, key vector, and value vector can be generated and calculated. The dot product similarity between the query and key is calculated, and after scaling the result, a Softmax function is applied. Then, the weights of the corresponding distributions are calculated, and the weighted sum of the attention distributions is calculated and output. Channel attention aims to explicitly model the correlations between different channels. It automatically acquires the importance of each feature channel through network learning, and finally assigns different weight coefficients to each channel to strengthen important features and suppress unimportant features. This is typically achieved through the following steps: Global average pooling or global max pooling is used to calculate the global feature information of each channel. The global features are processed using fully connected layers or convolutional layers to generate weight coefficients for each channel. These weight coefficients are then applied to the original feature map to obtain a weighted feature map.

[0067] In a dual attention module, each coding layer of the encoder produces an output R. i (i = 1, 2, ..., N), the current encoding layer can pass the original feature vector and the current output to the next encoding layer, embedding prior knowledge for use. Finally, a stacked output R = (R1, R2, ..., RN) is generated. N ), where R i =AddNorm(Uσ(VM) s (R i-1)+d)+e), R i-1 and R i Let U and V represent the input and output vectors of the i-th coding layer, respectively. U and V are learnable weight matrices, d and e are bias terms, and AddNorm represents a combination of staggered linking, layer normalization, and position feedforward.

[0068] Specifically, the features are first preprocessed to facilitate the use of the self-attention mechanism, which can be achieved through Equation 1.

[0069]

[0070] Where X represents the image fusion feature, [a,b] represents the stitching operation, and W... q W k W v Let M be the respective projection matrix. k M v As a learnable memory matrix, the values ​​of k and v can be determined by attention retrieval of prior knowledge that is not yet embedded in the query image.

[0071] Next, max pooling and average pooling operations are performed on the features to obtain two channel descriptions: F max ,F avg , of which F max F represents the max pooling feature. avg This represents the average pooling characteristic. Let F... max ,F avg The features are fed into a shared multilayer perceptron (MLP), and the channel attention is generated based on the features of the channel dimension and the importance of each channel, as shown in Equation 2.

[0072]

[0073] Where σ represents the sigmoid activation function and the weights of the MLP. and For shared input.

[0074] Then different transformation matrices W are used q W k W v and the learnable memory matrix M k M v The results from different heads are concatenated and linearly projected to generate self-attention, and the calculation process is as shown in Equation 3.

[0075]

[0076] Where K, Q, and V are the values ​​of the learned matrix with added memory, and d is the scaling factor. Formula 3 first calculates the inner product of each row vector of matrix Q and K. To prevent the inner product from being too large, it is divided by the square root of d. Then, Softmax is used to normalize the result of the above inner product, and the Softmax matrix is ​​multiplied by V to obtain the final output.

[0077] The output of the channel attention mechanism is fed into the self-attention mechanism for calculation to obtain the output of the coding layer. The calculation process is shown in Equation 4.

[0078]

[0079] in, F represents element-wise multiplication. C Let K′, Q′, V′ represent the three sets of vectors processed by the dual attention mechanism, and F represent the vectors. S This represents the output of each coding layer.

[0080] The dual-attention module's decoder fully utilizes grid-based cross-attention to maximize the output of the encoding layers, constructing a multi-layered structure. Specifically, for a given input vector Y and encoding layer output R, gated cross-attention connects all elements of Y and R, giving full attention to all encoding layers and coordinating global information. In practical applications, a multi-layered structure can be constructed using Equation 5.

[0081]

[0082] Where ⊙ denotes element-wise multiplication, θ denotes the cross-attention of the encoder and decoder, and θ(R) i Y) = Attention(W q Y,W k R i W v R i ), α i It is a weight matrix with the same size as the cross-attention matrix. The weights are freely adjustable, mainly depending on the individual contribution of each coding layer and the relative importance between different coding layers. These data, α, are calculated by measuring the correlation between the cross-attention result of each coding layer and the input vector. i =σ(W i [Y,θ(R i ,Y)]+b i ), [a, b] denotes a connection, σ denotes the sigmoid activation function, and W i Let b represent the weight matrix. i This represents a learnable bias term.

[0083] After the encoder finishes encoding, the output R of the encoding layer and the word embedding vector Y are combined.i As input to the decoder, it is connected to the fully connected layer via mesh cross-attention, thereby obtaining the coding layer weights α. i The query derived from the t-th element of the input sequence Y is concatenated with the key and value obtained from the left subsequence (i.e., Y≤t). Finally, after linear projection and normalization operations, the probabilities of words in the dictionary are encoded to generate image descriptive features. The specific generation process is shown in Equation 6.

[0084] Y i+1 =AddNorm(M mesh (R,AddNorm(S mesk (Y i )))) Formula 6

[0085] Among them, S mesk AddNorm represents a combination of residual connections, layer normalization, and position feedforward, where Y = {Y1, Y2, ..., Y}. N}

[0086] After determining the image description features S corresponding to the image, the model is trained using text data to vectorize S into a text feature vector IF. t For example, the dimension of the sentence vector generated by the text data training model can be set to 300, and finally a text feature vector with a length of

[300] will be generated.

[0087] Specifically, the query image is input into the feature extraction device, which determines the content feature vector (IF) corresponding to the query image based on the CLIP model. c The fusion feature X is determined based on the content feature vector output by the CLIP model and the feature vector output by the object detector. The image description feature S of the query image is determined based on the dual attention module. The text feature vector IF is determined by vectorizing the image description feature S. t Furthermore, the text feature vector IF can be... t and content feature vector IF c The image features identified are those of the query image.

[0088] This application inputs a query image into a feature extraction device, determines the content feature vector based on the CLIP model, extracts feature vectors through an object detector, and fuses the content feature vectors and the feature vectors extracted by the object detector to obtain fused features. These fused features can be used as the first layer input of the encoder in the dual attention module. The dual attention module adds channel attention to self-attention, calculates the output feature vectors after calculating average pooling and max pooling, performs channel rearrangement, and then calculates channel attention. This is combined with self-attention calculation to calculate the final image feature description. Finally, it is vectorized into text to generate text feature vectors, which are then combined with the content feature vectors to form the image feature vector. This approach overcomes the limitations of traditional convolutional neural network models in capturing fine-grained features, effectively capturing fine-grained visual features and high-order semantic features in images, thus enhancing image description.

[0089] The key generation center can randomly generate security parameters. For a single key generation process, different security parameters will result in different keys. That is, key generation is related to security parameters. By randomly generating security parameters, the uniqueness and security of key generation are guaranteed.

[0090] The key generation center can also generate the master key mk and public parameters pk, thus determining mk = (S, U1, U2) and pk = (G1, G2, h, g), where g is a generator of G1. The values ​​are randomly selected, G1×G1→G2, the partitioning matrix S is a randomly generated (2n+1)-dimensional binary vector, and {U1,U2} are two randomly generated (2n+1)×(2n+1)-dimensional invertible binary matrices. Corresponding keys can also be generated for the data owner, query user, and proxy server. For the data owner, u∈Z can be randomly selected. p Generate key (sk) DO =u,pk DO =g u For the querying user, z∈Z can be randomly selected. p Generate key For the proxy server, generate the proxy re-encryption key. It can also generate a unique user token for each querying user to enable multi-user queries. Specifically, a token of length K ≤ n is randomly selected, and a random vector Tok of length K is generated. u ={l1,l2,...,l K}, where the random number satisfies Tok u This refers to the user token used for querying the user.

[0091] By randomly constructing user tokens and dynamically allocating token embedding positions using a pseudo-random function, the query trapdoor information generated by different users for the same query image is not linked, thus solving the problem of cross-privacy leakage in multi-user scenarios and resisting collusion attacks.

[0092] A token-bound encrypted indexing mechanism enables dynamic matching of user permissions with data granularity, achieving fine-grained access control. In practical applications, user access permissions can be controlled through attribute policies; for example, by combining user roles or attributes to generate tokens, even more granular implementation of user access permissions can be achieved.

[0093] In determining the image features Q = {q1, q2, ..., q} of the query image, ... n After that, the image features can be expanded based on the user token to obtain a (2n+1) bit feature vector. Specifically, it can be based on the random scaling factor r≠0 and the user's search token Tok. u ={l1,l2,...,l K The feature vector Q is expanded to obtain the first expanded feature vector. The (2n+1)th term is a constant term of 1, and the other terms are 0. Secondly, the first split feature can be obtained by splitting the first extended feature. Specifically, the splitting matrix S can be used to split the first extended feature... The first splitting feature {Q′, Q″} is defined by the following rules: If S[i] = 0, then... Where τ is a random number, satisfying If S[i] = 1, then Furthermore, the trapdoor query information can be determined by processing the first splitting feature {Q′,Q″} using the invertible matrix {U1,U2}.

[0094] Of course, after determining the trapdoor query information corresponding to the query image, the querying user can send the trapdoor query information to the server so that the server can perform image retrieval based on the trapdoor query information.

[0095] As mentioned earlier, when multiple users perform image queries, trapdoor query information is generated for each user to perform image queries on the query image. This ensures that different users generate different trapdoor query information for the same query image, thus avoiding the problem of cross-privacy leakage in multi-user scenarios.

[0096] In this embodiment of the invention, the server determines to enable image query by receiving the trapdoor query information sent by the querying user.

[0097] Step 320: Determine the similarity between the trapdoor query information and the encrypted index information of each image.

[0098] In one implementation, step 320 may specifically include:

[0099] For each of the encrypted index information, a secure inner product operation is performed on the trapdoor query information and the encrypted index information, and the result of the operation is determined as the similarity corresponding to the encrypted index information.

[0100] The image owner determines the image features as I = {d1, d2, ..., d...} based on the feature extraction device. n After that, the image features can be expanded to obtain a (2n+1) bit feature vector. Specifically, the (n+Y)th of I i +1) The bit is set to a random number ε i By setting the (2n+1)th bit to a random offset t, the second extended feature is obtained. For each dimension, based on the segmentation matrix S, the second extended feature is... The split is divided into the second splitting feature {I′,I″}, with the following specific rules. If S[i]=0, then If S[i] = 1, then Where τ is a random number, satisfying Furthermore, the second splitting feature {I′,I″} can be processed by the invertible matrix {U1,U2} to determine the encrypted index information, i.e., to determine the encrypted index information. Of course, the image owner can upload the encrypted index information corresponding to the image to the server for storage.

[0101] Image owners can also encrypt images using a key to generate encrypted images. Specifically, they can encrypt image m using the key ck. i Encrypting (i∈[1,n]) yields the encrypted image c. i =Encrypt(ck,m i Therefore, the encrypted text CT corresponding to the image can be determined as {C1, C2, C3, CM}, where C1 = pk s =g us C2 = ck·e(g,h) s C3 = g H(ck) CM = {c1, c2, ..., c i}, where s is a secret value randomly generated by the image owner. The image owner sends the ciphertext corresponding to the image to the proxy server. The proxy server can re-encrypt the ciphertext corresponding to the image based on the conversion key tk, thereby determining CT′=(C′1,C2,C3,CM), where, The proxy server can also upload re-encrypted ciphertext to the server for storage.

[0102] By introducing a proxy server, a lightweight cloud-edge-device collaborative retrieval architecture can be built, reducing encryption and decryption computation overhead and achieving efficient retrieval with sublinear time complexity.

[0103] Furthermore, during the re-encryption stage on the proxy server, the hierarchical hash verification code generated by the query user's key and the server key can be used as a two-factor authentication label. When the query user decrypts, the hash chain verification is triggered to verify whether the returned image has been tampered with or whether the key has been replaced. This can ensure the authenticity of results in highly sensitive scenarios such as medical and judicial settings. By combining the two-factor authentication label with the hash chain verification, the one-way trust dependence on the server can be broken.

[0104] Specifically, after receiving the trapdoor query information sent by the querying user, the server can determine the similarity between the trapdoor query information and the encrypted index information corresponding to each image. Specifically, it can use Formula 6 to analyze the similarity between the trapdoor query information T and the encrypted index information corresponding to each image. Q With encrypted index information I ci Perform a safe inner product operation to obtain the result I. ci ·T Q .

[0105]

[0106] Where rQ·I is the original similarity, t represents the redundancy, and t represents the offset.

[0107] Furthermore, the result I can be calculated. ci ·T Q The similarity is determined to be the value corresponding to the encrypted index information.

[0108] In this embodiment of the invention, the similarity corresponding to the encrypted index information is determined by performing a secure inner product operation on the trapdoor query information and the encrypted index information of each image.

[0109] Step 330: Sort the similarities in descending order; determine the encrypted index information corresponding to the top N similarities as the target encrypted index information.

[0110] As mentioned earlier, the similarity scores of all images are sorted in descending order, and the encrypted index information corresponding to the top N similarity scores is determined as the target encrypted index information.

[0111] In this embodiment of the invention, the target encrypted index information is determined based on the similarity corresponding to the encrypted index information, thereby realizing image query based on the encrypted index information of the image.

[0112] Step 340: Determine the encrypted image corresponding to the target encrypted index information as the encrypted query result.

[0113] Specifically, the encrypted image in the encrypted information corresponding to the target encrypted index information stored on the server can be determined as the encrypted query result.

[0114] In practical applications, the encrypted retrieval result CT" = (C′1, C2, C3, CM') can be determined, where CM' is a subset of CM, including the encrypted images in the encrypted information corresponding to the K target encrypted index information.

[0115] Step 350: Send the encrypted query result to the querying user so that the querying user can determine the query result by decrypting the encrypted query result.

[0116] In one implementation, the process by which the querying user decrypts the encrypted query result to determine the query result includes:

[0117] The content key of the querying user is determined based on the private key of the querying user; for each encrypted query result, the content key of the querying user is verified based on the encrypted query result, and the verification result is determined to be either successful or unsuccessful; the encrypted query result is decrypted based on the content key that has been verified, and the query result is obtained.

[0118] Specifically, after receiving the encrypted query result, the user can first generate a content key based on the key, which can then determine the identity of the user. It can also verify the correctness of the content key by executing Formula 7.

[0119] C′3=g H(ck′) Formula 7

[0120] If the C′3 calculated by Formula 7 is consistent with the C3 contained in the encrypted retrieval result, then the verification result of the content key is determined to be successful; otherwise, the verification result of the content key is determined to be unsuccessful.

[0121] Furthermore, the query result can be determined by decrypting the encrypted query result using the content key that passed the verification, thus confirming the query result M' = Dec. ck′ (CM')

[0122] In this embodiment of the invention, after the encrypted query result is sent to the querying user, the querying user can decrypt the encrypted query result to determine the query result, thus realizing the determination of the query result.

[0123] The image query method provided in this invention includes: receiving trapdoor query information sent by a querying user; determining the similarity between the trapdoor query information and the encrypted index information of each image; sorting the similarities in descending order; determining the encrypted index information corresponding to the top N similarities as the target encrypted index information, where N is a natural number; determining the encrypted image corresponding to the target encrypted index information as the encrypted query result; and sending the encrypted query result to the querying user so that the querying user can decrypt the encrypted query result to determine the query result. In this technical solution, after receiving the trapdoor query information sent by the querying user, the server initiates an image query, determines the similarity corresponding to the encrypted index information by performing a secure inner product operation on the trapdoor query information and the encrypted index information of each image, determines the target encrypted index information based on the similarity, and determines the encrypted image in the encrypted information corresponding to the target encrypted index information as the encrypted query result, thus realizing image query based on encrypted index information, and further realizing encrypted image query. After sending the encrypted query result to the querying user, the querying user can decrypt the encrypted query result to determine the query result, thereby realizing the determination of the query result. By assigning a unique user token to each querying user and constructing trapdoor query information based on the user token, it ensures that different querying users generate different trapdoor query information for the same query image. This solves the risk of user behavior association caused by duplicate trapdoors in traditional solutions, guarantees the unlinkability of trapdoors in multi-user scenarios, and avoids cross-privacy leakage issues in multi-user scenarios.

[0124] Furthermore, by extracting image features through a feature extraction device and a dual attention module, the limitations of traditional convolutional neural network models in capturing fine-grained features are overcome, effectively capturing fine-grained visual features and high-order semantic features in images, thereby enhancing image description.

[0125] Figure 5 This is a schematic diagram of an image query device provided in an embodiment of the present invention. This device is applicable to situations requiring encrypted image queries in multi-user scenarios. The device can be implemented through software and / or hardware and is generally integrated into electronic devices, such as computer devices.

[0126] like Figure 5 As shown, the device includes:

[0127] The receiving module 510 is used to receive trapdoor query information sent by the querying user, wherein the trapdoor query information is determined by the querying user based on the query image and the user token;

[0128] The determining module 520 is used to determine the similarity between the trapdoor query information and the encrypted index information of each image, and to determine the target encrypted index information based on the similarity.

[0129] The query module 530 is used to determine the encrypted query result based on the target encrypted index information, and send the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

[0130] The image query device provided in this embodiment receives trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token; determines the similarity between the trapdoor query information and the encrypted index information of each image, determines target encrypted index information based on the similarity, determines an encrypted query result based on the target encrypted index information, and sends the encrypted query result to the querying user so that the querying user can decrypt the encrypted query result to determine the query result. In the above technical solution, after the server receives the trapdoor query information sent by the querying user, it initiates an image query, determines the target encrypted index information based on the similarity between the trapdoor query information and the encrypted index information of each image, and determines the encrypted image corresponding to the target encrypted index information as the encrypted query result, thus realizing image query based on encrypted index information, thereby realizing encrypted image query. After sending the encrypted query result to the querying user, the querying user can decrypt the encrypted query result to determine the query result, thus realizing the determination of the query result. By assigning a unique user token to each querying user and constructing trapdoor query information based on the user token, it ensures that different querying users generate different trapdoor query information for the same query image. This solves the risk of user behavior association caused by duplicate trapdoors in traditional solutions, guarantees the unlinkability of trapdoors in multi-user scenarios, and avoids cross-privacy leakage issues in multi-user scenarios.

[0131] In one embodiment, the process by which the querying user determines the trapdoor query information based on the query image includes: determining the image features of the query image; performing feature expansion on the image features based on the user token to obtain extended features; performing feature splitting on the extended features to obtain a first split feature; and processing the first split feature using an invertible matrix to obtain the trapdoor query information.

[0132] Based on the above embodiments, the determining module 520 is specifically used for: performing a secure inner product operation on the trapdoor query information and the encrypted index information for each of the encrypted index information, and determining the operation result as the similarity corresponding to the encrypted index information; sorting each of the similarities in descending order; and determining the encrypted index information corresponding to the top N similarities as the target encrypted index information, where N is a natural number.

[0133] Based on the above embodiments, the query module 530 is specifically used to: determine the encrypted image corresponding to the target encrypted index information as the encrypted query result; and send the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

[0134] In one embodiment, the process by which the querying user decrypts the encrypted query result to determine the query result includes: determining the querying user's content key based on the querying user's private key; for each encrypted query result, verifying the querying user's content key based on the encrypted query result, and determining whether the verification result is successful or unsuccessful; and decrypting the encrypted query result based on the content key with a successful verification result to obtain the query result.

[0135] The image query device provided in the embodiments of the present invention can execute the image query method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image query method.

[0136] It is worth noting that in the above-described embodiments of the image query device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0137] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 6 A block diagram is shown of an exemplary electronic device 6 suitable for implementing embodiments of the present invention. Figure 6 The electronic device 6 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0138] like Figure 6 As shown, electronic device 6 is represented in the form of a general-purpose computing electronic device. The components of electronic device 6 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0139] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0140] Electronic device 6 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 6, including volatile and non-volatile media, removable and non-removable media.

[0141] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 6 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0142] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0143] Electronic device 6 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 6, and / or with any device that enables electronic device 6 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 6 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 6 As shown, network adapter 20 communicates with other modules of electronic device 6 via bus 18. It should be understood that, although... Figure 6 Not shown, it can be combined with electronic device 6 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0144] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing the image query method provided in this embodiment of the invention. The method includes: receiving trap query information sent by a query user, wherein the trap query information is determined by the query user based on a query image and a user token; determining the similarity between the trap query information and the encrypted index information of each image, and determining target encrypted index information based on the similarity; determining an encrypted query result based on the target encrypted index information, and sending the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

[0145] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the image query method provided in any embodiment of the present invention.

[0146] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements, for example, the image query method provided in this invention. The method includes: receiving trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token; determining the similarity between the trapdoor query information and encrypted index information of each image; determining target encrypted index information based on the similarity values; determining an encrypted query result based on the target encrypted index information; and sending the encrypted query result to the querying user so that the querying user can determine the query result by decrypting the encrypted query result.

[0147] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0148] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0149] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0150] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0152] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant laws and regulations.

[0153] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An image query method, characterized in that, include: Receive trapdoor query information sent by the querying user, wherein the trapdoor query information is determined by the querying user based on the query image and the user token; Determine the similarity between the trapdoor query information and the encrypted index information of each image, and determine the target encrypted index information based on the similarity. The encrypted query result is determined based on the target encrypted index information, and the encrypted query result is sent to the query user so that the query user can determine the query result by decrypting the encrypted query result.

2. The image query method according to claim 1, characterized in that, The process by which the querying user determines the trapdoor query information based on the query image and the user token includes: Determine the image features of the query image; Based on the user token, the image features are extended to obtain extended features, and the extended features are split to obtain the first split features; The trapdoor query information is obtained by processing the first splitting feature using an invertible matrix.

3. The image query method according to claim 1, characterized in that, Determining the similarity between the trapdoor query information and the encrypted index information of each image includes: For each of the encrypted index information, a secure inner product operation is performed on the trapdoor query information and the encrypted index information, and the result of the operation is determined as the similarity corresponding to the encrypted index information.

4. The image query method according to claim 1, characterized in that, The target encrypted index information is determined based on the aforementioned similarities, including: The similarities are sorted in descending order. The encrypted index information corresponding to the top N similarities is determined as the target encrypted index information, where N is a natural number.

5. The image query method according to claim 1, characterized in that, Determining the encrypted query result based on the target encrypted index information includes: The encrypted image corresponding to the target encrypted index information is determined as the encrypted query result.

6. The image query method according to claim 1, characterized in that, The process by which the querying user decrypts the encrypted query result to determine the query result includes: The content key of the querying user is determined based on the querying user's private key; The encrypted query result is decrypted based on the content key to obtain the query result.

7. The image query method according to claim 6, characterized in that, After determining the content key of the querying user based on the querying user's private key, the process further includes: For each of the encrypted query results, the content key of the querying user is verified based on the encrypted query result to determine whether the verification result is successful or unsuccessful.

8. An image query device, characterized in that, include: A receiving module is used to receive trapdoor query information sent by a querying user, wherein the trapdoor query information is determined by the querying user based on a query image and a user token; The determining module is used to determine the similarity between the trapdoor query information and the encrypted index information of each image, and to determine the target encrypted index information based on the similarity. The query module is used to determine the encrypted query result based on the target encrypted index information, and send the encrypted query result to the query user so that the query user can determine the query result by decrypting the encrypted query result.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image query method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the image query method as described in any one of claims 1-7.

Images