An image storage method, device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide an image storage method and device, electronic equipment and storage medium, which are applied to the technical field of images. The method of the embodiments of the present application can encode a to-be-stored image through an encoder of a deep learning model, classify the first image feature after encoding through a classifier, decode the first latent space feature after sampling through a decoder, and obtain a first decoded image. A second loss value is calculated according to the first decoded image and the to-be-stored image, and a first loss value is calculated according to the first classification result and the extraction code. The parameters of the deep learning model are adjusted through the loss values to realize the training of the deep learning model. The present application realizes the storage of the to-be-stored image through the training of the deep learning model, and stores the first encoding information after encoding the to-be-stored image in the latent space. The storage device cannot directly view the to-be-stored image, and the image can be read only by using the extraction code, thereby improving the security of image storage.

Description

Technical Field

[0001] This application relates to the field of image technology, and in particular to an image storage method, apparatus, electronic device and storage medium. Background Technology

[0002] In the information age, the sheer volume of data necessitates substantial storage capacity from storage devices. For instance, as people demand higher pixel quality from images, the image resolution increases, consequently requiring greater storage capacity. However, storage devices have limited space; therefore, the increased number of images puts immense pressure on them.

[0003] To reduce the storage space occupied by images, image storage technologies typically employ compression tools to compress and encode the images to be stored. For example, online compression tools can be used to compress images to obtain compressed versions. However, anyone can use a compression tool to decompress a compressed image, thus posing a risk of image leakage when using commonly used compression tools for image compression and storage. Summary of the Invention

[0004] The purpose of this application is to provide an image storage method, apparatus, electronic device, and storage medium to improve the security of image storage. The specific technical solution is as follows:

[0005] A first aspect of this application provides an image storage method, the method comprising:

[0006] Obtain the image to be stored and its extraction code;

[0007] The image to be stored is encoded using an encoder of a deep learning model to obtain first encoded information, and the first encoded information is stored in a latent space, wherein the first encoded information includes first image features;

[0008] The classifier of the deep learning model is used to classify and predict the first image features to obtain a first classification result;

[0009] Based on the extraction code and the first classification result, calculate the first loss value of the deep learning model;

[0010] The potential space is sampled to obtain the first potential space features;

[0011] The deep learning model's decoder is used to decode the extracted code and the first latent spatial features to obtain a first decoded image;

[0012] The second loss value of the deep learning model is calculated based on the first decoded image and the image to be stored;

[0013] The parameters of the deep learning model are adjusted based on the first loss value and the second loss value.

[0014] In one possible implementation, sampling the latent space to obtain first latent space features includes:

[0015] The potential space is sampled to obtain first sampled feature information, and the mean and variance of each element in the first sampled feature information are calculated.

[0016] Obtain a preset matrix, calculate the product of the variance and the preset matrix to obtain a first matrix, wherein the preset matrix follows a normal distribution;

[0017] The mean value is added to each element in the first matrix to obtain the first latent space feature.

[0018] In one possible implementation, the encoder of the deep learning model includes an image mapping module and an encoding module.

[0019] The step of encoding the image to be stored using an encoder based on a deep learning model to obtain first encoded information, and storing the first encoded information in a latent space, includes:

[0020] The image mapping module is used to perform a convolution operation on the image to be stored to obtain a first encoded image feature; wherein, the first encoded image feature includes the image features after convolution of each image block of the image to be stored;

[0021] The encoding module is used to encode the features of the first encoded image to obtain the first encoded information, and the first encoded information is stored in the latent space.

[0022] In one possible implementation, the encoding module includes multiple encoding sub-modules, which are sequentially connected in an ordered manner; each encoding sub-module includes a first normalization network, a first multi-head attention mechanism network, a second normalization network, and a first feedforward neural network. The step of using the encoding module to encode the first encoded image features to obtain the first encoded information, and storing the first encoded information in the latent space, includes:

[0023] For each encoding submodule, the first normalization network of the encoding submodule is used to normalize the input image features to be encoded to obtain the second encoded image features. When the encoding submodule is the first encoding submodule, the image features to be encoded are the first encoded image features; otherwise, the image features to be encoded are the seventh encoded image features output by the previous encoding submodule.

[0024] Using the first multi-head attention mechanism network of the encoding submodule, feature extraction and encoding are performed on the second encoded image features to obtain the third encoded image features;

[0025] The third encoded image feature of the encoding submodule and the image feature to be encoded are residually joined to obtain the fourth encoded image feature;

[0026] The second normalization network of this encoding submodule is used to normalize the input fourth encoded image features to obtain the fifth encoded image features.

[0027] The first feedforward neural network of this coding submodule is used to enhance the features of the input fifth coded image to obtain the sixth coded image features;

[0028] The sixth coded image feature and the fourth coded image feature are residually concatenated to obtain the seventh coded image feature; wherein, the first image feature is the seventh coded image feature output by the last coded submodule, and the first coded information includes at least the first image feature.

[0029] In one possible implementation, the first feedforward neural network includes a first fully connected subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second fully connected subnetwork, and a second regularization subnetwork.

[0030] The step of using the first feedforward neural network to enhance the input fifth encoded image features to obtain the sixth encoded image features includes:

[0031] The first fully connected sub-network is used to perform a linear transformation on the fifth encoded image features to obtain the first encoded image sub-features;

[0032] Using the first activation function sub-network, the first encoded image sub-features are nonlinearly transformed to obtain the second encoded image sub-features;

[0033] The second encoded image sub-features are regularized using the first regularization sub-network to obtain the third encoded image sub-features;

[0034] Using the second fully connected sub-network, a linear transformation is performed on the third encoded image sub-feature to obtain the fourth encoded image sub-feature;

[0035] The fourth encoded image sub-feature is regularized using the second regularization sub-network to obtain the sixth encoded image feature.

[0036] In one possible implementation, the decoder of the deep learning model includes a decoding module and an image generation module.

[0037] The step of decoding the extracted code and the first latent spatial features using the decoder of the deep learning model to obtain the first decoded image includes:

[0038] The decoding module is used to decode the extraction code and the first latent spatial feature to obtain the second image feature; wherein the second image feature includes the image features after decoding multiple image blocks;

[0039] Using the image generation module, the image blocks in the second image features are arranged and combined to generate the first decoded image.

[0040] In one possible implementation, the decoding module includes multiple decoding sub-modules, which are connected sequentially in an ordered manner; each decoding sub-module includes a third normalization network, a first fully connected network, a masked multi-head attention mechanism network, a second fully connected network, and a second feedforward neural network.

[0041] The step of using the decoding module to decode the extracted code and the first latent spatial features to obtain the second image features includes:

[0042] For each decoding submodule, the third normalization network of the decoding submodule is used to normalize the feature to be decoded to obtain the first decoded image feature; wherein, when the decoding submodule is the first decoding submodule, the feature to be decoded is the first latent space feature; otherwise, the feature to be decoded is the seventh decoded image feature output by the previous decoding submodule.

[0043] Using the first fully connected network of the decoding submodule, the first decoded image features are linearly transformed to obtain the second decoded image features;

[0044] The masked multi-head attention mechanism network of the decoding submodule is used to decode the second decoded image features to obtain the third decoded image features;

[0045] Using the second fully connected network of the decoding submodule, the third decoded image features are linearly transformed to obtain the fourth decoded image features;

[0046] The feature to be decoded is residually concatenated with the fourth decoded image feature to obtain the fifth decoded image feature;

[0047] The second feedforward neural network of the decoding submodule is used to enhance the features of the fifth decoded image to obtain the features of the sixth decoded image.

[0048] The fifth decoded image feature is residually concatenated with the sixth decoded image feature to obtain the seventh decoded image feature; wherein, the second image feature is the seventh decoded image feature output by the last decoding submodule.

[0049] In one possible implementation, adjusting the parameters of the deep learning model based on the first loss value and the second loss value includes:

[0050] Based on the first loss value, adjust the parameters of the encoder and classifier in the deep learning model;

[0051] Based on the second loss value, adjust the parameters of the encoder and decoder in the deep learning model.

[0052] In one possible implementation, the method further includes:

[0053] During the image reading process, the extraction code is obtained;

[0054] The extraction code is input into the deep learning model to obtain the read image.

[0055] In a second aspect of this application, an image storage device is provided, the device comprising:

[0056] An image acquisition module is used to acquire the image to be stored and its extraction code.

[0057] An image encoding module is used to encode the image to be stored using an encoder of a deep learning model to obtain first encoding information, and to store the first encoding information in a latent space, wherein the first encoding information includes first image features;

[0058] The image classification module is used to classify and predict the first image features using the classifier of the deep learning model to obtain a first classification result;

[0059] The first loss value calculation module is used to calculate the first loss value of the deep learning model based on the extraction code and the first classification result.

[0060] A sampling module is used to sample the latent space to obtain first latent space features;

[0061] An image decoding module is used to decode the extraction code and the first latent spatial features using the decoder of the deep learning model to obtain a first decoded image;

[0062] The second loss value calculation module is used to calculate the second loss value of the deep learning model based on the first decoded image and the image to be stored.

[0063] The parameter adjustment module is used to adjust the parameters of the deep learning model based on the first loss value and the second loss value.

[0064] In one possible implementation, the sampling module includes:

[0065] The sampling calculation submodule is specifically used to sample the potential space to obtain first sampling feature information, and to calculate the mean and variance of each element in the first sampling feature information;

[0066] The matrix multiplication submodule is specifically used to obtain a preset matrix, calculate the product of the variance and the preset matrix to obtain a first matrix, wherein the preset matrix follows a normal distribution;

[0067] The mean superposition submodule is specifically used to add the mean to each element in the first matrix to obtain the first latent spatial features.

[0068] In one possible implementation, the encoder of the deep learning model includes an image mapping module and an encoding module.

[0069] The image encoding module includes:

[0070] The image convolution submodule is specifically used to perform a convolution operation on the image to be stored using the image mapping module to obtain a first encoded image feature; wherein, the first encoded image feature includes the image features after convolution of each image block of the image to be stored;

[0071] The image encoding submodule is specifically used to encode the features of the first encoded image using the encoding module to obtain the first encoded information, and to store the first encoded information in the latent space.

[0072] In one possible implementation, the encoding module includes multiple encoding sub-modules, which are sequentially connected in an ordered manner; each encoding sub-module includes a first normalization network, a first multi-head attention mechanism network, a second normalization network, and a first feedforward neural network, wherein the image encoding sub-module includes:

[0073] The first normalization unit is specifically used to normalize the input image features to be encoded using the first normalization network of the encoding submodule for each encoding submodule, so as to obtain the second encoded image features. When the encoding submodule is the first encoding submodule, the image features to be encoded are the first encoded image features; otherwise, the image features to be encoded are the seventh encoded image features output by the previous encoding submodule.

[0074] The encoding unit is specifically used to extract and encode the second encoded image features using the first multi-head attention mechanism network of the encoding submodule to obtain the third encoded image features.

[0075] The first residual connection unit is specifically used to perform residual connection between the third coded image feature of the coding submodule and the image feature to be encoded to obtain the fourth coded image feature;

[0076] The second normalization unit is specifically used to normalize the input fourth coded image features using the second normalization network of the encoding submodule to obtain the fifth coded image features.

[0077] The feature enhancement unit is specifically used to enhance the features of the input fifth encoded image using the first feedforward neural network of the encoding submodule to obtain the sixth encoded image features;

[0078] The second residual connection unit is specifically used to perform residual connection on the sixth coded image feature and the fourth coded image feature to obtain the seventh coded image feature; wherein, the first image feature is the seventh coded image feature output by the last coded submodule, and the first coded information includes at least the first image feature.

[0079] In one possible implementation, the first feedforward neural network includes a first fully connected subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second fully connected subnetwork, and a second regularization subnetwork.

[0080] The feature enhancement unit includes:

[0081] The first linear transformation subunit is specifically used to perform a linear transformation on the fifth coded image features through the first fully connected subnetwork to obtain the first coded image sub-features;

[0082] The first nonlinear transformation subunit is specifically used to perform a nonlinear transformation on the first encoded image sub-features using the first activation function sub-network to obtain the second encoded image sub-features.

[0083] The first regularization processing subunit is specifically used to perform regularization processing on the second coded image sub-features using the first regularization sub-network to obtain the third coded image sub-features;

[0084] The second linear transformation subunit is specifically used to perform a linear transformation on the third encoded image sub-feature using the second fully connected sub-network to obtain the fourth encoded image sub-feature.

[0085] The second regularization processing subunit is specifically used to perform regularization processing on the fourth coded image sub-features through the second regularization sub-network to obtain the sixth coded image features.

[0086] In one possible implementation, the decoder of the deep learning model includes a decoding module and an image generation module.

[0087] The image decoding module includes:

[0088] The image decoding submodule is specifically used to decode the extraction code and the first latent spatial features using the decoding module to obtain the second image features; wherein, the second image features include image features after decoding multiple image blocks;

[0089] The image generation submodule is specifically used to arrange and combine the image blocks in the second image features using the image generation module to generate the first decoded image.

[0090] In one possible implementation, the decoding module includes multiple decoding sub-modules, which are connected sequentially in an ordered manner; each decoding sub-module includes a third normalization network, a first fully connected network, a masked multi-head attention mechanism network, a second fully connected network, and a second feedforward neural network.

[0091] The image decoding submodule includes:

[0092] The third normalization unit is specifically used to normalize the features to be decoded for each decoding submodule using the third normalization network of that decoding submodule to obtain the first decoded image features; wherein, when the decoding submodule is the first decoding submodule, the feature to be decoded is the first latent space feature; otherwise, the feature to be decoded is the seventh decoded image feature output by the previous decoding submodule.

[0093] The first linear transformation unit is specifically used to perform a linear transformation on the first decoded image features using the first fully connected network of the decoding submodule to obtain the second decoded image features.

[0094] The image decoding unit is specifically used to decode the second decoded image features using the masked multi-head attention mechanism network of the decoding submodule to obtain the third decoded image features;

[0095] The second linear transformation unit is specifically used to perform a linear transformation on the third decoded image features using the second fully connected network of the decoding submodule to obtain the fourth decoded image features.

[0096] The third residual connection unit is specifically used to perform a residual connection between the feature to be decoded and the fourth decoded image feature to obtain the fifth decoded image feature;

[0097] The second feature enhancement unit is specifically used to enhance the features of the fifth decoded image using the second feedforward neural network of the decoding submodule to obtain the sixth decoded image features.

[0098] The fourth residual connection unit is specifically used to perform a residual connection between the fifth decoded image feature and the sixth decoded image feature to obtain the seventh decoded image feature; wherein, the second image feature is the seventh decoded image feature output by the last decoding submodule.

[0099] In one possible implementation, the parameter adjustment module includes:

[0100] The first parameter adjustment submodule is specifically used to adjust the parameters of the encoder and classifier in the deep learning model according to the first loss value.

[0101] The second parameter adjustment submodule is specifically used to adjust the parameters of the encoder and decoder in the deep learning model according to the second loss value.

[0102] In one possible implementation, the device further includes:

[0103] The extraction code acquisition module is used to acquire the extraction code during the image reading process;

[0104] The image reading module is used to input the extraction code into the deep learning model to obtain the read image.

[0105] Another aspect of the embodiments of this application also provides an electronic device, including:

[0106] Memory, used to store computer programs;

[0107] When a processor executes a program stored in memory, it implements any of the image storage methods described in the embodiments of this application.

[0108] In another aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements any of the image storage methods described above.

[0109] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the image storage methods described above.

[0110] Beneficial effects of the embodiments in this application:

[0111] This application provides an image storage method, apparatus, electronic device, and storage medium. The method involves encoding an image to be stored using an encoder in a deep learning model, storing the encoded first information in the latent space of the deep learning model, classifying the encoded first image features using a classifier in the deep learning model to establish a correspondence between an extraction code and the first image features, and decoding the sampled first latent space features using a decoder in the deep learning model to obtain a first decoded image. Then, based on the first decoded image and the image to be stored, a second loss value is calculated; based on the first classification result and the extraction code, a first loss value is calculated; and the parameters of the deep learning model are adjusted using the first and second loss values ​​to train the deep learning model.

[0112] Compared to existing technologies, this application's embodiments achieve image storage by training a deep learning model. The first encoded information of the image is stored in the latent space of the deep learning model, making the image invisible on the storage device, thus improving image storage security. Furthermore, this application's embodiments require inputting an extraction code into the trained deep learning model to obtain the corresponding image, further ensuring the security of image data storage.

[0113] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0114] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0115] Figure 1 This is a schematic flowchart of an image storage method provided in an embodiment of this application;

[0116] Figure 2A schematic diagram illustrating sampling of the potential space provided in an embodiment of this application;

[0117] Figure 3 A schematic diagram of a network structure of an encoder provided in an embodiment of this application;

[0118] Figure 4 A schematic diagram of a network structure for an encoding submodule provided in an embodiment of this application;

[0119] Figure 5 A schematic diagram of a network structure of a first feedforward neural network provided in an embodiment of this application;

[0120] Figure 6 A schematic diagram of a network structure for a decoder provided in an embodiment of this application;

[0121] Figure 7 A schematic diagram of a network structure for a decoding submodule provided in an embodiment of this application;

[0122] Figure 8 A schematic diagram of a network structure for a deep learning model provided in an embodiment of this application;

[0123] Figure 9 This is a schematic diagram of the structure of an image storage device provided in an embodiment of this application;

[0124] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0125] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0126] In image storage technology, to reduce the storage space occupied by images, compression tools are typically used to compress and encode the images to be stored. For example, online compression tools are used to compress images to obtain compressed images. However, anyone can use the same compression tools to decompress and obtain the compressed image. Therefore, using commonly used compression tools to compress and store images carries the risk of image leakage.

[0127] To address at least one of the aforementioned problems, a first aspect of this application provides an image storage method, the method comprising: Figure 1 The steps shown are as follows:

[0128] Step S101: Obtain the image to be stored and its extraction code.

[0129] The image to be stored can be an RGB (Red / Green / Blue) color image or a grayscale image. This application embodiment does not impose restrictions on parameters such as resolution, shape, and size of the image to be stored. The extraction code is used to represent the category of the image to be stored. This category is a unique identifier for the image to be stored and can be the image's ID (identity document) information, name, or text content describing the content of the image to be stored.

[0130] In practical applications, the method of this application embodiment can be implemented by a terminal device, which can be an electronic device such as a computer or server that can be used for training deep learning models.

[0131] Step S102: Use the encoder of the deep learning model to encode the image to be stored to obtain the first encoded information, and store the first encoded information in the latent space.

[0132] The deep learning model is a neural network model that includes modules such as an encoder, classifier, and decoder. In one example, the deep learning model in this embodiment is a transformer (natural language processing) neural network. The encoder in the deep learning model has multiple layers of encoding structures, which are connected in an ordered manner. Each layer of encoding structures encodes the input image to be stored or image features to obtain encoded image features. The first encoding information may include the encoding information output by each layer of encoding structure, or it may only include the encoding information output by some of the layers. The first image feature is the encoding information output by the last layer of encoding structure, and the first encoding information should include at least the first image feature.

[0133] Step S103: Use the classifier of the deep learning model to classify and predict the first image features to obtain the first classification result.

[0134] The first classification result represents the category of the image to be stored. To intuitively represent the probability of the predicted image category, the Softmax function can be used to classify the first image features. The Softmax function exponentially normalizes and normalizes the raw output of the deep learning model, mapping it to probability values, with the sum of the probabilities of each classification result being 1. In practical applications, the classifier can be connected to the last layer of the encoder's coding structure, allowing the first image features to be input into the classifier for classification.

[0135] Step S104: Calculate the first loss value of the deep learning model based on the extraction code and the first classification result.

[0136] The first loss is the image category loss. It can be calculated using the first classification result as the prediction result and the extraction code as the ground truth, through cross-entropy. In one example, the first loss can be calculated using the following formula:

[0137]

[0138] Where C represents the vector dimension of the true label; x represents the input data, x i y represents the i-th element of the real label. i This represents the probability that the deep learning model predicts x belongs to the i-th category.

[0139] In this embodiment, the input data is the first image feature, and the true label is the extraction code of the image to be stored. The smaller the first loss value, the closer the probability distribution of the model's predicted first classification result is to the probability distribution of the actual extraction code, and the higher the accuracy of the deep learning model's classifier prediction.

[0140] Step S105: Sample the latent space to obtain the first latent space features.

[0141] To obtain image information of the image to be stored and to simplify the representation of image features, the latent space can be sampled to obtain low-dimensional features, which facilitates subsequent processing of image feature data. In one example, a VAE (Variational Autoencoder) can be used to sample the latent space. The VAE's encoder maps the image features of the image to be stored to the latent space and calculates the distribution parameters of the latent space. Based on these distribution parameters, random sampling is performed on the latent space to obtain a latent vector, which is the first latent space feature in this embodiment.

[0142] Step S106: Use the decoder of the deep learning model to decode the extracted code and the first latent space features to obtain the first decoded image.

[0143] Similarly, a decoder can also include multiple decoding structures, which are connected sequentially in an ordered manner. The combined effect of these decoding structures restores the first latent spatial features from vectors to an image. In one example, a deep learning model learns the mapping relationship between feature vectors and images, and the first decoded image is reconstructed based on this mapping relationship and the first latent spatial features.

[0144] Step S107: Calculate the second loss value of the deep learning model based on the first decoded image and the image to be stored.

[0145] The second loss is the image loss, which can be calculated by either calculating the mean-squared error (MSE) or the cross-entropy loss. The cross-entropy loss can be calculated in the same way as in step S104, and the mean-squared error can be calculated using the following formula:

[0146] MSE = (A i -A , i ) 2

[0147] Among them, A i Indicates the image to be stored, A , i This represents the first decoded image.

[0148] Step S108: Adjust the parameters of the deep learning model based on the first loss value and the second loss value.

[0149] The first loss value is used to adjust the parameters of the encoder and classifier in the deep learning model; the second loss value is used to adjust the parameters of the encoder and decoder in the deep learning model. In practical applications, the encoder, classifier, and decoder are trained using the backpropagation algorithm. The calculated loss values ​​are used to backpropagate and adjust the network parameters. The adjustment of the deep learning model parameters needs to meet the following requirement: when the image to be stored and its extraction code are input into the adjusted deep learning model, both the recalculated first and second loss values ​​are less than a preset value. This preset value is used to ensure that the difference between the first decoded image and the image to be stored is within a tolerable range.

[0150] In one example, based on the first loss value, the parameters of the encoder and classifier in the deep learning model are adjusted, i.e., the weights and biases of the encoder and classifier are adjusted to reduce the first loss value. Based on the second loss value, the parameters of the encoder and decoder in the deep learning model are adjusted, i.e., the weights and biases of the encoder and decoder are adjusted to reduce the second loss value. This process is iterated until the first loss value is less than a first loss threshold and the second loss value is less than a second loss threshold, thus completing the training of the deep learning model. Here, the weights are used to determine how image feature data propagates in the deep learning model, ensuring that the deep learning model learns an effective representation of the image to be stored; the biases are used to enable the deep learning model to better adapt to the distribution of feature data.

[0151] The method of this application embodiment encodes the image to be stored using an encoder of a deep learning model, stores the encoded first information in the latent space of the deep learning model, classifies the encoded first image features using a classifier of the deep learning model, establishes a correspondence between the extraction code and the first image features, and decodes the sampled first latent space features using a decoder of the deep learning model to obtain a first decoded image. Then, based on the first decoded image and the image to be stored, a second loss value is calculated, and based on the first classification result and the extraction code, a first loss value is calculated. The parameters of the deep learning model are adjusted using the first and second loss values ​​to train the deep learning model. Compared to existing technologies, this application embodiment achieves the storage of the image to be stored by training a deep learning model, and stores the encoded first information of the image to be stored in the latent space of the deep learning model. The image to be stored cannot be directly seen on the storage device, and it is impossible to reconstruct the image by deciphering the latent space, thereby improving the security of image storage. Furthermore, when acquiring an image, this application embodiment requires inputting an extraction code into the trained deep learning model to obtain the image corresponding to the extraction code, further ensuring the security of image data storage.

[0152] In one possible implementation, step S105 can be achieved by sampling the potential space through the following steps:

[0153] Step 1: Sample the potential space to obtain the first sampled feature information, and calculate the mean and variance of each element in the first sampled feature information.

[0154] Taking the first image feature as an example, during sampling, random sampling can be performed based on the distribution of the first image feature, ensuring that the sampled first feature information matches the distribution of the first image feature. Compared to the first image feature, the first sampled feature has a lower dimensionality, thus simplifying the image feature data. In one example, the mean and variance of each element in the first image feature can be calculated to obtain the probability distribution of the first image feature. Random sampling can then be performed based on this probability distribution to obtain the first sampled feature information, and the mean and variance of each element in the first sampled feature information can be calculated.

[0155] Step 2: Obtain the preset matrix, and calculate the product of the variance and the preset matrix to obtain the first matrix.

[0156] The preset matrix follows a normal distribution. For example... Figure 2 As shown, the preset matrix is ​​a vector ε sampled from a standard normal distribution N(0, I) by the autoencoder VAE. By multiplying the variance σ of the first sampled feature information with the preset matrix ε, the generated preset matrix can be associated with the first sampled feature information, which facilitates subsequent data processing.

[0157] Step 3: Add the mean to each element in the first matrix to obtain the first latent space features.

[0158] like Figure 2 As shown, the encoder of the deep learning model stores the first encoded information in the latent space, samples the latent space to obtain the first sampled feature information, and calculates the mean μ and variance σ of each element in the first sampled feature information. ε, which conforms to a normal distribution, is sampled from N(0, I), and the first matrix is ​​obtained through εxσ. Then, the first latent space feature z = μ + εxσ. The first latent space feature is input into the decoder for decoding to obtain the first decoded image.

[0159] The method described in this application allows sampling of the latent space according to a probability distribution. This ensures that the first latent space features, while reducing dimensionality and simplifying the data, are similar to the first encoded information, thus guaranteeing the accuracy of the image data. Furthermore, by sampling a noise ε from a standard normal distribution and then adjusting this noise using μ and σ to obtain the final first latent space features, the method ensures that randomness is introduced into the deep learning model while allowing the gradient of the deep learning model to be backpropagated. This allows the deep learning model to learn and optimize during training, thereby completing the storage of the image data.

[0160] In one possible implementation, the encoder of the deep learning model includes an image mapping module and an encoding module. In step S102, the encoder can encode and store the image to be stored through the following steps:

[0161] Step A: Using the image mapping module, perform a convolution operation on the image to be stored to obtain the first encoded image features.

[0162] The first encoded image feature includes the image features of each image patch of the image to be stored. In practical applications, the image mapping module can divide the image to be stored into image patches, and then flatten each image patch to transform it from a two-dimensional image into a one-dimensional vector, thus mapping the image into a feature vector. The feature vectors of each image patch are concatenated and combined to obtain the first encoded image feature. In practical applications, the segmentation and feature mapping of the image to be stored can be directly achieved through convolution operations.

[0163] Step B: Using the encoding module, the features of the first encoded image are encoded to obtain the first encoded information, and the first encoded information is stored in the latent space.

[0164] When the encoding module encodes the features of the first encoded image, an attention mechanism network can be used to extract important feature information of the first encoded image, thereby achieving an effective representation of the image to be stored. In practical applications, the encoding module includes multiple encoding sub-modules, each of which encodes the features of the first encoded image. The first encoded information can include the image feature information extracted by each encoding sub-module.

[0165] like Figure 3 As shown, the encoder 31 of the deep learning network includes an image mapping module 311 and an encoding module 312. The encoding module includes multiple encoding sub-modules 3121, which are connected sequentially, with the last encoding sub-module connected to the classifier. The image mapping module convolves the image to be stored, transforming it from an image into a first encoded image feature represented by a feature vector, and inputs it into the first encoding sub-module. The first encoding sub-module encodes the first encoded image feature and inputs the encoded image feature into the next encoding sub-module, and so on, until the last encoding sub-module outputs the first image feature. Each encoding sub-module can also store its own output image features in the latent space.

[0166] By applying the method of this application embodiment, the image to be stored can be converted from a two-dimensional image into a vector through the image mapping module, and the first encoded image features can be encoded through the encoding module to obtain the feature information of the image to be stored. Thus, the feature information, i.e. the first encoded information, can be stored in the latent space of the deep learning model, thereby improving the security of image data storage.

[0167] In one possible implementation, the encoding module includes multiple encoding sub-modules, which are connected sequentially in an ordered manner; each encoding sub-module includes a first normalization network, a first multi-head attention mechanism network, a second normalization network, and a first feedforward neural network. Then step B can be implemented through the following steps:

[0168] Step a: For each encoding submodule, the first normalization network of that encoding submodule is used to normalize the input image features to be encoded, thereby obtaining the second encoded image features.

[0169] Wherein, when the encoding submodule is the first encoding submodule, the image feature to be encoded is the first encoded image feature; otherwise, the image feature to be encoded is the seventh encoded image feature output by the previous encoding submodule;

[0170] Step b: Using the first multi-head attention mechanism network of the encoding submodule, feature extraction and encoding are performed on the second encoded image features to obtain the third encoded image features.

[0171] Step c: Perform residual connection between the third encoded image feature and the image feature to be encoded of the encoding submodule to obtain the fourth encoded image feature.

[0172] Step d: Using the second normalization network of this coding submodule, the input fourth coded image features are normalized to obtain the fifth coded image features.

[0173] Step e: Use the first feedforward neural network of the encoding submodule to perform feature enhancement on the input fifth encoded image features to obtain the sixth encoded image features.

[0174] Step f: Perform residual concatenation on the sixth coded image features and the fourth coded image features to obtain the seventh coded image features.

[0175] The first image feature is the seventh encoded image feature output by the last encoded submodule, and the first encoded information includes at least the first image feature.

[0176] like Figure 4 The diagram shows the network structure of one of the encoding submodules within the encoding module. This network structure includes a first normalization network, a first multi-head attention mechanism network, a second normalization network, and a first feedforward neural network. The first and second normalization networks have the same structure, being Layer Norm networks. In one example, the first or second normalization network calculates the mean and standard deviation of each element in the input image features to be encoded or the fourth encoded image features. Based on these mean and standard deviation, each feature value of the image features is normalized. The normalized feature values ​​are then represented using learnable parameters (scaling factor, offset factor) to obtain the normalized second or fifth encoded image features.

[0177] The first multi-head attention mechanism network transforms the input second encoded image features into a vector set of Q (Query), K (Key), and V (Value) through a linear transformation matrix. Based on the Q, K, and V values ​​of each feature value, the attention score of that feature value is calculated. The attention scores are then normalized and weighted to obtain the third encoded image features representing important information of the image.

[0178] The first feedforward neural network (FNN) consists of multiple network layers connected in a weighted manner. Image feature data flows from the input layer to the output layer in a single direction. Each network layer continuously extracts and weights features from the image feature data, thereby achieving feature enhancement and preserving important image feature information. In one example, the feedforward neural network in this embodiment is an MLP (Multilayer Perceptron) Block.

[0179] In practical applications, the encoding submodule also includes a regularization network, which discards image features at a certain ratio to prevent overfitting during deep learning model training. This regularization network can perform regularization using dropout or drop path methods.

[0180] like Figure 4 As shown, if the encoding submodule is the first encoding submodule, the image features input to the first normalization network are the first encoded image features; otherwise, the image features input to the first normalization network are the image features output by the previous encoding submodule, i.e., the seventh encoded image features in this embodiment. The first normalization network normalizes the input image features to be encoded to obtain the second encoded image features; the second encoded image features are then processed by the first multi-head attention mechanism network for feature extraction and encoding to obtain the third encoded image features. In one example, after the second encoded image passes through the first multi-head attention mechanism network, it is also processed by the first regularization network to obtain the third encoded image features. The third encoded image features and the image features to be encoded are then residually connected to obtain the fourth encoded image features. Through residual connections, the image features to be encoded can directly skip the output connections between multiple network layers, thereby avoiding the problems of information loss, gradient vanishing, or gradient exploding during model training.

[0181] The fourth encoded image is normalized by the second normalization network to obtain the fifth encoded image, which is then enhanced by the first feedforward neural network to obtain the sixth encoded image. Similarly, after feature enhancement, the image features can be regularized to obtain the sixth encoded image. The features of the fourth and sixth encoded images are residually concatenated to obtain the seventh encoded image features, which are then input into the next encoding submodule. When this encoding submodule is the last one, its output seventh encoded image features are the features shown in the first image above. The first encoding information may also include the seventh encoded image features output by each encoding submodule.

[0182] By applying the method of this application embodiment, multiple encoding sub-modules are sequentially connected to encode the features of the first encoded image, which can obtain important feature information of the image to be stored, thereby storing the encoded first encoded information. The image to be stored cannot be directly viewed on the storage device, thereby improving the security of image data storage.

[0183] In practical applications, the first feedforward neural network includes a first fully connected sub-network, a first activation function sub-network, a first regularization sub-network, a second fully connected sub-network, and a second regularization sub-network. The first feedforward neural network can then perform feature enhancement through the following steps:

[0184] Step 1: Perform a linear transformation on the fifth encoded image features through the first fully connected sub-network to obtain the first encoded image sub-features.

[0185] Step 2: Using the first activation function sub-network, perform a nonlinear transformation on the first coded image sub-features to obtain the second coded image sub-features.

[0186] Step 3: Use the first regularization sub-network to perform regularization processing on the second encoded image sub-features to obtain the third encoded image sub-features.

[0187] Step 4: Using the second fully connected sub-network, perform a linear transformation on the third encoded image sub-features to obtain the fourth encoded image sub-features.

[0188] Step 5: The fourth encoded image sub-features are regularized using the second regularization sub-network to obtain the sixth encoded image features.

[0189] like Figure 5 As shown, the first or second fully connected sub-network is a linear layer, primarily used to multiply the input image feature vector by the weight matrix and add a bias vector to perform a linear transformation on the fifth or third encoded image sub-features. The first activation function sub-network is an activation function layer, which can be an activation function such as ReLU, GELU, or SiLU. In one example, the activation function is GELU, used to add non-linear elements to the deep learning model, enabling the first activation function sub-network to perform a non-linear transformation on the first encoded image sub-features, allowing the deep learning model to handle non-linear relationships.

[0190] The first or second regularization subnetwork can be regularized using the same or different methods as the first regularization network. In one example, the dropout method is used to regularize the sub-features in the second or fourth encoded image.

[0191] By applying the method of the embodiments of this application, after the fifth encoded image features pass through each sub-network of the first feedforward neural network, the overfitting of the deep learning network can be avoided by multiple regularization networks. The sixth encoded image features after feature enhancement can adapt to more complex linear or nonlinear environments by performing linear and nonlinear transformations through fully connected sub-networks and activation function sub-networks.

[0192] In one possible implementation, the decoder of the deep learning model includes a decoding module and an image generation module.

[0193] The deep learning model can obtain the first decoded image through the following steps:

[0194] Step 1: Using the decoding module, decode the extraction code and the first latent spatial features to obtain the second image features.

[0195] The decoding module decodes the extraction code and, based on the decoded extraction code, retrieves the first latent space feature corresponding to the extraction code from the latent space, thereby decoding the first latent space feature. Since the encoder divides the image to be stored into image blocks and encodes each image block when encoding the image to be stored, the second image feature obtained by the decoding module when decoding the first latent space feature also includes the image features decoded from each image block.

[0196] Step II: Using the image generation module, arrange and combine the image blocks in the second image features to generate the first decoded image.

[0197] When encoding the image to be stored, the encoder's image mapping module maps the image from a two-dimensional image to a feature vector. Similarly, when decoding the first latent spatial features, the decoder's image generation module can reconstruct the first latent spatial features into a first decoded image based on this mapping relationship. In one example, the image features of each image block in the second image features can be reconstructed into an image based on the mapping relationship between the image and the feature vector. Then, based on the positional relationship of each image block, the images corresponding to each image block are stitched together to obtain the first decoded image.

[0198] like Figure 6 As shown, the decoder 61 of the deep learning network includes an image generation module 611 and a decoding module 612. The decoding module includes multiple decoding sub-modules 6121, with each encoding sub-module connected sequentially. The first decoding sub-module decodes the extracted code and the first latent space features, and inputs the output decoded image features into the next decoding sub-module to continue the decoding operation until the last decoding sub-module obtains the output second image features. The image generation module reconstructs the second image features into the first decoded image based on the mapping relationship between the feature vector and the image.

[0199] By applying the method of this application embodiment, the extraction code and the first latent space features can be decoded by the decoding module of the deep learning model to obtain the second image features. The second image features can then be reconstructed into the first decoded image by the image generation module. This achieves the goal of obtaining the image by inputting the extraction code into the deep learning model during the image acquisition process, thereby improving the security of image storage.

[0200] In one possible implementation, the decoding module includes multiple decoding sub-modules, which are connected sequentially in an ordered manner; each decoding sub-module includes a third normalization network, a first fully connected network, a masked multi-head attention mechanism network, a second fully connected network, and a second feedforward neural network.

[0201] The decoding submodule in the decoding module can obtain the second image features through the following steps:

[0202] Step (1): For each decoding submodule, the third normalization network of the decoding submodule is used to normalize the features to be decoded to obtain the first decoded image features.

[0203] Specifically, when the decoding submodule is the first decoding submodule, the feature to be decoded is the first latent space feature; otherwise, the feature to be decoded is the seventh decoded image feature output by the previous decoding submodule.

[0204] Step (2): Using the first fully connected network of the decoding submodule, perform a linear transformation on the first decoded image features to obtain the second decoded image features.

[0205] Step (3): Use the masked multi-head attention mechanism network of the decoding submodule to decode the second decoded image features to obtain the third decoded image features.

[0206] Step (4): Using the second fully connected network of the decoding submodule, perform a linear transformation on the third decoded image features to obtain the fourth decoded image features.

[0207] Step (5): Perform residual connection between the feature to be decoded and the fourth decoded image feature to obtain the fifth decoded image feature.

[0208] Step (6): Use the second feedforward neural network of the decoding submodule to enhance the features of the fifth decoded image to obtain the features of the sixth decoded image.

[0209] Step (7): Perform residual connection between the fifth decoded image feature and the sixth decoded image feature to obtain the seventh decoded image feature; wherein, the second image feature is the seventh decoded image feature output by the last decoding submodule.

[0210] like Figure 7As shown, the third normalization network is an RMSNorm (Root Mean Square Layer Normalization) network layer, which is used to calculate the root mean square of each feature value of the input feature to be decoded, and normalize each feature value according to the root mean square to obtain the first decoded image feature.

[0211] The first fully connected network is a linear layer that performs a linear transformation on the feature values ​​of the first decoded image to obtain the second decoded image features. The masked multi-head attention mechanism network introduces a mask tensor, which can mask or ignore information at certain locations of the second decoded image features, thus focusing only on the effective information. For example... Figure 7 As shown, for each feature value of the second decoded image feature, Q (Query), K (Key), and V (Value) of the feature value are calculated. Q and K are encoded in the manner of RoPE (Rotary Position Embedding). The attention score is calculated in combination with the mask tensor. The weights are normalized by the SoftMax function. The normalized weights are combined with Value and weighted summation is performed to obtain the third decoded image feature output by the mask multi-head attention mechanism network.

[0212] After passing through the second fully connected network (Linear layer), the third decoded image feature is linearly transformed to obtain the fourth decoded image feature. The fourth decoded image feature is then residually connected with the feature to be decoded input to the decoding submodule to obtain the fifth decoded image feature.

[0213] The second feedforward neural network consists of multiple fully connected layers, normalization layers, and activation functions. It performs linear and nonlinear transformations on the input fifth decoded image features to obtain the sixth decoded image features. The fifth decoded image features and the sixth decoded image features are then residually connected to obtain the seventh decoded image features.

[0214] like Figure 7 As shown, the second feedforward neural network includes a first normalization layer (RMSNorm), a first fully connected layer (Linear), a second fully connected layer (Linear), a first activation function layer (SiLU), and a third fully connected layer (Linear). The fifth decoded image features are normalized by the first normalization layer to obtain the first decoded sub-image features. The first and second fully connected layers perform linear transformations on the first decoded sub-image features to obtain the second and third decoded sub-image features, respectively. The first activation function layer performs a non-linear transformation on the third decoded sub-image features to obtain the fourth decoded sub-image features. The fourth decoded sub-image features are superimposed with the second decoded sub-image features to obtain the fifth decoded sub-image features. Finally, the third fully connected layer performs a linear transformation on the fifth decoded sub-image features to obtain the sixth decoded image features.

[0215] By applying the method of this application embodiment, multiple decoding sub-modules can be connected sequentially to decode the first latent spatial features and the extraction code. Through multiple decoding and superposition, the accuracy of generating the first decoded image can be improved.

[0216] In one possible implementation, the image to be read can be obtained through the following steps:

[0217] Step (a): Obtain the extraction code during the image reading process.

[0218] Step (b): Input the extraction code into the deep learning model to obtain the read image.

[0219] After the deep learning model is trained, during image reading, an extraction code is input into the trained deep learning model. Upon receiving the extraction code, the model uses a decoder to decode it. Based on the decoded extraction code, it extracts the first latent space features corresponding to the extraction code from the latent space. Then, through various decoding submodules of the decoder, the first latent space features are decoded to obtain the decoded image features of the image to be read. Finally, the image generation module of the decoder restores the decoded image features into a two-dimensional image, resulting in the read image.

[0220] By applying the method of this application embodiment, during the process of acquiring an image, people need to input an extraction code into the deep learning model. The decoder of the deep learning model can then obtain the first latent spatial feature corresponding to the extraction code, decode the first latent spatial feature, and obtain the read image, thereby improving the security of image storage.

[0221] In one possible implementation, the following examples are used to specifically illustrate the image storage method of the embodiments of this application:

[0222] like Figure 8 As shown, the original image and extraction code are input into the deep learning model, where the original image is the image to be stored. The image mapping module (Embedded Patches) performs a convolution operation on the original image, transforming it into first encoded image features. These first encoded image features are then encoded by multiple encoding sub-modules (Encoder Blocks) to obtain the first image features.

[0223] The first image features are input into the classifier of the deep learning model to obtain the first classification result. This classifier includes AVG (Average Pooling Layer), FC (fully connected layer), and CELows (Cross-Entropy Loss). The first loss value between the first classification result and the extraction code is calculated using the cross-entropy loss function. Based on this first loss value, the parameters of the encoder and classifier of the deep learning model are adjusted by reverse calculation.

[0224] After the encoder obtains image features, it stores these features in the latent space of the deep learning model. The stored image features are the first encoded information, which includes at least the first image features output by the last encoding submodule, and may also include image features output by other encoding submodules. The latent space is sampled using a variational autoencoder to obtain the first sampled features. The mean μ and variance σ of each element in the first sampled feature information are calculated. A preset matrix ε is obtained, which follows a normal distribution N(0, I). The first latent space features are calculated using the formula z = μ + εxσ.

[0225] Each decoding sub-module of the deep learning network decodes the extracted code and the first latent space features to obtain the second image features. These second image features are then reconstructed into the first decoded image by the image generation module (unpachifide). A second loss value is calculated using the cross-entropy loss function CELouss between the first decoded image and the original image. This loss is then used to inversely calculate and adjust the parameters of the encoder and decoder of the deep learning model, ultimately achieving the training of the deep learning model. When the deep learning model training is complete, the original image is also stored in the latent space of the deep learning model in the form of the first encoded information.

[0226] The method of this application embodiment encodes the image to be stored using an encoder of a deep learning model, stores the encoded first information in the latent space of the deep learning model, classifies the encoded first image features using a classifier of the deep learning model, establishes a correspondence between the extraction code and the first image features, and decodes the sampled first latent space features using a decoder of the deep learning model to obtain a first decoded image. Then, based on the first decoded image and the image to be stored, a second loss value is calculated, and based on the first classification result and the extraction code, a first loss value is calculated. The parameters of the deep learning model are adjusted using the first and second loss values ​​to train the deep learning model. Compared to existing technologies, this application embodiment stores the image to be stored by training a deep learning model, and stores the encoded first information of the image to be stored in the latent space of the deep learning model. The image to be stored cannot be directly seen on the storage device, thus improving the security of image storage. Furthermore, when acquiring an image, this application embodiment also requires inputting an extraction code into the trained deep learning model to obtain the image corresponding to the extraction code, further ensuring the security of image data storage.

[0227] In a second aspect of the embodiments of this application, an image storage device is provided, such as Figure 9 As shown, the device includes:

[0228] Image acquisition module 901 is used to acquire the image to be stored and its extraction code.

[0229] The image encoding module 902 is used to encode the image to be stored using the encoder of the deep learning model to obtain first encoding information, and to store the first encoding information in the latent space, wherein the first encoding information includes first image features.

[0230] The image classification module 903 is used to classify and predict the features of the first image using a classifier of a deep learning model to obtain the first classification result.

[0231] The first loss value calculation module 904 is used to calculate the first loss value of the deep learning model based on the extraction code and the first classification result.

[0232] The sampling module 905 is used to sample the latent space to obtain the first latent space features.

[0233] The image decoding module 906 is used to decode the extracted code and the first latent spatial features using the decoder of the deep learning model to obtain the first decoded image.

[0234] The second loss value calculation module 907 is used to calculate the second loss value of the deep learning model based on the first decoded image and the image to be stored.

[0235] The parameter adjustment module 908 is used to adjust the parameters of the deep learning model based on the first loss value and the second loss value.

[0236] In one possible implementation, the sampling module includes:

[0237] The sampling calculation submodule is specifically used to sample the potential space to obtain the first sampling feature information, and to calculate the mean and variance of each element in the first sampling feature information.

[0238] The matrix multiplication submodule is specifically used to obtain a preset matrix, calculate the product of the variance and the preset matrix to obtain a first matrix, wherein the preset matrix follows a normal distribution.

[0239] The mean superposition submodule is specifically used to add the mean to each element in the first matrix to obtain the first latent space features.

[0240] In one possible implementation, the encoder of the deep learning model includes an image mapping module and an encoding module.

[0241] Image encoding module, including:

[0242] The image convolution submodule is specifically used to perform convolution operations on the image to be stored using the image mapping module to obtain the first encoded image features; wherein, the first encoded image features include the image features after convolution of each image block of the image to be stored.

[0243] The image encoding submodule is specifically used to encode the features of the first encoded image using the encoding module to obtain the first encoded information, and to store the first encoded information in the latent space.

[0244] In one possible implementation, the encoding module includes multiple encoding sub-modules, which are sequentially connected in an ordered manner; each encoding sub-module includes a first normalization network, a first multi-head attention mechanism network, a second normalization network, and a first feedforward neural network; the image encoding sub-module includes:

[0245] The first normalization unit is specifically used to normalize the input image features to be encoded using the first normalization network of each encoding submodule to obtain the second encoded image features. When the encoding submodule is the first encoding submodule, the image features to be encoded are the first encoded image features; otherwise, the image features to be encoded are the seventh encoded image features output by the previous encoding submodule.

[0246] The encoding unit is specifically used to extract and encode features from the second encoded image features using the first multi-head attention mechanism network of the encoding submodule, so as to obtain the third encoded image features.

[0247] The first residual connection unit is specifically used to perform residual connection between the third coded image feature and the image feature to be encoded of the coding submodule to obtain the fourth coded image feature.

[0248] The second normalization unit is specifically used to normalize the input fourth coded image features using the second normalization network of the coding submodule to obtain the fifth coded image features.

[0249] The feature enhancement unit is specifically used to enhance the features of the input fifth encoded image using the first feedforward neural network of the encoding submodule to obtain the sixth encoded image features.

[0250] The second residual connection unit is specifically used to perform residual connection on the sixth coded image feature and the fourth coded image feature to obtain the seventh coded image feature; wherein, the first image feature is the seventh coded image feature output by the last coded submodule, and the first coded information includes at least the first image feature.

[0251] In one possible implementation, the first feedforward neural network includes a first fully connected subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second fully connected subnetwork, and a second regularization subnetwork.

[0252] Feature enhancement unit, including:

[0253] The first linear transformation subunit is specifically used to perform a linear transformation on the fifth coded image features through the first fully connected subnetwork to obtain the first coded image sub-features.

[0254] The first nonlinear transformation subunit is specifically used to perform a nonlinear transformation on the first coded image sub-features using the first activation function sub-network to obtain the second coded image sub-features.

[0255] The first regularization processing subunit is specifically used to perform regularization processing on the second coded image sub-features using the first regularization sub-network to obtain the third coded image sub-features.

[0256] The second linear transformation subunit is specifically used to perform a linear transformation on the third coded image sub-features using the second fully connected sub-network to obtain the fourth coded image sub-features.

[0257] The second regularization processing subunit is specifically used to perform regularization processing on the fourth coded image sub-features through the second regularization sub-network to obtain the sixth coded image features.

[0258] In one possible implementation, the decoder of the deep learning model includes a decoding module and an image generation module.

[0259] The image decoding module includes:

[0260] The image decoding submodule is specifically used to decode the extraction code and the first latent spatial features using the decoding module to obtain the second image features; wherein, the second image features include the image features after decoding multiple image blocks.

[0261] The image generation submodule is specifically used to arrange and combine the image blocks in the second image features to generate the first decoded image.

[0262] In one possible implementation, the decoding module includes multiple decoding sub-modules, which are connected sequentially in an ordered manner; each decoding sub-module includes a third normalization network, a first fully connected network, a masked multi-head attention mechanism network, a second fully connected network, and a second feedforward neural network.

[0263] The image decoding submodule includes:

[0264] The third normalization unit is specifically used to normalize the features to be decoded for each decoding submodule using the third normalization network of that decoding submodule to obtain the first decoded image features; wherein, when the decoding submodule is the first decoding submodule, the features to be decoded are the first latent space features; otherwise, the features to be decoded are the seventh decoded image features output by the previous decoding submodule.

[0265] The first linear transformation unit is specifically used to perform a linear transformation on the first decoded image features using the first fully connected network of the decoding submodule to obtain the second decoded image features.

[0266] The image decoding unit is specifically used to decode the second decoded image features using the masked multi-head attention mechanism network of the decoding submodule to obtain the third decoded image features.

[0267] The second linear transformation unit is specifically used to perform a linear transformation on the third decoded image features using the second fully connected network of the decoding submodule to obtain the fourth decoded image features.

[0268] The third residual connection unit is specifically used to perform residual connection between the feature to be decoded and the fourth decoded image feature to obtain the fifth decoded image feature.

[0269] The second feature enhancement unit is specifically used to enhance the features of the fifth decoded image by utilizing the second feedforward neural network of the decoding submodule to obtain the features of the sixth decoded image.

[0270] The fourth residual connection unit is specifically used to perform residual connection between the fifth decoded image feature and the sixth decoded image feature to obtain the seventh decoded image feature; wherein, the second image feature is the seventh decoded image feature output by the last decoding submodule.

[0271] In one possible implementation, the parameter adjustment module includes:

[0272] The first parameter adjustment submodule is specifically used to adjust the parameters of the encoder and classifier in the deep learning model based on the first loss value.

[0273] The second parameter adjustment submodule is specifically used to adjust the parameters of the encoder and decoder in the deep learning model based on the second loss value.

[0274] In one possible implementation, the device further includes:

[0275] The extraction code acquisition module is used to obtain the extraction code during the image reading process.

[0276] The image reading module is used to input the extraction code into the deep learning model to obtain the read image.

[0277] The apparatus of this application embodiment can encode the image to be stored using the encoder of a deep learning model, store the encoded first encoded information in the latent space of the deep learning model, classify the encoded first image features using the classifier of the deep learning model, establish a correspondence between the extraction code and the first image features, and decode the sampled first latent space features using the decoder of the deep learning model to obtain a first decoded image. Then, based on the first decoded image and the image to be stored, a second loss value is calculated, and based on the first classification result and the extraction code, a first loss value is calculated. The parameters of the deep learning model are adjusted using the first and second loss values ​​to train the deep learning model. Compared to the prior art, this application embodiment achieves the storage of the image to be stored by training a deep learning model, and stores the encoded first encoded information of the image to be stored in the latent space of the deep learning model. The image to be stored cannot be directly seen on the storage device, thereby improving the security of image storage. Furthermore, when acquiring an image, this application embodiment also requires inputting the extraction code into the trained deep learning model to obtain the image corresponding to the extraction code, further ensuring the security of image data storage.

[0278] This application also provides an electronic device, such as... Figure 10 As shown, it includes:

[0279] Memory 1001 is used to store computer programs;

[0280] When processor 1002 executes a program stored in memory 1001, it performs the following steps:

[0281] Obtain the image to be stored and its extraction code;

[0282] The encoder of the deep learning model encodes the image to be stored to obtain first encoded information, and stores the first encoded information in the latent space, wherein the first encoded information includes first image features;

[0283] The first image features are classified and predicted using a classifier based on a deep learning model to obtain the first classification result;

[0284] Calculate the first loss value of the deep learning model based on the extraction code and the first classification result;

[0285] The first latent space features are obtained by sampling the latent space;

[0286] The first decoded image is obtained by decoding the extracted code and the first latent space features using the decoder of the deep learning model.

[0287] Calculate the second loss value of the deep learning model based on the first decoded image and the image to be stored.

[0288] The parameters of the deep learning model are adjusted based on the first and second loss values.

[0289] Furthermore, the aforementioned electronic devices may also include a communication bus and / or a communication interface, with the processor xx2, the communication interface, and the memory xx1 communicating with each other via the communication bus.

[0290] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0291] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0292] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0293] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0294] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image storage methods.

[0295] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image storage methods described above.

[0296] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0297] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0298] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for apparatus, electronic devices, and computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0299] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image storage method characterized by, The method comprises: acquiring a to-be-stored image and an extraction code of the to-be-stored image; encoding the to-be-stored image by using an encoder of a deep learning model to obtain first encoding information, and storing the first encoding information to a latent space, wherein the first encoding information comprises first image features; classifying and predicting the first image features by using a classifier of the deep learning model to obtain a first classification result; calculating a first loss value of the deep learning model according to the extraction code and the first classification result; sampling the latent space to obtain first latent space features; decoding the extraction code and the first latent space features by using a decoder of the deep learning model to obtain a first decoded image; calculating a second loss value of the deep learning model according to the first decoded image and the to-be-stored image; adjusting parameters of the encoder and the classifier in the deep learning model according to the first loss value; adjusting parameters of the encoder and the decoder in the deep learning model according to the second loss value.

2. The method of claim 1, wherein, The sampling of the latent space to obtain first latent space features comprises: sampling the latent space to obtain first sampling feature information, and calculating the mean and variance of each element in the first sampling feature information; acquiring a preset matrix, calculating the product of the variance and the preset matrix to obtain a first matrix, wherein the preset matrix is subject to a normal distribution; adding the mean to each element in the first matrix to obtain the first latent space features.

3. The method of claim 1, wherein, The encoder of the deep learning model comprises an image mapping module and an encoding module, The encoding of the to-be-stored image by using the encoder of the deep learning model to obtain the first encoding information, and the storing of the first encoding information to the latent space, comprises: convolving the to-be-stored image by using the image mapping module to obtain first encoding image features; wherein the first encoding image features comprise image features of each image block of the to-be-stored image after convolution; encoding the first encoding image features by using the encoding module to obtain the first encoding information, and storing the first encoding information to the latent space.

4. The method of claim 3, wherein, The encoding module comprises a plurality of encoding sub-modules, and each encoding sub-module is sequentially connected; each encoding sub-module comprises a first normalization network, a first multi-head attention mechanism network, a second normalization network and a first feedforward neural network; the encoding of the first encoding image features by using the encoding module to obtain the first encoding information, and the storing of the first encoding information to the latent space, comprises: for each encoding sub-module, normalizing the input to-be-encoded image features by using the first normalization network of the encoding sub-module to obtain second encoding image features, wherein when the encoding sub-module is the first encoding sub-module, the to-be-encoded image features are the first encoding image features; otherwise, the to-be-encoded image features are the seventh encoding image features output by the previous encoding sub-module. The first multi-head attention mechanism network of the encoding submodule is used for feature extraction and encoding of the second encoded image feature, to obtain a third encoded image feature; The third encoded image feature of the encoding submodule and the to-be-encoded image feature are connected in residual, to obtain a fourth encoded image feature; The second normalization network of the encoding submodule is used for normalization processing of the input fourth encoded image feature, to obtain a fifth encoded image feature; The first feedforward neural network of the encoding submodule is used for feature enhancement of the input fifth encoded image feature, to obtain a sixth encoded image feature; The sixth encoded image feature and the fourth encoded image feature are connected in residual, to obtain the seventh encoded image feature; wherein the first image feature is the seventh encoded image feature output by the last encoding submodule, and the first encoding information at least includes the first image feature.

5. The method of claim 4, wherein, The first feedforward neural network includes a first full connection subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second full connection subnetwork, and a second regularization subnetwork, The first feedforward neural network includes a first full connection subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second full connection subnetwork, and a second regularization subnetwork, The first feedforward neural network includes a first full connection subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second full connection subnetwork, and a second regularization subnetwork, The first feedforward neural network includes a first full connection subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second full connection subnetwork, and a second regularization subnetwork, The first feedforward neural network includes a first full connection subnetwork, a first activation function subnetwork, a first regularization subnetwork, a second full connection subnetwork, and a second regularization subnetwork, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, 6. The method of claim 1, wherein, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, 7. The method of claim 6, wherein, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, The decoder of the deep learning model includes a decoding module and an image generation module, For each decoding submodule, the third normalization network of the decoding submodule is used to normalize the to-be-decoded feature to obtain a first decoded image feature; when the decoding submodule is the first decoding submodule, the to-be-decoded feature is the first latent space feature; otherwise, the to-be-decoded feature is the seventh decoded image feature output by the previous decoding submodule; The first fully connected network of the decoding submodule is used to linearly transform the first decoded image feature to obtain a second decoded image feature; The mask multi-head attention mechanism network of the decoding submodule is used to decode the second decoded image feature to obtain a third decoded image feature; The second fully connected network of the decoding submodule is used to linearly transform the third decoded image feature to obtain a fourth decoded image feature; The to-be-decoded feature and the fourth decoded image feature are connected in residual to obtain a fifth decoded image feature; The second feedforward neural network of the decoding submodule is used to enhance the fifth decoded image feature to obtain a sixth decoded image feature; The fifth decoded image feature and the sixth decoded image feature are connected in residual to obtain the seventh decoded image feature; the second image feature is the seventh decoded image feature output by the last decoding submodule.

8. The method of claim 1, wherein, The method further comprises: In the image reading process, the extraction code is obtained; The extraction code is input into the deep learning model to obtain the read image.

9. An image storage apparatus characterized by comprising: The device comprises: An image acquisition module is configured to acquire a to-be-stored image and an extraction code of the to-be-stored image; An image encoding module is configured to encode the to-be-stored image by using an encoder of a deep learning model to obtain first encoding information, and store the first encoding information into a latent space, wherein the first encoding information comprises a first image feature; An image classification module is configured to classify and predict the first image feature by using a classifier of the deep learning model to obtain a first classification result; A first loss value calculation module is configured to calculate a first loss value of the deep learning model according to the extraction code and the first classification result; A sampling module is configured to sample the latent space to obtain a first latent space feature; An image decoding module is configured to decode the extraction code and the first latent space feature by using a decoder of the deep learning model to obtain a first decoded image; A second loss value calculation module is configured to calculate a second loss value of the deep learning model according to the first decoded image and the to-be-stored image; A first parameter adjustment submodule is configured to adjust parameters of the encoder and the classifier in the deep learning model according to the first loss value; A second parameter adjustment submodule is configured to adjust parameters of the encoder and the decoder in the deep learning model according to the second loss value.

10. An electronic device, comprising: It comprises: A memory is configured to store a computer program; A processor is configured to execute the program stored in the memory to implement the method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-8.

Images